WORKING WITH THE SUSE 2.6.x and 3.x KERNEL SOURCES

Andreas Gruenbacher <agruen@suse.de>, SUSE Labs, 2003, 2004, 2005, 2006

Michal Marek <mmarek@suse.de>, SUSE Labs, 2010

This document gives an overview of how SUSE Linux kernels are

created, and describes tasks like building individual kernels

and creating external kernel modules.

A companion HOWTO that describes how to build driver update disks (among

other things) is available at:

    http://developer.novell.com/wiki/index.php/Creating_a_Driver_Update_Disk_%28DUD%29

TABLE OF CONTENTS

  Overview

  Compiling your own kernel

  Building additional (external) modules

  Supported vs. unsupported modules

  Patch selection mechanism

  Where to find configuration files

  How to configure the kernel sources

  Module load paths

OVERVIEW

The kernels for SUSE are generated from the vanilla Linux kernel sources

found at http://ftp.kernel.org, on top of which a number of patches are

applied. The resulting kernel source tree is configured and built,

resulting in a binary kernel.

The add-on patches and configuration files are maintained in

a GIT repository at

    https://github.com/openSUSE/kernel-source

A script (scripts/tar-up.sh) packs up the files in the repository in a

form suitable for rpmbuild. When building the RPM packages, the

following binary packages get created:

  *  kernel-source

     The kernel source tree, generated by unpacking the vanilla kernel

     sources and applying the patches. The kernel sources are used by

     a number of other packages. They can also be used for compiling

     additional kernel modules.

  *  kernel-devel

     A subset of the kernel-source package that is needed to build

     external modules.

  *  kernel-$FLAVOR

     A number of binary kernels (for example, kernel-default for

     uniprocessor machines, kernel-smp for smp machines, etc.). These

     packages are all generated from the same kernel sources, and

     differ in the kernel configurations used.

  *  kernel-$FLAVOR-base

     A subset of kernel-$FLAVOR, for use in paravirtualized quests that

     only require a few device and filesystem drivers.

  *  kernel-$FLAVOR-devel

     The files used for generating kernel module packages for use with

     kernel-$FLAVOR.

  *  kernel-syms

     A meta package that pulls in the relevant kernel-$FLAVOR-devel

     packages for a given architecture.

The repository contains the configuration files (.config) for all SUSE

kernel flavors. All configuration files are included in the dependencies

of the kernel-syms package (see WHERE TO FIND CONFIGURATION FILES below).

In the installed system, the kernel-source package installs files in the

following directories:

  *  /usr/src/linux-$VERSION-$RELEASE/

     The kernel sources.

  *  /usr/src/linux

     A symbolic link to /usr/src/linux-$VERSION-$RELEASE.

  *  /usr/share/doc/packages/kernel-source/

     This document.

The kernel-$FLAVOR-devel packages install the following files:

  *  /usr/src/linux-$VERSION-$RELEASE-obj/$ARCH/$FLAVOR/

     Kernel build object files for one kernel flavor. These

     files are used for compiling additional kernel modules.

  *  /usr/src/linux-obj/$ARCH/$FLAVOR

     A symbolic link to /usr/src/linux-$VERSION-$RELEASE-obj/$ARCH/$FLAVOR.

COMPILING YOUR OWN KERNEL

The kernel sources are found in the kernel-source package. The

recommended way to produce a binary kernel is:

  (1)  Install kernel-source. Change to the /usr/src/linux directory.

  (2)  Create a build directory for use in configuring and building

       the kernel. Using /usr/src/linux directly requires root priviledges

       and will cause problems if you need to build kernel modules for

       other installed kernels.

  (3)  Configure the kernel (for example, ``make -C /usr/src/linux

       O=$(pwd) oldconfig'', see HOW TO CONFIGURE THE KERNEL SOURCES).

  (4)  Build the kernel and all its modules (``make'').

  (5)  Make sure that /etc/modprobe.d/unsupported-modules contains

           allow_unsupported_modules 1

       otherwise modprobe will refuse to load any modules.

  (6)  Install the kernel and the modules (``make modules_install'',

       followed by ``make install''). This will automatically create

       an initrd for the new kernel as well (see ``mkinitrd -h'').

  (7)  Add the kernel to the boot manager. When using lilo, run ``lilo''

       to update the boot map.

Instead of building binary kernels by hand, you can also build

one of the kernel-$FLAVOR packages using RPM.

BUILDING ADDITIONAL (EXTERNAL) MODULES

A single binary kernel module generally only works for a specific

version of the kernel source tree, for a specific architecture and

configuration. This means that for each binary kernel that SUSE ships, a

custom module must be built. This requirement is to some extent relaxed

by the modversion mechanism: modversions attach a checksum to each

symbol (function or variable) exported to modules by the kernel. This

allows to use kernel modules that have been built for a kernel with a

different version or release number in many cases, as long as none of

the symbols the module uses have changed between the two kernel

versions.

When releasing maintenance or security update kernels for a specific

product, we carefully try to keep the kernel ABI stable.  Despite this,

we sometimes have no choice but to break binary compatibility. In this

case, those kernel modules must be rebuilt.

Additional kernel modules for one of the SUSE kernel flavors can be

built in three different ways:

  (1) by configuring the kernel sources in a separate build directory

      (see HOW TO CONFIGURE THE KERNEL SOURCES), or

  (2) by using one of the standard configurations in

      /usr/src/linux-obj/$ARCH/$FLAVOR, or

  (3) by creating a Kernel Module Package (KMP) as described in the

      Kernel Module Packages Manual, https://drivers.suse.com/doc/kmpm/.

The first method involves the following steps:

  (1)  Install the kernel-source package.

  (2)  Configure the kernel, see HOW TO CONFIGURE THE KERNEL SOURCES.

  (3)  Create files required for compiling external modules:

       ``make prepare'' and ``make scripts''.

  (4)  Compile the module(s) by changing into the module source directory

       and typing ``make -C $(your_build_dir) M=$(pwd)''.

  (5)  Install the module(s) by typing

       ``make -C $(your_build_dir) M=$(pwd) modules_install''.

The second method involves the following steps:

  (1)  Install the kernel-devel package.

  (2)  Install the kernel-$FLAVOR-devel package. This is necessary for

       symbol version information (CONFIG_MODVERSIONS).

  (3)  Compile the module(s) by changing into the module source directory

       and typing ``make -C /usr/src/linux-obj/$ARCH/$FLAVOR M=$(pwd)''.

       Substitute $ARCH and $FLAVOR with the architecture and flavor

       for which to build the module(s).

       If the installed kernel sources match the running kernel, you

       can build modules for the running kernel by using the path

       /lib/modules/$(uname -r)/build as the -C option in the above

       command. (build is a symlink to /usr/src/linux-obj/$ARCH/$FLAVOR).

  (4)  Install the module(s) with

       ``make -C /usr/src/linux-obj/$ARCH/$FLAVOR M=$(pwd) modules_install''.

Whenever building modules, please use the kernel build infrastructure as

much as possible, and do not try to circumvent it. The

Documentation/kbuild directory in the kernel sources documents kbuild

makefiles.

SUPPORTED VS. UNSUPPORTED MODULES

As an extension to the mainline kernel, modules can be tagged as

supported (directly by SUSE, or indirectly by a third party) or

unsupported. Modules which are known to be flakey or for which SUSE does

not have the necessary expertise are marked as unsupported.  Modules for

which SUSE has third-party support agreements are marked as externally

supported. Modules for which SUSE provides direct support are marked as

supported.

The support status of a module can be queried with the modinfo tool.

Modinfo will report one of the following:

  - direct support by SUSE: "supported: yes"

  - third-party support: "supported: external"

  - unsupported modules: no supported tag.

At runtime, the setting of the" unsupported" kernel command line

parameter and /proc/sys/kernel/unsupported determines whether

unsupported modules can be loaded or not, and whether or not loading an

unsupported module causes a warning in the system log:

  0 = only allow supported modules,

  1 = warn when loading unsupported modules,

  2 = don't warn.

Irrespective of this setting, loading an externally supported or unsupported

module both set a kernel taint flag. The taint flags are included in

Oopses. The taint status of the kernel can be inspected in

/proc/sys/kernel/tainted: Bits 0 to 4 have the following meanings:

  bit  0 = a module with a GPL-incompatible license was loaded (tainted & 1),

  bit  1 = module load was enforced (tainted & 2),

  bit  2 = an SMP-unsafe module was loaded (tainted & 4),

  bit  3 = (reserved),

  bit  4 = an unsupported module was loaded (tainted & 16),

  bit  5 = a module with third-party support was loaded (tainted & 32).

  bit 10 = a machine check exception has occurred (taint & 1024; x86_64 only

	   so far).

The corresponding codes for the taint flags in Oopses are (x = unknown):

  - "Pxxx" if bit 0 set or else

    "Gxxx" if bit 0 unset,

  - "xFxx" if bit 1 set or else

    "x xx" if bit 1 unset,

  - "xxSx" if set or else

    "xx x" if bit 2 unset,

  - "xxxU" if bit 4 set or else

    "xxxX" if bit 5 set or else

    "xxx ".

By default, out-of-tree modules will not have the supported flag (that

is, they will be marked as unsupported). For building externally

supported modules, please get in touch with Solid Driver Program team,

led by Scott Bahling, at <soliddriver@suse.com>.

PATCH SELECTION MECHANISM

The SUSE kernels consist of the vanilla kernel sources on top of which a

number of patches is applied. The file series.conf determines which

patches are applied and which are excluded. A script named "guards"

converts series.conf into a plain list of patch files to be applied.

Guards decides which patches to include and exclude based on a list of

symbols. From the kernel-source.src.rpm package, a fully patched

kernel source tree can be generated from vanilla sources + patches like

this:

    # Install the package:

      $ rpm -i kernel-source.src.rpm

    # Unpack the patches and the kernel sources:

      $ cd /usr/src/packages/SOURCES

      $ for f in patches.*.tar.bz2; do		\

	  tar -xjf "$f" || break;		\

	done

      $ tar -xjf linux-2.6.5.tar.bz2

    # Apply the patches

      $ for p in $(./guards < series.conf); do

	  patch -d linux-2.6.5 -p1 < $p || break

	done

The configuration script config.conf which is similar to series.conf is

used for configuration file selection. See the section WHERE TO FIND

CONFIGURATION FILES.

The file format of series.conf and config.conf should be obvious from

the comments in series.conf, and from the guards(1) manual page. (The

guards(1) manual page can be generated by running pod2man on the guards

script.)

WHERE TO FIND CONFIGURATION FILES

Kernel configuration files are stored in the kernel GIT repository. When

packing up the repository, they end up in config.tar.bz2.

The kernel-$FLAVOR packages are based on config/$ARCH/$FLAVOR.

(kernel-default is based on config/$ARCH/default, for example). The

kernel-$FLAVOR packages install their configuration files as

/boot/config-$VER_STR (for example, /boot/config-2.6.5-99-default). The

config is also packaged in the kernel-$FLAVOR-devel package as

/usr/src/linux-obj/$ARCH/$FLAVOR/.config.

In addition, the running kernel exposes a gzip compressed version of its

configuration file as /proc/config.gz. The kernel sources can be

configured based on /proc/config.gz with ``make silentoldconfig''.

HOW TO CONFIGURE THE KERNEL SOURCES

Before a binary kernel is built or an additional loadable module

for an existing kernel is created, the kernel must be configured.

In order for a loadable module to work with an existing kernel, it must

be created with a configuration that is identical to the kernel's

configuration, or at least very close to that. Each configuration is

contained in a single file. The kernel-syms package installs

configurations for all standard SUSE kernel variants, so for building

only external kernel modules it is not necessary to configure the kernel

sources.

Configuring the kernel sources for a specific configuration is

straightfoward:

  - Locate the configuration file you want to use. (See WHERE TO FIND

    CONFIGURATION FILES above).

  - Copy the configuration to the file .config in your build directory.

  - Run the following commands in sequence to apply the configuration,

    generate version information files, etc.:

        make -C /usr/src/linux O=$PWD clean

        make -C /usr/src/linux O=$PWD oldconfig

    Alternatively to ``make oldconfig'', you can also use ``make

    menuconfig'' for a text menu oriented user interface. If the kernel

    sources do not match the configuration file exactly, ``make

    oldconfig'' will prompt for settings that are undefined. Once this

    step is completed, a Makefile will have been created that eliminates

    the need to specify the locations of the kernel source and the build

    directory.

    For configuring the kernel to match the running kernel, you can use

    the file /proc/config.gz, unpack it into into .config, and then run

    ``make oldconfig''.

HOW TO ADD CUSTOM PATCHES

Typically patches are added to the appropriate patches.* directory (e.g.

patches.fixes) and to series.conf. When the kernel-source package is

exported from the git repository, the patch will be automatically added

to the appropriate patch tarball.

If your goal is to create a kernel with only a few additional patches

and you don't want to be bothered with using the git repository, there

is an easier way.

The kernel-source SRPM ships with two empty archives that can be be

filled and automatically expanded when building the kernel. You can use

these to add your own patches and config options without disturbing the

rest of the kernel package. This is useful if you are using the openSUSE

Build Service and link to the main kernel-source project instead of

creating your own branch. The advantage to this is that your project

will automatically receive all the changes that go into the main project

without any further effort.

To add a patch using this mechanism, just add it to the

patches.addon.tar.bz2 archive and add an entry to the series.conf file.

The archive will be expanded automatically with the other kernel patches

when the source tree is constructed.

Some patches may add new Kconfig options. The config.addon.tar.bz2

archive contains the same hierarchy as config.tar.bz2, but is under

config.addon. You can add your new config options to files named after

their config/ counterparts.

For example, the file used to configure the i386 default kernel is named

config/i386/default. To add config options to that kernel, you would

create a new file called config.addon/i386/default with the options as

formatted in a normal Linux kernel .config file. This is important

because the kernel build is non-interactive and will fail if it

encounters new config options without entries in the config file.

MODULE LOAD PATHS

Modules that belong to a specific kernel release are installed in

/lib/modules/2.6.5-99-smp and similar. Note that this path contains the

kernel package release number. Modules from KMPs must be installed

below /lib/modules/2.6.5-99-smp/updates/ and similar: modules below

updates/ have priority over other modules.

When KMPs contain modules that are compatible between multiple installed

kernels, symlinks are used to make those modules available to those

compatible kernels like this:

  /lib/modules/2.6.16-100-smp/weak-updates/foo.ko ->

      /lib/modules/2.6.16-99-smp/updates/foo.ko

Modules in the weak-updates directory have lower priority than modules

in /lib/modules/2.6.16-100-smp/updates/, and higher priority than other

modules in /lib/modules/2.6.16-100-smp.

REFERENCES

General

  Documentation in the kernel source tree.

  Linux Documentation Project, http://www.tldp.org/

  Linux Weekly News, http://lwn.net

  Rusty's Remarkably Unreliable Guides (Kernel Hacking

    and Kernel Locking guides),

    http://www.netfilter.org/unreliable-guides/

  Kernel newbies, http://www.kernelnewbies.org/

Loadable Kernel Modules

  Peter Jay Salzman and Ori Pomerantz: Linux Kernel Module

    Programming Guide, Version 2.4, April 2003,

    http://www.tldp.org/guides.html

Kernel Module Packages

    Andreas Gruenbacher: Kernel Module Packages Manual.

    Versions for CODE9 (SLES9, SUSE LINUX 10.0) and CODE10

    (SUSE Linux 10.1, SLES10),

    https://drivers.suse.com/doc/kmpm/