| Working with the SUSE Kernel Sources |
| ==================================== |
| |
| This document gives an overview of how SUSE Linux kernels are created, and |
| describes tasks like building individual kernels and creating external kernel |
| modules. |
| |
| |
| Overview |
| -------- |
| |
| The SUSE kernels are generated from the upstream Linux kernel sources found at |
| <https://kernel.org/>, on top of which a number of patches are applied. The |
| expanded kernel source tree is configured and built, resulting in a binary |
| kernel. |
| |
| The add-on patches and configuration files are maintained in a Git repository at |
| <https://github.com/SUSE/kernel-source>, with an alternative mirror at |
| <https://github.com/openSUSE/kernel-source>. |
| |
| A script named `scripts/tar-up.sh` packs up the files in the repository in a |
| form suitable for rpmbuild. When building the RPM packages, the following |
| binary packages get created: |
| |
| * kernel-source |
| |
| The kernel source tree, generated by unpacking the vanilla kernel sources and |
| applying the patches. The kernel sources are used by a number of other |
| packages. They can also be used for compiling additional kernel modules. |
| |
| * kernel-devel |
| |
| A complement of the kernel-source package that is needed to build external |
| modules. |
| |
| * kernel-`$FLAVOR` |
| |
| A number of binary kernels, for example, kernel-default for standard use, |
| kernel-debug with extra debugging features, etc. These packages are all |
| generated from the same kernel sources, and differ in the kernel |
| configurations used. |
| |
| * kernel-`$FLAVOR`-base |
| |
| A subset of kernel-`$FLAVOR`, for use in virtualized guests that only require |
| a few device and filesystem drivers. |
| |
| * kernel-`$FLAVOR`-devel |
| |
| The files used for generating kernel module packages for use with |
| kernel-`$FLAVOR`. |
| |
| * kernel-syms |
| |
| A meta package that pulls in the relevant kernel-`$FLAVOR`-devel packages for |
| a given architecture. |
| |
| The repository contains configuration files (`.config`) for all SUSE kernel |
| flavors. All configuration files are included in the dependencies of the |
| kernel-syms package, see [Where to find configuration |
| files]( |
| |
| In the installed system, the kernel-source package installs files in the |
| following directories: |
| |
| * `/usr/src/linux-$VERSION-$RELEASE/` |
| |
| The kernel sources. |
| |
| * `/usr/src/linux` |
| |
| A symbolic link to `/usr/src/linux-$VERSION-$RELEASE/`. |
| |
| * `/usr/share/doc/packages/kernel-source-$VERSION-$RELEASE/` |
| |
| This document. |
| |
| The kernel-`$FLAVOR`-devel packages install the following files: |
| |
| * `/usr/src/linux-$VERSION-$RELEASE-obj/$ARCH/$FLAVOR/` |
| |
| Kernel build object files for one kernel flavor. These files are used for |
| compiling additional kernel modules. |
| |
| * `/usr/src/linux-obj/$ARCH/$FLAVOR` |
| |
| A symbolic link to `/usr/src/linux-$VERSION-$RELEASE-obj/$ARCH/$FLAVOR/`. |
| |
| |
| Compiling your own kernel |
| ------------------------- |
| |
| The kernel sources are found in the kernel-source package. A binary kernel can |
| be built from these sources as follows: |
| |
| 1. Install the kernel-source package. |
| |
| 2. Create a build directory for use in configuring and building the kernel. |
| Using `/usr/src/linux/` directly requires root privileges and will cause |
| problems if you need to build kernel modules for other installed kernels. |
| |
| 3. Configure the kernel, see [How to configure the kernel |
| sources]( |
| `make -C /usr/src/linux O=$PWD oldconfig`. |
| |
| 4. Build the kernel and all its modules by invoking `make`. |
| |
| 5. Make sure that `/etc/modprobe.d/10-unsupported-modules.conf` contains |
| |
| allow_unsupported_modules 1 |
| |
| otherwise modprobe will refuse to load any modules. |
| |
| 6. Install the modules and kernel: `make modules_install`, followed by |
| `make install`. This automatically creates an initrd for the new kernel and |
| adds an entry to the GRUB boot menu. |
| |
| Instead of building binary kernels by hand, you can also build one of the |
| kernel-`$FLAVOR` packages using RPM. |
| |
| |
| Building additional (external) modules |
| -------------------------------------- |
| |
| A single binary kernel module generally only works for a specific version of the |
| kernel source tree, for a specific architecture and configuration. This means |
| that for each binary kernel that SUSE ships, a custom module must be built. This |
| requirement is to some extent relaxed by the modversion mechanism: modversions |
| attach a checksum to each symbol (function or variable) exported to modules by |
| the kernel. This allows to use kernel modules that have been built for a kernel |
| with a different version or release number in many cases, as long as none of the |
| symbols the module uses have changed between the two kernel versions. |
| |
| When releasing maintenance or security update kernels for a specific product, we |
| carefully try to keep the kernel ABI stable. Despite this, we sometimes have no |
| choice but to break binary compatibility. In this case, those kernel modules |
| must be rebuilt. |
| |
| Additional kernel modules for one of the SUSE kernel flavors can be built in the |
| following ways: |
| |
| 1. by doing an ad-hoc module build, using one of the standard configurations in |
| `/usr/src/linux-obj/$ARCH/$FLAVOR`, or |
| |
| 2. by creating a Kernel Module Package (KMP) as described in the Kernel Module |
| Packages Manual, |
| <https://documentation.suse.com/sbp/all/html/SBP-KMP-Manual-SLE12SP2/index.html>. |
| |
| The first method involves the following steps: |
| |
| 1. Install the kernel-devel package. |
| |
| 2. Install the kernel-`$FLAVOR`-devel package. This is necessary for symbol |
| version information (`CONFIG_MODVERSIONS`). |
| |
| 3. Compile the module(s) by changing into the module source directory and typing |
| `make -C /usr/src/linux-obj/$ARCH/$FLAVOR M=$PWD`. Substitute `$ARCH` and |
| `$FLAVOR` with the architecture and flavor for which to build the module(s). |
| |
| If the installed kernel sources match the running kernel, you can build |
| modules for the running kernel by using the path |
| `/lib/modules/$(uname -r)/build` as the `-C` option in the above command. |
| (`build` is a symlink to `/usr/src/linux-obj/$ARCH/$FLAVOR`.) |
| |
| 4. Install the module(s) with |
| `make -C /usr/src/linux-obj/$ARCH/$FLAVOR M=$PWD modules_install`. |
| |
| Whenever building modules, please use the kernel build infrastructure as much as |
| possible, and do not try to circumvent it. The `Documentation/kbuild` directory |
| in the kernel sources documents kbuild makefiles. |
| |
| |
| Supported vs. unsupported modules |
| --------------------------------- |
| |
| As an extension to the mainline kernel, modules can be tagged as supported |
| (directly by SUSE, or indirectly by a third party) or unsupported. Modules which |
| are known to be flakey or for which SUSE does not have the necessary expertise |
| are marked as unsupported. Modules for which SUSE has third-party support |
| agreements are marked as externally supported. Modules for which SUSE provides |
| direct support are marked as supported. |
| |
| The support status of a module can be queried with the modinfo tool. Modinfo |
| will report one of the following: |
| |
| * direct support by SUSE: "supported: yes", |
| * third-party support: "supported: external", |
| * unsupported modules: no supported tag. |
| |
| At runtime, the setting of the "unsupported" kernel command line parameter and |
| `/proc/sys/kernel/unsupported` determines whether unsupported modules can be |
| loaded or not, and whether or not loading an unsupported module causes a warning |
| in the system log: |
| |
| * 0 = only allow supported modules, |
| * 1 = warn when loading unsupported modules, |
| * 2 = don't warn. |
| |
| Irrespective of this setting, loading an externally supported or unsupported |
| module both set a kernel taint flag. The taint flags are included in Oopses. The |
| taint status of the kernel can be inspected in `/proc/sys/kernel/tainted`. |
| Relevant bits have the following meaning: |
| |
| | Bit | Log | Number | Reason that got the kernel tainted | |
| |----:|----:|-----------:|----------------------------------------------------| |
| | 0 | G/P | 1 | proprietary module was loaded | |
| | 12 | ␣/O | 4096 | externally-built ("out-of-tree") module was loaded | |
| | 13 | ␣/E | 8192 | unsigned module was loaded | |
| | 16 | ␣/X | 65536 | module with third-party support was loaded | |
| | 31 | ␣/N | 2147483648 | unsupported module was loaded | |
| |
| Bits 16 and 31 are specific to the SUSE kernels. |
| |
| Out-of-tree modules do not have the supported flag set by default; that |
| is, they are marked as unsupported. For building externally supported |
| modules, please get in touch with the Solid Driver Program team, led by |
| Scott Bahling, at <mailto:soliddriver@suse.com>. |
| |
| |
| Patch selection mechanism |
| ------------------------- |
| |
| The SUSE kernels consist of the upstream kernel sources on top of which a number |
| of patches is applied. The file `series.conf` determines which patches are |
| applied and which are excluded. A script named `guards` converts `series.conf` |
| into a plain list of patch files to be applied. Guards decides which patches to |
| include and exclude based on a list of symbols. From the kernel-source.src.rpm |
| package, a fully patched kernel source tree can be generated from vanilla |
| sources + patches like this: |
| |
| 1. Install the package: |
| |
| # rpm -i kernel-source-$RPMVERSION-$RPMRELEASE.src.rpm |
| |
| 2. Unpack the patches and the kernel sources: |
| |
| # cd /usr/src/packages/SOURCES/ |
| # for f in patches.*.tar.bz2; do \ |
| tar -xjf "$f" || break; \ |
| done |
| # tar -xJf linux-$SRCVERSION.tar.xz |
| |
| 3. Apply the patches |
| |
| # for p in $(./guards < series.conf); do \ |
| patch -d linux-$SRCVERSION -p1 < $p || break; \ |
| done |
| |
| The configuration script `config.conf` which is similar to `series.conf` is |
| used for configuration file selection, see the section [Where to find |
| configuration files](#where-to-find-configuration-files). |
| |
| The file format of `series.conf` and `config.conf` should be apparent from the |
| comments in `series.conf` and from the guards(1) manual page. You can generate |
| this page by running `pod2man` on the `guards` script. |
| |
| |
| Where to find configuration files |
| --------------------------------- |
| |
| Kernel configuration files are stored in the kernel Git repository. When packing |
| up the repository, they end up in `config.tar.bz2`. |
| |
| The kernel-`$FLAVOR` packages are based on `config/$ARCH/$FLAVOR`, for instance, |
| kernel-default is based on `config/$ARCH/default`. The kernel-`$FLAVOR` packages |
| install their configuration files as `/boot/config-$VERSION-$RELEASE-$FLAVOR`. |
| The config is also packaged in the kernel-`$FLAVOR`-devel package as |
| `/usr/src/linux-obj/$ARCH/$FLAVOR/.config`. |
| |
| In addition, the running kernel exposes a gzip compressed version of its |
| configuration file as `/proc/config.gz`. |
| |
| |
| How to configure the kernel sources |
| ----------------------------------- |
| |
| Before a binary kernel is built or an additional loadable module for an existing |
| kernel is created, the kernel must be configured. |
| |
| In order for a loadable module to work with an existing kernel, it must be |
| created with a configuration that is identical to the kernel's configuration, or |
| at least very close to that. Each configuration is contained in a single file. |
| The kernel-syms package installs configurations for all standard SUSE kernel |
| variants, so for building only external kernel modules it is not necessary to |
| configure the kernel sources. |
| |
| Configuring the kernel sources for a specific configuration is straightforward: |
| |
| * Locate the configuration file you want to use, see [Where to find |
| configuration files]( |
| |
| * Copy the configuration to the file `.config` in your build directory. |
| |
| * Run the following commands in sequence to apply the configuration, generate |
| version information files, etc.: |
| |
| $ make -C /usr/src/linux O=$PWD clean |
| $ make -C /usr/src/linux O=$PWD oldconfig |
| |
| If the kernel sources do not match the configuration file exactly, |
| `make oldconfig` will prompt for settings that are undefined. When no |
| `.config` file is initially copied to the target build directory, the command |
| automatically uses `/boot/config-$(uname -r)` as the starting configuration. |
| Alternatively to `make oldconfig`, you can also use `make menuconfig` for |
| a text menu oriented user interface. |
| |
| Once this step is completed, a `Makefile` will have been created that |
| eliminates the need to specify the locations of the kernel source and the |
| build directory. |
| |
| * Update the configuration appropriately for the target use. |
| |
| Configuration files for SUSE kernels include settings to integrate with |
| signing support provided by the Open Build Service. When using such a |
| configuration file directly, the build might fail due to missing files needed |
| for signing kernel modules. |
| |
| The minimal steps to enable module signing and have the kernel build |
| automatically generate a new key pair are as follows: |
| |
| $ ./source/scripts/config --enable CONFIG_MODULE_SIG \ |
| --enable CONFIG_MODULE_SIG_ALL --undefine CONFIG_MODULE_SIG_KEY |
| $ make olddefconfig |
| |
| Please refer to the upstream documentation located at |
| `/usr/src/linux/Documentation/admin-guide/module-signing.rst` for complete |
| information on how to configure module signing. |
| |
| |
| How to add custom patches |
| ------------------------- |
| |
| Patches are typically added to the `patches.suse/` directory and an appropriate |
| place in `series.conf`. When the kernel-source package is exported from the Git |
| repository, the patch will be automatically added to the matching patch tarball. |
| |
| If your goal is to create a kernel with only a few additional patches and you |
| don't want to be bothered with using the Git repository, there is an easier way. |
| |
| The kernel-source SRPM ships with two empty archives that can be filled and |
| automatically expanded when building the kernel. You can use these to add your |
| own patches and config options without disturbing the rest of the kernel |
| package. This is useful if you are using the openSUSE Build Service and link to |
| the main kernel-source project instead of creating your own branch. The |
| advantage to this is that your project will automatically receive all the |
| changes that go into the main project without any further effort. |
| |
| To add a patch using this mechanism, just add it to the `patches.addon.tar.bz2` |
| archive and add an entry to a `series` file inside the archive. The archive will |
| be expanded automatically after the other kernel patches when the source tree is |
| constructed. |
| |
| Some patches may add new Kconfig options. The `config.addon.tar.bz2` archive |
| contains the same hierarchy as `config.tar.bz2`, but is under `config.addon/`. |
| You can add your new config options to files named after their `config/` |
| counterparts. |
| |
| For example, the file used to configure the x86_64 default kernel is named |
| `config/x86_64/default`. To add config options to that kernel, you would create |
| a new file called `config.addon/x86_64/default` with the options as formatted in |
| a normal Linux kernel `.config` file. This is important because the kernel build |
| is non-interactive and will fail if it encounters new config options without |
| entries in the config file. |
| |
| |
| Module load paths |
| ----------------- |
| |
| Modules that belong to a specific kernel release are installed in |
| `/lib/modules/$VERSION-$RELEASE-$FLAVOR/`. Modules from KMPs must be installed |
| below `/lib/modules/$VERSION-$RELEASE-$FLAVOR/updates/` and similar: modules |
| below `updates/` have priority over other modules. |
| |
| When KMPs contain modules that are compatible between multiple installed |
| kernels, symlinks are used to make those modules available to those compatible |
| kernels like this: |
| |
| `/lib/modules/$VERSION-$OTHER_RELEASE-$FLAVOR/weak-updates/foo.ko` → |
| `/lib/modules/$VERSION-$RELEASE-$FLAVOR/updates/foo.ko` |
| |
| Modules in the `weak-updates/` directory have lower priority than modules in |
| `/lib/modules/$VERSION-$OTHER_RELEASE-$FLAVOR/updates/`, and higher priority |
| than other modules in `/lib/modules/$VERSION-$OTHER_RELEASE-$FLAVOR/`. |
| |
| |
| Driver update disks |
| ------------------- |
| |
| A Driver Update Disk (DUD) is an update archive which makes it possible to use |
| new device drivers to run installation of an (open)SUSE distribution. It allows |
| to install the distribution on devices that were not supported at the time the |
| distribution was created and be able to boot the installed system afterwards |
| without having to manually install the new device drivers after the |
| installation. |
| |
| For information how to create such an update, refer to the mkdud and mksusecd |
| tools, and their documentation: |
| |
| * <https://github.com/openSUSE/mkdud>, |
| * <https://github.com/openSUSE/mksusecd>. |
| |
| |
| References |
| ---------- |
| |
| General: |
| |
| * Documentation in the kernel source tree, |
| * LWN.net (Linux Weekly News), <https://lwn.net/>, |
| * Kernel newbies, <https://kernelnewbies.org/>. |
| |
| Loadable kernel modules: |
| |
| * Peter Jay Salzman, Michael Burian, Ori Pomerantz: The Linux Kernel Module |
| Programming Guide, Version 2.6, |
| <https://tldp.org/LDP/lkmpg/2.6/html/index.html>. |
| |
| Kernel module packages: |
| |
| * Kernel Module Packages Manual, SUSE Linux Enterprise 12 SP2 or later and SUSE |
| Linux Enterprise 15, |
| <https://documentation.suse.com/sbp/all/html/SBP-KMP-Manual-SLE12SP2/index.html>, |
| * SUSE SolidDriver Program, <https://drivers.suse.com/doc/SolidDriver/>. |
| |