Kernel Repository Guidelines

============================

Table Of Contents

=================

    Getting Started

    Patch Headers

    Before You Commit -- Things To Check

    Config Option Changes

    Committing and Log Messages

    What Is The Kernel ABI?

    Kernel ABI Changes

    Embargoed Patches

    Related Information

Getting Started

===============

Make sure you have a decent git version (1.5.x should work) and quilt

installed.

Introduce yourself if you haven't done so already:

    $ git config --global user.name "Your Name"

    $ git config --global user.email your@email

If you ommit the --global option, the setting will only apply to this

clone.

Set up some git hooks and helpers:

    $ ./scripts/install-git-hooks

To hack on the kernel sources:

    $ ./scripts/sequence-patch.sh --quilt

    $ cd tmp/linux-$version-$branch

    $ quilt new patches.fixes/fix-foo-and-bar

    $ quilt edit some/file.c

    $ ./refresh_patch.sh

    $ quilt header -e # see next chapter

Refer to quilt documentation for details. When you are done, add the new

patch to an appropriate place in the series.conf file and run

./scripts/log to commit it. Patches should be named such that they

consist of alphanumeric characters, '-', and '.'. Typically, patches are

named by filtering the Subject of the patch to a lower-case, dash-separated

form like the one in the example above.

To build rpm packages:

    $ ./scripts/tar-up.sh

This creates a source package in the kernel-source directory. Use

    $ ./scripts/osc_wrapper [kernel-source/kernel-$flavor.spec]

to build a kernel package locally, or

    $ ./scripts/osc_wrapper upload [--ibs]

to have all flavors and architectures built by the Open Build Service.

The --ibs option uses the SUSE internal instance.

Patch Headers

=============

Each patch must have a RFC822-style header that at a minimum describes

what the patch does, who wrote it, and who inside SUSE we'll

"blame" about problems with the patch.  The rules for patch headers are:

 * Each patch must have a From: tag that identifies the author of the

   patch.

 * Each patch must have a Subject: tag that briefly describes what

   the patch does.  A brief summary is it could show up in a change

   log makes the most sense in most cases.

 * Unless the author identified in the From: tag has a @suse.de,

   @suse.com or @suse.cz, address, the patch must include a Signed-off-by:,

   Acked-by: or Reviewed-by: header which identifies the person in one of

   these domains who feels responsible for the patch inside the company.

 * The patch must include a Patch-mainline: tag that identifies where

   the patch came from (for backports from mainline), or when it is

   expected to be added to mainline. The format is one of:

   For backports from mainline:

        Patch-mainline: <upstream version, ex. "v3.5-rc1">

        Git-commit: <git hash>

   If the commit is from a maintainer repository or some other

   repository that isn't Linus's:

        Patch-mainline: Queued in subsystem maintainer repository

        Git-repo: <url>

        Git-commit: <git hash>

   If the patch is not upstream, depending on the situation:

        Patch-mainline: Submitted, <timestamp - destination>

        Patch-mainline: Not yet, <reason>

        Patch-mainline: Never, <reason>

 * The patch should include a References: tag that identifies the Bugzilla bug

   number, JIRA epic, etc. where the patch is discussed.  Please prefix

   bugzilla.suse.com bug numbers with bsc# and JIRA epic numbers with jsc#.

   Please make sure you reference a JIRA epic when referencin JIRA features.

   Have a look at

   http://en.opensuse.org/openSUSE:Packaging_Patches_guidelines#Current_set_of_abbreviations

   for a full list of abbreviations.

 * The patch header may (and often, should) include a more extensive

   description of what the patch does, why, and how.  The idea is to

   allow others to quickly identify what each patch is about, and to

   give enough information for reviewing.

More details about valid patch headers can be found in

scripts/patch-tag-template.  The helper script scripts/patch-tag can be

used for managing these tags.  Documentation for patch-tag can be found

at the top of the script itself.

Example usage of scripts/patch-tag-template:

    $ cp scripts/patch-tag-template ~/.patchtag

    Edit ~/.patchtag with any default values you want

    $ patch-tag -e file.diff

Example patch header:

    | Date: Fri, Sep 26 2008 15:20:10 +1000

    | From: Peter Leckie <pleckie@sgi.com>

    | References: SGI:PV986789 bnc#482148

    | Subject: Clean up dquot pincount code

    | Patch-mainline: v2.6.28

    |

    | Clean up dquot pincount code.

    |

    | This is a code cleanup and optimization that removes a per mount point

    | spinlock from the quota code and cleans up the code.

    |

    | The patch changes the pincount from being an int protected by a spinlock

    | to an atomic_t allowing the pincount to be manipulated without holding

    | the spinlock.

    |

    | This cleanup also protects against random wakup's of both the aild and

    | xfssyncd by reevaluating the pincount after been woken. Two latter patches

    | will address the Spurious wakeups.

    |

    | Signed-off-by: Peter Leckie <pleckie@sgi.com>

    | Acked-by: Jan Kara <jack@suse.cz>

Patch sorting

=============

Patches added to the "sorted patches" section of series.conf must be sorted

according to the upstream order of the commit that they backport.

After you've added a patch file to one of the patches.* directory and added the

required tags described in the section "Patch Headers", run

    $ ./scripts/git_sort/series_insert.py <patch>

to insert an entry for a new patch file to the sorted section of series.conf.

For more information, please read "scripts/git_sort/README.md".

Before You Commit -- Things To Check

====================================

Make sure that all patches still apply after your changes.  One way of

doing this is using scripts/sequence-patch.sh:

    $ export SCRATCH_AREA=/var/tmp/scratch

    $ scripts/sequence-patch.sh

    Creating tree in /var/tmp/scratch/linux-5.3-SLE15-SP3

    Cleaning up from previous run

    Linking from /var/tmp/scratch/linux-5.3.orig

    Applying 50892 patches using 4 threads...

    Saving modified files...

    [ Tree: /var/tmp/scratch/linux-5.3-SLE15-SP3 ]

    [ Generating Module.supported ]

    [ Copying config/x86_64/default ]

Note the Tree: line output by the sequence-patch.sh script which

specifies the location of the expanded kernel tree that is configured

for local build.

Please test-compile the kernel or even test-build kernel packages,

depending on the impact of your changes.  Use scripts/tar-up.sh for

creating an OBS package directory.

The kernel source tree that scripts/sequence-patch.sh creates can be

test-compiled. Before that, make sure all prerequisites are installed.

These include libopenssl-devel, libelf-devel, and dwarves (look into

rpm/kernel-binary.spec.in for a complete list). Then, the compilation

can be done as follows:

    $ cd /var/tmp/scratch/linux-5.3-SLE15-SP3

    $ make oldconfig

    $ make

When committing a patch series try to make the series easy bisectable,

i.e. when applying only the first x patches (1 <= x <= n, n being the

number of patches in the series) the kernel is still buildable and

functional.

This means especially, that just adding upstream patches unmodified

to a series and doing a cleanup patch at the end of the series to

ensure the kernel is buildable and functional is to be avoided. Each

patch from upstream should be modified as required to fit into the

kernel it is backported to, both for build time and runtime.

In cases where upstream patches did cause a breakage (either build or

functional) which was corrected by a later patch and both patches are

part of the series, try to place the correcting patch as near as

possible to the patch causing breakage.

Config Option Changes

=====================

We are building kernel packages for various architectures and

configurations from the same sources.  Each such kernel has its own

configuration file in config/$ARCH/$FLAVOR.  There are checks in place

that abort the kernel build when those configuration files are missing

config options.

When adding patches that add kernel config options, please also update

all config files as follows:

    $ scripts/sequence-patch.sh

    $ cd /var/tmp/scratch/linux-5.3-SLE15-SP3

    $ patches/scripts/run_oldconfig.sh

Committing and Log Messages

===========================

Any commit which affects the kernel package (rather than internals of the

repository such as helper scripts) should be documented in

kernel-source.changes.  The log entry must include the timestamp

(`date`), email address of the committer, and a change summary, and

should include the names of the affected files, as in the following

example:

    | -------------------------------------------------------------------

    | Wed Dec  1 18:29:44 CET 2004 - agruen@suse.de

    |

    | - patches.fixes/serialize-dgram-read.diff: Serialize dgram read

    |   using semaphore just like stream (#48427).

    |

There is a simple helper script for creating changelog entries in this

format (/work/src/bin/vc).

An advanced script (scripts/log) for creating changelog entries from

patch headers, and then automatically committing the change, exists as

well.  This script extracts Subject: and References: headers from added

or modified patches and generates a changelog entry proposal that the

user can further modify.  This approach works well for new or removed

patches.  When modifying existing patches, it usually is necessary to

modify the generated changelog entry by hand.  (scripts/log requires the

vc helper script either in the PATH, or in /work/src/bin/).

What Is The Kernel ABI?

=======================

All symbols that the kernel exports for use by modules, and all symbols

that modules export for use by other modules, are associated with a

so-called modversion, which is a checksum of the type of the symbol

(including all sub-types involved).  Symbols that a module imports are

associated with the identical checksum.

When a module is loaded, the kernel makes sure that the checksums of the

exported symbols match the checksums of the imported symbols.

The kernel exports a large number of symbols (in the range of 5000).  We

could Provide and Require all those symbols as RPM package dependencies,

so that then those dependencies would make sure that all packages

containing kernel modules would have a matching kernel installed.

RPM does not easily handle such large amounts of dependencies though,

and so instead of modeling dependencies based on individual symbols, we

compute distinct classes of symbols, and we create one "kernel(...)"

RPM dependency per class.

Modifying, adding, or removing kernel exports will also change the

kernel(...) symbols.

Kernel ABI Changes

==================

During kernel builds, two things related to the kernel ABI happen:

 * If a reference symvers file (/boot/symvers-* in kernel-$FLAVOR

   packages) for the particular architecture and flavor is available, we

   check how severe the ABI changes are compared to this reference.

   These reference files are located in kabi/$ARCH/symvers-$FLAVOR.  Too

   severe changes will abort the build.  See rpm/kernel-binary.spec.in

   and scripts/kabi-checks (SLES9 - SLES10) or rpm/symsets.pl (SLES11)

   for details.

 * We want to avoid losing kernel(...) symbols when additional symbols

   are added, but all previous symbols are still available: in this

   case, all modules will continue to load into the new kernel just

   fine.

   If a reference symsets file (/boot/symsets-* in kernel-$FLAVOR

   packages) for the particular architecture and flavor is available, we

   check which of the symbol sets in the reference file can still be

   exported, even though symbols have meanwhile been added.  We also

   export the kernel(...) symbols from reference symset files.

To update the reference files, use scripts/update-symvers:

    $ ./scripts/update-symvers kernel-default-2.6.27.25-0.1.1.x86_64.rpm \

         kernel-default-2.6.27.25-0.1.1.i586.rpm ...

It is also possible to update only a subset of the symbols, e.g.:

    $ ./scripts/update-symvers --filter=fs/xfs ...

In either case, ask the branch owner and kernel packager for permission

before touching the kabi files.

Ignoring Kernel ABI Changes

---------------------------

Sometimes we want to tolerate particular kernel ABI changes (and not

abort the builds).  At the same time, we do not want to update the

reference symvers and symsets files, because we still want to monitor the

relative changes, and we want to continue preserving all symbol sets

which are still compatible.

Particular kernel can be marked so that kernel ABI changes are ignored.

This is done by creating kabi/$ARCH/ignore-$FLAVOR files (e.g.,

kabi/x86_64/ignore-xen, kabi/s390/ignore-default).  The kernel ABI checks

will still be produced, but the build will not be aborted.  The file

contents d not matter.

All kernel ABI changes in all kernel packages can be ignored by creating

a file called IGNORE-KABI-BADNESS in the kernel-source/ sub-directory of

the repository that scripts/tar-up.sh creates.  (This may be necessary

for PTF kernels occasionally.)

Embargoed Patches

=================

At certain times during development, the kernel may include "embargoed"

patches, i.e., patches that must not be mad available to parties outside

of SUSE before an agreed-upon time.  Such patches usually have a

date of publication that has been coordinated among linux distributors,

etc. Such patches must not be committed to the usual branches, because

these are pushed to a public mirror, but instead to a branch named with

an _EMBARGO suffix, e.g. SLE11_BRANCH_EMBARGO. The KOTD scripts will

testbuild such branches, but won't publish them. Once the fix becomes

public, the branch needs to be merged back info the "mainline" branch,

e.g.:

    $ git checkout SLE11_BRANCH

    $ git merge SLE11_BRANCH_EMBARGO

Related Information

===================

Internal:

https://wiki.suse.net/index.php/SUSE-Labs_Publications/Kernel_Building

https://wiki.suse.net/index.php/SUSE-Labs_Publications/kernel_patches_rules

Public:

TBD