world leader in high performance signal processing
Trace: » maintenace

Blackfin Kernel Maintenance

Introduction

This covers the process of merging patches from the ADI vendor tree hosted in gforge on http://blackfin.uclinux.org/ to the upstream trees.

Version Control System

You'll need to be familiar with git. This is what we use to maintain our own kernel tree (migrated from SVN, and from CVS before that), and this is what all relevant upstream maintainers use.

Repository Trees

During Blackfin kernel development, developers or kernel users should understand several important source repository trees.

Blackfin Linux project

This git tree is our main development repository. The Blackfin Linux team maintains this repository. It has several branches according to the stable releases such as 2010R1, 2009R1, etc… But we mainly talk about the GIT trunk branch in this document.

Blackfin kernel.org tree

This git tree is hosted in kernel.org and maintained by Bob Liu lliubbo@gmail.com. It has a few important branches:

  • for-linus branch: Blackfin arch related patches ready for merging into mainline will be pushed to this branch, where the linux-next people merge nightly for testing, and ultimately Linus will pull from to get patches into mainline.
  • trunk branch: This branch includes all the patches maintained by the Blackfin Linux project. It is used to keep the blackfin.uclinux.org git tree in sync with mainline. It is often rewritten as patches are added or removed to the gforge tree.

Other branches will spring up from time to time where the name will be for-<subsystem>. These are patches for the specified subsystem and are in the process of being merged into them.

Linus mainline tree

This is the “official” Linux kernel tree. All code will eventually make it here before being bundled up and released onto http://kernel.org/ as tarballs.

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=summary

Stable tree

Each major release will have a stable tree associated with it. This is used to rebase the gforge trunk branch onto any stable updates made after the major release.

So for the 2.6.38 release, the tree would be:
http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.38.y.git;a=summary

In there you can find 2.6.38.1, 2.6.38.2, and so on.

Development Cycle

Blackfin uClinux Project

Blackfin uClinux project is a fully open source project, so all the components of it are open source. We have to integrate all the components as a Blackfin uClinux distribution and test it, then provide it to our customers. Our own development cycle almost depends on customer requirements and our development progress, normally it is half a year to one year. For example, we released 2 stable release in 2008, they are 2008R1 and 2008R1.5. In 2009, we plan to release at least one stable release 2009R1.

Linux Kernel Mainline and Stable Tree

On the other hand, Linux kernel mainline has its own development cycle. Generally speaking, Linus release a mainline kernel version every 2 or 3 months. So there are 5 kernel releases in 2008: from 2.6.24 to 2.6.28. It releases more than us. After Linus release the mainline kernel, Greg KH will start a branch of that mainline kernel to do stable release. Linus always works on his mainline tree. Greg will do stable kernel release depends on his policy, so we got some stable kernel version such as 2.6.28.6 which normally includes critical bug fixing.

Kernel Development Process

For Blackfin Linux team and other developers, we do our development highly based on our gforge git tree. Before a release, we commit our code to the trunk branch.

When code is frozen, a release branch will be created from trunk such as 2008R1 branch. In the release branch we choose one mainline kernel release for our Blackfin uClinux release, such as 2.6.22 for 2008R1 and 2.6.28 for 2009R1.

But for the trunk branch, we should still sync with mainline kernel release to get new features from upstream. For example, after branched 2008R1, our kernel was 2.6.22. We did lots of development based on 2.6.22. When kernel mainline released 2.6.23, we have to upgrade our trunk branch to 2.6.23 and do not lose our development since branching 2008R1. Also on the other hand, our development in git trunk need to be sent out for mainline or upstream, request them to merge our development.

There are 2 main kinds kernel maintenance tasks: patch management and release updates. we will discuss these 2 tasks in detail as following.

Patch Management

Since both the upstream linux tree and our gforge tree are maintained in git, it is easy to have both in a single local git repo. Changes made to one can be cherry picked and applied to the other.

So as people make commits to the gforge trunk branch, those changes are cherry picked, cleaned up, squashed/dropped, and merged into the blackfin.git trunk branch on kernel.org. Then the changeset goes through review on the relevant mailing lists, and is further refined.

Prepare merged tree

  • Checkout the latest Linus tree:
    $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git blackfin
  • Merge the gforge tree:
    $ cd blackfin
    $ git remote add -t trunk --no-tags gforge git://sources.blackfin.uclinux.org/git/linux-kernel
    $ git fetch gforge
    
  • Create local branch for merging mainline and gforge based on current release:
    $ git checkout -b trunk v2.6.38
  • Create patchsets for every commit in gforge until there are no differences between the trees:
    $ git cherry-pick <commit in gforge>
    $ git diff -p --stat trunk..gforge/trunk

Submit patches to subsystem upstream

We use git send-email to send out patches to upstream review. Please take a look at the manual of this command.

Request git-pull

For Blackfin architecture related patches, we push them to the for-linus branch of kernel.org blackfin git tree and send an email to Linus to request him to pull the patches into his mainline kernel tree.

  • Sync master branch with Linus mainline kernel:
    $ git checkout master
    $ git pull origin master
  • Preparing for-linus and guilt branch patch:
    $ git checkout guilt
    $ git merge master
    $ guilt-push blackfin_arch__last_one_in_the_blackfin_arch_related_patches.patch
    $ git checkout for-linus
    $ git merge guilt
    $ git checkout guilt
    $ guilt-push -a
  • Push out all the patches:
    $ git push --force blackfin
  • Send out email to Linus Torvalds torvalds@linux-foundation.org and Cc LKML linux-kernel@vger.kernel.org. Please use the script to generate the email content
#!/bin/sh

#
# usage: mkmsg [branch] [output text file]
#

BRANCH="$1"
TEXT_OUT="$2"

PWD=`pwd`
REPO=`basename $PWD`

echo "Hi Linus,"
echo ""
echo "Please pull from '$BRANCH' branch of" > $TEXT_OUT
echo "master.kernel.org:/pub/scm/linux/kernel/git/cooloney/blackfin.git/ $BRANCH" \
        >> $TEXT_OUT
echo "" >> $TEXT_OUT
echo "to receive the following updates:" >> $TEXT_OUT
echo "" >> $TEXT_OUT

git diff -M --stat --summary master..$BRANCH >> $TEXT_OUT
echo "" >> $TEXT_OUT
git log --no-merges master..$BRANCH | git shortlog >> $TEXT_OUT
git diff master..$BRANCH >> $TEXT_OUT

Then use your email client to send out the email like this.

Release Updates

Update gforge trunk to sync with upstream mainline kernel

Our development tracks the latest release, so when Linus posts a new update, we need to update to that. Since we have both trees in git, we can simply use that to generate an appropriate patch.

First, make sure your local trunk branch (which is based on mainline git) is up to date with the gforge trunk branch (which has its own unique history based on SVN/CVS). This should show no differences:

$ git diff trunk..gforge/trunk

Now rebase your local trunk branch onto the new release (e.g. upgrading from 2.6.38 to 2.6.39). This will often generate conflicts when local patches have been merged upstream, or when the code patches touch has been modified.

$ git rebase --onto v2.6.39 v2.6.38

Create a diff between your updated tree and the current gforge tree:

$ git diff trunk..gforge/trunk > upgrade.patch

Finally apply that diff to the gforge trunk branch and push it out.

Stable Release Updates

In the upstream mainline kernel maintenance, Linus will release the mainline stable kernel from his mainline git tree, such as 2.6.28. After that, Greg or other member of stable kernel team will branch a git tree based on Linus release for further stable release updates. Because some critical bugs will be found after Linus release stable kernel, those fixings should also be backported to stable release kernel. The stable kernel team and the new stable kernel tree will take care of that.

When Blackfin Linux project choose a stable kernel as our release kernel such as 2.6.28 for 2009R1, we should update our kernel to sync with stable kernel release updates such as 2.6.28.7.

We update our kernel to sync with stable release until the Blackfin Linux kernel code freeze, even if we create our kernel release branch such as 2009R1. The stable release updates should also go to the 2009R1 branch. After code freeze, we will do lots of testing. But upstream stable release such as 2.6.28.7 contains lots of changes which might break our whole Blackfin Linux distribution. So stable release updates only happens before code freeze.