Difference between revisions of "Debian Packaging Workflow"

Revision as of 15:31, 30 March 2022

Notes on Maintaining the Debian Packages

Although you may still find a debian subdirectory in our OPM repositories this is obsolete and should not be used as it is rather outdated. The Debian packaging effort takes place on salsa.debian.org and is managed by the Debian Science Team. To contribute you will have to create an account on salsa.debian.org and apply for being added to the Debian Science Team. If it takes long then you might want to send an email to the Debian Science mailinglist debian-science@lists.debian.org.

Repositories on Salsa

For each module there is a corresponding repository on sala. These are:

The packaging effort uses git-buildpackage as the main tool and bases the packaging on release tarballs and quilt for patching the upstream versions found in OPM's git repositories.

git-buildpackage instruction

repository management with git-buildpackage

To clone a repository please use gbp use e.g.: gbp clone https://salsa.debian.org/science-team/opm-common

This will clone the repository with the three banches master (Where the debian packaging effort is), upstream (The upstream repository as of the last release that was imported), and pristine-tar (The branch holding the deltas for reconstructing the original tarball).

To download the recent changes to the packaging effort from other do gbp pull
To upload your recent changes (to all relevant branches) use gbp push

Import new (final) upstream release

For more information see gbp documentation.

For importing a new upstream version, please follow the following recipe:

If not done yet, add the upstream git to repo , e.g. git remote add upstream https://github.com/OPM/opm-common
make sure the patch-queue branch is up to date
update upstream: git fetch upstream
update necessary branches from debian repo (in case of changes from others): gbp pull
import new version: gbp import-orig --uscan
Change changelog: dch --newversion <new-version>
rebase patch queue: gbp pq rebase
resolve conflicts, etc.
export the patch queue: gbp pq export --commit
change the commit message to something more meaningful
Do further work on the package.

Import a prerelease version using an upstream git commit

Instead of importing the new version using uscan as above, you manually create a tarball based on a git checkout of upstream. This assumes that you have checked out the version that you need (e.g. by git checkout upstream/master).

PACKAGE=opm-common # name of the package
NEXTVERSION=2022.04 # next stable version
VERSION=$(git log -1 --date=format:%Y%m%d --pretty=$NEXTVERSION~git%cd.%h)
TAG=$(git rev-parse --short HEAD)
PREFIX=${PACKAGE}_${VERSION}; DIR_PREFIX=${PACKAGE}-${VERSION};
git archive --format=tar --prefix=$DIR_PREFIX/ HEAD | gzip -c > ../$PREFIX.orig.tar.gz
git checkout master # branch with the debian packaging
gbp import-orig --upstream-vcs-tag=$TAG ../$PREFIX.orig.tar.gz
dch --newversion $VERSION

Doing it this way will make sure that branch upstream contains the complete upstream git history. Note that one could also use a git tag instead of a git commit hash instead.

Building the packages in a chroot using cowbuilder

These instructions are for using cowbuilder to build the packages in a chroot. Please follow the instructions described in the previous link to set it up.

Additional setup instructions

To make the following work, please make sure that there is a $HOME/.pbuilderrc with an entry that sets the HOOKDIR like

# the hook dir may already be set/populated!
HOOKDIR="$HOME/.pbuilder/hooks/"

Running autopkgtest

The packages of the OPM modules come with autopkgtest that will test the installed packages for usability. As it is really handy to have these tests run when building the packages, you should copy the file /usr/share/doc/pbuilder/examples/B20autopkgtest to $HOME/.pbuilder/hooks (you might need to create that directory first.

Running Lintian

Lintian will do a lot of sanity checks for the built packages. To run it automatically you should copy the file /usr/share/doc/pbuilder/examples/B20autopkgtest to $HOME/.pbuilder/hooks (you might need to create that directory first).

Opening a shell in case of a failed build

To investigate problems if a build is failing, it is handy to open a shell in the chroot in case of failure. To do that simply copy the file /usr/share/doc/pbuilder/examples/B20autopkgtest to $HOME/.pbuilder/hooks (you might need to create that directory first).

Performing builds using locally built packages

As OPM consists of multiple modules that depend on each other, for a new release we need to build all of this modules and test them at once. To do that we copy the built modules to /var/cache/pbuilder/deps/opm-sid/</sid> and let cowbuilder create a local repository for that. To do that create a file $HOME/.pbuilder/hooks/D05deps/ with the following content:

DEPSPATH="$DEPSBASE/$DEPS"
if [ -n "$DEPS" ] && [ -d "$DEPSPATH" ] ; then
    apt-get install --assume-yes apt-utils
    ( cd "$DEPSPATH"; apt-ftparchive packages . > Packages )
    echo "deb [trusted=yes] file://$DEPSPATH ./" >> /etc/apt/sources.list
    apt-get update
fi

Also add the following code to $HOME/.pbuilderrc:

if [ -n "$DEPS" ] ; then
    export DEPSBASE=/var/cache/pbuilder/deps
    BINDMOUNTS=$DEPSBASE
    BUILDRESULT=$DEPSbase/$DEPS
else
    BUILDRESULT=/var/cache/pbuilder/result
fi

Building the package

Building the package using pbuilder in the chroot is done using the command:

DEPS=opm-sid gbp buildpackage --git-ignore-new \
 --git-export-dir=../build-area-pbuilder-sid --git-pbuilder

@@ Line 33: / Line 33: @@
 * make sure the patch-queue branch is up to date
 * update upstream: <code>git fetch upstream</code>
-* update necessary branches from debian repo (in case of changes from others): gbp pull
+* update necessary branches from debian repo (in case of changes from others): <code>gbp pull</code>
 * import new version: <code>gbp import-orig --uscan</code>
+* Change changelog: <code>dch --newversion <new-version></code>
 * rebase patch queue: <code>gbp pq rebase</code>
 * resolve conflicts, etc.
 * export the patch queue: <code>gbp pq export --commit</code>
 * change the commit message to something more meaningful
+* Do further work on the package.
 ==== Import a prerelease version using an upstream git commit ====
@@ Line 53: / Line 55: @@
   git checkout master # branch with the debian packaging
   gbp import-orig --upstream-vcs-tag=$TAG ../$PREFIX.orig.tar.gz
+ dch --newversion $VERSION
 Doing it this way will make sure that branch upstream contains the complete upstream git history. Note that one could also use a git tag instead of a git commit hash instead.