Difference between revisions of "Debian Packaging Workflow"

Latest revision as of 19:42, 30 March 2022

Notes on Maintaining the Debian Packages

Although you may still find a debian subdirectory in our OPM repositories this is obsolete and should not be used as it is rather outdated. The Debian packaging effort takes place on salsa.debian.org and is managed by the Debian Science Team. To contribute you will have to create an account on salsa.debian.org and apply for being added to the Debian Science Team. If it takes long then you might want to send an email to the Debian Science mailinglist debian-science@lists.debian.org.

Repositories on Salsa

For each module there is a corresponding repository on salsa. These are:

The packaging effort uses git-buildpackage as the main tool and bases the packaging on release tarballs and quilt for patching the upstream versions found in OPM's git repositories.

git-buildpackage instruction

repository management with git-buildpackage

To clone a repository please use gbp use e.g.: gbp clone https://salsa.debian.org/science-team/opm-common. This will clone the repository with the three banches master (Where the debian packaging effort is), upstream (The upstream repository as of the last release that was imported), and pristine-tar (The branch holding the deltas for reconstructing the original tarball).
To download the recent changes to the packaging effort from other do gbp pull
To upload your recent changes (to all relevant branches) use gbp push

Import new (final) upstream release

For more information see gbp documentation.

For importing a new upstream version, please follow the following recipe:

If not done yet, add the upstream git to repo , e.g. git remote add upstream https://github.com/OPM/opm-common
make sure the patch-queue branch is up to date
update upstream: git fetch upstream
update necessary branches from debian repo (in case of changes from others): gbp pull
import new version: gbp import-orig --uscan
Change changelog: dch --newversion <new-version>
rebase patch queue: gbp pq rebase
resolve conflicts, etc.
export the patch queue: gbp pq export --commit
change the commit message to something more meaningful
Do further work on the package.

Import a pre-release version using an upstream git commit or tag

Instead of importing the new version using uscan as above, you manually create a tarball based on a git checkout of upstream. This assumes that you have checked out the version that you need (e.g. by git checkout upstream/master).

PACKAGE=opm-common # name of the package
NEXTVERSION=2022.04 # next stable version
VERSION=$(git log -1 --date=format:%Y%m%d --pretty=$NEXTVERSION~git%cd.%h)
TAG=$(git rev-parse --short HEAD)
PREFIX=${PACKAGE}_${VERSION}; DIR_PREFIX=${PACKAGE}-${VERSION};
git archive --format=tar --prefix=$DIR_PREFIX/ HEAD | gzip -c > ../$PREFIX.orig.tar.gz
git checkout master # branch with the debian packaging
gbp import-orig --upstream-vcs-tag=$TAG ../$PREFIX.orig.tar.gz
dch --newversion $VERSION

Doing it this way will make sure that the branch upstream contains the complete upstream git history. Note that one could also use a git tag instead of a git commit hash instead. Note. that the version includes the date of the commit to make the order of the releases sensible.

Building the packages in a chroot using cowbuilder

These instructions are for using cowbuilder to build the packages in a chroot. Please follow the instructions described in the previous link to set it up.

Additional setup instructions

To make the following work, please make sure that there is a $HOME/.pbuilderrc with an entry that sets the HOOKDIR like

# the hook dir may already be set/populated!
HOOKDIR="$HOME/.pbuilder/hooks/"

and create that directory if it is not present

Running autopkgtest

The packages of the OPM modules come with autopkgtest that will test the installed packages for usability. As it is really handy to have these tests run when building the packages, you should copy the file /usr/share/doc/pbuilder/examples/B20autopkgtest to $HOME/.pbuilder/hooks (you might need to create that directory first.

Running Lintian

Lintian will do a lot of sanity checks for the built packages. To run it automatically you should copy the file /usr/share/doc/pbuilder/examples/B20autopkgtest to $HOME/.pbuilder/hooks (you might need to create that directory first).

Opening a shell in case of a failed build

To investigate problems if a build is failing, it is handy to open a shell in the chroot in case of failure. To do that simply copy the file /usr/share/doc/pbuilder/examples/B20autopkgtest to $HOME/.pbuilder/hooks (you might need to create that directory first).

Performing builds using locally built packages

As OPM consists of multiple modules that depend on each other, for a new release we need to build all of this modules and test them at once. To do that we copy the built modules to /var/cache/pbuilder/deps/opm-sid/</sid> and let cowbuilder create a local repository for that. To do that create a file $HOME/.pbuilder/hooks/D05deps/ with the following content:

DEPSPATH="$DEPSBASE/$DEPS"
if [ -n "$DEPS" ] && [ -d "$DEPSPATH" ] ; then
    apt-get install --assume-yes apt-utils
    ( cd "$DEPSPATH"; apt-ftparchive packages . > Packages )
    echo "deb [trusted=yes] file://$DEPSPATH ./" >> /etc/apt/sources.list
    apt-get update
fi

Also add the following code to $HOME/.pbuilderrc:

if [ -n "$DEPS" ] ; then
    export DEPSBASE=/var/cache/pbuilder/deps
    BINDMOUNTS=$DEPSBASE
    #BUILDRESULT=$DEPSbase/$DEPS
else
    #BUILDRESULT=/var/cache/pbuilder/result
fi

Building the package

Building the package using pbuilder in the chroot is done using the command:

DEPS=opm-sid gbp buildpackage --git-ignore-new \
 --git-export-dir=../build-area-pbuilder-sid --git-pbuilder

Finishing touches and uploading of the release

Update debian/changelog: gbp dch
Polish debian/changelog.
push changes, e.g. by gbp push
Ask Markus (he is a Debian Maintainer) to upload the release

The uploader should:

Test the package build
Release debian/changelog: dch -r
Tag the release: gbp tag --sign-tags
Push the changes: gbp push

@@ Line 108: / Line 108: @@
   DEPS=opm-sid gbp buildpackage --git-ignore-new \
    --git-export-dir=../build-area-pbuilder-sid --git-pbuilder
+=== Finishing touches and uploading of the release ===
+* Update debian/changelog: <code>gbp dch</code>
+* Polish debian/changelog.
+* push changes, e.g. by <code>gbp push</code>
+* Ask Markus (he is a Debian Maintainer) to upload the release
+The uploader should:
+* Test the package build
+* Release debian/changelog: <code>dch -r</code>
+* Tag the release: <code>gbp tag --sign-tags</code>
+* Push the changes: <code>gbp push</code>