Difference between revisions of "How I created the release branches."

From opm
Jump to: navigation, search
(Creating the release branches)
(Creating the release branches)
Line 184: Line 184:
  
 
  ./alldo.bash git diff upstream/master
 
  ./alldo.bash git diff upstream/master
 +
 +
The next command I did was an error:
 +
 +
./alldo.bash git checkout release/2016.10
 +
 +
The missing "-b" option meant that this would only access existing release branches:
 +
 +
============ Running command: 'git checkout release/2016.10' =============
 +
=== Entering module: opm-common
 +
error: pathspec 'release/2016.10' did not match any file(s) known to git.
 +
=== Entering module: opm-parser
 +
Branch release/2016.10 set up to track remote branch release/2016.10 from upstream.
 +
Switched to a new branch 'release/2016.10'
 +
=== Entering module: opm-material
 +
error: pathspec 'release/2016.10' did not match any file(s) known to git.
 +
=== Entering module: opm-output
 +
Branch release/2016.10 set up to track remote branch release/2016.10 from upstream.
 +
Switched to a new branch 'release/2016.10'
 +
=== Entering module: opm-core
 +
error: pathspec 'release/2016.10' did not match any file(s) known to git.
 +
=== Entering module: opm-grid
 +
error: pathspec 'release/2016.10' did not match any file(s) known to git.
 +
=== Entering module: opm-simulators
 +
error: pathspec 'release/2016.10' did not match any file(s) known to git.
 +
=== Entering module: opm-upscaling
 +
error: pathspec 'release/2016.10' did not match any file(s) known to git.
 +
=== Entering module: ewoms
 +
error: pathspec 'release/2016.10' did not match any file(s) known to git.
 +
=== Entering module: opm-data
 +
error: pathspec 'release/2016.10' did not match any file(s) known to git.
 +
=== Entering module: opm-utilities
 +
error: pathspec 'release/2016.10' did not match any file(s) known to git.
 +
============ Done running command: 'git checkout release/2016.10' =============
 +
 +
However, this reminded me that opm-parser and opm-output were already handled by the maintainer of those repos (Joakim Hove).
 +
I therefore *removed those repos* from the "modules" file before continuing. Then I did:
 +
 +
./alldo.bash git checkout -b release/2016.10
 +
 +
and all was well, as evidenced by running
 +
 +
./alldo.bash git branch
 +
 +
(showing that all are on the release branch) and
 +
 +
./alldo.bash git diff upstream/master
 +
 +
(showing that there are no diffs).
 +
Finally, it was time to push the branches to the main repositories:
 +
 +
./alldo.bash git push upstream

Revision as of 20:20, 25 October 2016

Setup and local checkout

I created a local directory to hold the local checkout (and a subdir for the sources):

mkdir release-opm
cd release-opm
mkdir src

Then I added the following as a file "modules",

opm-common
opm-parser
opm-material
opm-output
opm-core
opm-grid
opm-simulators
opm-upscaling
ewoms
opm-data
opm-utilities

and the following script as "checkout.bash"

#!/bin/bash

for m in $(cat modules); do
    echo "=== Checking out module: $m"
    (
        cd src;
        if [ ! -d "$m" ]; then
            git clone git@github.com:atgeirr/$m.git
            cd $m
            git remote add upstream git@github.com:OPM/$m.git
            git remote update
            git merge upstream/master
        else
            echo "Module $m already checked out."
        fi
    )
done

Then I ran the script, which takes some time since opm-data is quite large at this point. Also note that the script should be edited to use your GitHub user name instead of "atgeirr".

After that, you will have repositories with ssh remotes to both "origin" (your clone) and "upstream" (the OPM main repo), so be careful.

Building locally

This step is strictly not necessary to do the release branching, but it is an important sanity check. These are the scripts I used, while other exist, I used these because I had used them successfully before.

For this we need some more files. The modules to build list in the file "modules_build":

opm-common
opm-parser
opm-material
opm-output
opm-core
opm-grid
opm-simulators
opm-upscaling
ewoms

A script for building in "build.bash":

#!/bin/bash

for m in $(cat modules_build); do
    echo "==================================================="
    echo "=        Building module: $m"
    echo "==================================================="
    (
        cd src
        builddir="$m-build"
        if [ ! -d "$builddir" ]; then
            mkdir "$builddir"
            cd "$builddir"
            cmake -C ../../releaseopts.cmake "../$m"
            make -j 7
            make test
        else
            echo "******** Skipping $m, build dir $builddir already exists."
        fi
    )
done

Note that the "make -j 7" is fine with 4 cores and hyper-threading as my machine has, you might want to increase or decrease it though. And finally the CMake cache preload file I used, "releaseopts.cmake":

set(CMAKE_C_COMPILER "clang" CACHE STRING "C compiler")
set(CMAKE_CXX_COMPILER "clang++" CACHE STRING "C++ compiler")
set(CMAKE_Fortran_COMPILER "/usr/local/bin/gfortran" CACHE STRING "Fortran compiler")
set(OPM_COMMON_ROOT "/Users/atgeirr/release-opm/src/opm-common" CACHE STRING "OPM build system")
# set(DUNE_ROOT "/Users/atgeirr/dune" CACHE STRING "Dune core modules")
set(CMAKE_PREFIX_PATH "/Users/atgeirr/dune/dune-istl"
                      "/Users/atgeirr/dune/dune-common"
                      "/Users/atgeirr/dune/dune-localfunctions"
                      "/Users/atgeirr/dune/dune-grid"
                      "/Users/atgeirr/dune/dune-geometry" CACHE STRING "Dune core modules")
set(SUPERLU_ROOT "/Users/atgeirr/install/SuperLU_4.3" CACHE STRING "Path to SuperLU")
set(PRECOMPILE_HEADERS OFF CACHE BOOL "Precompiled headers")
set(SILENCE_EXTERNAL_WARNINGS ON CACHE BOOL "Silence warnings from external libraries")
set(USE_MPI ON CACHE BOOL "Use MPI")
set(USE_OPENMP OFF CACHE BOOL "Use OpenMP")
set(USE_PTHREAD OFF CACHE BOOL "Use pthreads")
set(CMAKE_BUILD_TYPE "Release" CACHE STRING "Build type")
set (CMAKE_CXX_FLAGS
        "-pipe -std=c++0x -pedantic -Wall -Wextra -Wformat-nonliteral -Wcast-align -Wpointer-arith -Wmissing-declarations -Wcast-qual -Wshadow -Wwrite-strings -Wchar-subscripts -Wredundant-decls -Wno-unused-local-typedef -Wno-keyword-macro -Wno-missing-braces"
        CACHE STRING "Flags used by the compiler during all build types")
set (CMAKE_C_FLAGS_DEBUG
        "-O0 -g"
        CACHE STRING "Flags used during Debug builds")
set (CMAKE_CXX_FLAGS_DEBUG
        "-O0 -g"
        CACHE STRING "Flags used during Debug builds")
set (CMAKE_C_FLAGS_RELEASE
        "-O2 -DNDEBUG -march=native -mtune=native"
        CACHE STRING "Flags used during Release builds")
set (CMAKE_CXX_FLAGS_RELEASE
        "-O2 -DNDEBUG -march=native -mtune=native"
        CACHE STRING "Flags used during Release builds")

The above contain paths that could be totally wrong for your system, and settings (MPI on, OpenMP off etc.) that does not reflect the settings you want to use for your sanity testing.

More utilities

We're going to want some more utilities to ease the job. First, a tool that executes a command in all the module repos (for git commands):

#!/bin/bash
echo "============ Running command: '$@' ============="
for m in $(cat modules); do
    echo "=== Entering module: $m"
    ( cd src
      cd $m
      $@ )
done
echo "============ Done running command: '$@' ============="

And its companion, a tool that does the same, in the build directories (for make etc.):

#!/bin/bash
echo "============ Running command: '$@' ============="
for m in $(cat modules_build); do
    echo "=== Entering module build dir: $m-build"
    ( cd src
      cd "$m-build"
      $@ )
done
echo "============ Done running command: '$@' ============="

Creating the release branches

Finally we have reached the part where we actually make the branches!

First we should do some sanity checking. The following command:

./alldo.bash git branch

should give the following output:

============ Running command: 'git branch' =============
=== Entering module: opm-common
* master
=== Entering module: opm-parser
* master
=== Entering module: opm-material
* master
=== Entering module: opm-output
* master
=== Entering module: opm-core
* master
=== Entering module: opm-grid
* master
=== Entering module: opm-simulators
* master
=== Entering module: opm-upscaling
* master
=== Entering module: ewoms
* master
=== Entering module: opm-data
* master
=== Entering module: opm-utilities
* master
============ Done running command: 'git branch' =============

After this, ensure that we are identical to the upstream master branches:

./alldo.bash git diff upstream/master

The next command I did was an error:

./alldo.bash git checkout release/2016.10

The missing "-b" option meant that this would only access existing release branches:

============ Running command: 'git checkout release/2016.10' =============
=== Entering module: opm-common
error: pathspec 'release/2016.10' did not match any file(s) known to git.
=== Entering module: opm-parser
Branch release/2016.10 set up to track remote branch release/2016.10 from upstream.
Switched to a new branch 'release/2016.10'
=== Entering module: opm-material
error: pathspec 'release/2016.10' did not match any file(s) known to git.
=== Entering module: opm-output
Branch release/2016.10 set up to track remote branch release/2016.10 from upstream.
Switched to a new branch 'release/2016.10'
=== Entering module: opm-core
error: pathspec 'release/2016.10' did not match any file(s) known to git.
=== Entering module: opm-grid
error: pathspec 'release/2016.10' did not match any file(s) known to git.
=== Entering module: opm-simulators
error: pathspec 'release/2016.10' did not match any file(s) known to git.
=== Entering module: opm-upscaling
error: pathspec 'release/2016.10' did not match any file(s) known to git.
=== Entering module: ewoms
error: pathspec 'release/2016.10' did not match any file(s) known to git.
=== Entering module: opm-data
error: pathspec 'release/2016.10' did not match any file(s) known to git.
=== Entering module: opm-utilities
error: pathspec 'release/2016.10' did not match any file(s) known to git.
============ Done running command: 'git checkout release/2016.10' =============

However, this reminded me that opm-parser and opm-output were already handled by the maintainer of those repos (Joakim Hove). I therefore *removed those repos* from the "modules" file before continuing. Then I did:

./alldo.bash git checkout -b release/2016.10

and all was well, as evidenced by running

./alldo.bash git branch

(showing that all are on the release branch) and

./alldo.bash git diff upstream/master

(showing that there are no diffs). Finally, it was time to push the branches to the main repositories:

./alldo.bash git push upstream