Continuous integration process consists in automatically building software sources, run tests, and publish documentation and binaries of packages.

This page describes how to deal with continuous integration, based on the use of a gitlab server:

Install continuous integration

In following subsections we explain de process that administrator have to follow to put in place contiunous integration. This process is based on gitlab so we suppose that projects repositories are available on a gitlab server. In gitlab, continuous integration is already integrated, but is divided into two parts: coordinator server and ci-runners. The coordinator server is the server hosting the gitlab application so you have no more to do except deploying a gitlab server like gite.lirmm.fr. The ci-runners are the application that trully run the build/test process depending on the coordinator ask them, and then they can give back resulting artefacts to the coordinator once the build/test process is finished. The following subsection will mainly speak about how to deploy and configure a runner (either on a server or on a workstation, this does not really matter even if it is preferable to have a runner always running and accessible on a server). We will call the machine that host the ci-runner the build server.

Install docker on the build server

Docker engine is required to execute the ci-runner with docker container management. You can follow the gitlab help page to kno how to installa runner or follow these instructions, considering you are working on a debian like system:

  • update the server packages database:
sudo apt-get update
  • Add the GPG key for the official Docker repository to the system:
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
  • Add the Docker repository to APT sources:
sudo apt-add-repository 'deb https://apt.dockerproject.org/repo ubuntu-xenial main'
  • Update the package database with the Docker packages from the newly added repo:
sudo apt-get update
  • You should see output similar to the follow:
docker-engine:
  Installed: (none)
  Candidate: 1.11.1-0~xenial
  Version table:
     1.11.1-0~xenial 500
        500 https://apt.dockerproject.org/repo ubuntu-xenial/main amd64 Packages
     1.11.0-0~xenial 500
        500 https://apt.dockerproject.org/repo ubuntu-xenial/main amd64 Packages

Notice that docker-engine is not installed, but the candidate for installation is from the Docker repository for Ubuntu 16.04. The docker-engine version number might be different.

  • Install the cadidate package:
sudo apt-get install -y docker-engine
  • Docker should now be installed, the daemon started, and the process enabled to start on boot. Check that it’s running:
sudo systemctl status docker
  • The output should be similar to the following, showing that the service is active and running:
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2016-05-01 06:53:52 CDT; 1 weeks 3 days ago
     Docs: https://docs.docker.com
 Main PID: 749 (docker)
  • If you want to avoid typing sudo whenever you run the docker command, add your username to the docker group:
sudo usermod -aG docker $(whoami)

Uninstall docker

If you want to uninstall docker do:

sudo apt-get purge docker-ce
sudo rm -rf /var/lib/docker

Install gitlab-ci runner on the server

  • Add GitLab’s official repository:
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.deb.sh | sudo bash
  • install the runner:
sudo apt-get install gitlab-ci-multi-runner

update runner version

sudo apt-get update
sudo apt-get install gitlab-ci-multi-runner

uninstall runner

sudo apt-get remove gitlab-ci-multi-runner

Create a dedicated docker image for PID in gitlab-ci

You need to setup a specific docker image that contains all the required dependencies in order for PID CI system to work:

  • Find the base image corresponding to the kernel in use. For instance on a ubuntu server we will use the ubuntu image:
sudo docker search ubuntu

The output gives you all available images for ubuntu on docker hub, something like:

NAME                                         DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
ubuntu                                       Ubuntu is a Debian-based Linux operating s...   5969      [OK]       
rastasheep/ubuntu-sshd                       Dockerized SSH service, built on top of of...   84                   [OK]
ubuntu-upstart                               Upstart is an event-based replacement for ...   71        [OK]       
ubuntu-debootstrap                           debootstrap --variant=minbase --components...   30        [OK]       
torusware/speedus-ubuntu                     Always updated official Ubuntu docker imag...   27                   [OK]
nuagebec/ubuntu                              Simple always updated Ubuntu docker images...   20                   [OK]
nickistre/ubuntu-lamp                        LAMP server on Ubuntu                           17                   [OK]
nimmis/ubuntu                                This is a docker images different LTS vers...   7                    [OK]
...

Choose the adequate image, for now we use the basic ubuntu image. To select an adeqaute version of ubuntu use docker tags like ubuntu:18.04 to get the corresponding version of the distribution.

  • Installing this image locally:
sudo docker pull ubuntu:18.04

Now we can use it to create a new docker image.

  • Create the new image by using an interactive approach:
sudo docker run -it ubuntu:18.04

Now you enter in a bash session.

  • configure the container in order to manage ssh and generate ssh keys WARNING: do not enter any passphrase when creating keys otherwise the runner would prompt for it when running tests !!
# launch the ssh service
which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )
eval `ssh-agent -s`
ssh-keygen -t rsa -b 2048
# create a ssh key pair (do not enter a passphrase), then add the private key to the known identities
ssh-add /root/.ssh/id_rsa
# copy the content of the id_rsa.pub key just generated and paste it somewhere you can easily find it again
  • To find out the hostkeys of your gitlab server, put the result of the ssh-keyscan YOUR_SERVER command into your ssh known hosts:
ssh-keyscan gite.lirmm.fr > /root/.ssh/known_hosts
  • adding required dependencies for a full featured “PID ready” runner:
apt update
apt install -y curl git build-essential clang cmake git doxygen jekyll cppcheck lcov gfortran lsb-release
  • configure the runner git account:
git config --global user.email "passama@lirmm.fr"
git config --global user.name "PID GitlabRunner"

Here simply use same name and email as those used when the PID GitlabRunner identity has been created in gitlab. Content generated by this runner will so be tagged as written by PID GitlabRunner.

  • adding git lfs dependency:
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash
apt-get install git-lfs
git lfs install
  • adding sphinx dependency (only useful when runner must build the pid framework, to optimize a bit the configuration process):
apt install python python-pip
pip install sphinx
pip install sphinx_theme
pip install sphinx_rtd_theme
  • exitting the interactive mode :

simple type ‘exit’ in the shell.

  • Registering the container as a reusable image:
  1. Getting the id of the container you just exitted
sudo docker ps -l

The output gives you the id of this container, something like:

CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                     PORTS               NAMES
0132729b8dfc        ubuntu              "/bin/bash"         28 minutes ago      Exited (0) 4 minutes ago                       optimistic_easley
  1. Create the new image:

You have to replace names and IDs with the adequate informations.

sudo docker commit -m "environment for PID CI runner is ready" -a "Robin Passama" 0132729b8dfc pid-runner-x86_64

The image is now cached, to see it look into the local repository:

sudo docker images

Do not worry if the image is quite big is normal because we have a fully functional development environment.

Note: You can install more package to get a more complete environment. This may be useful if you want your runner to reflect the configuration of a given computer/server.

Register the runner in gitlab

  • In gitlab create a dummy account for the runner (e.g. with username pid-gitlab-runner), then add the public ssh key to this account by copy/pasting the content of the public generated in the docker container. You can otherwise add an existing PID runner.

  • As a final step, in gitlab add the new PID GitlabRunner user to the pid-workspace project. Go to Settings > Member and add it as a Developper of the project. Another way to do that is to set the PID GitlabRunner user as a member of the group containing the official pid-workspace project.

Executing the runner

This step consists in running the gitlab runner service and in regitering this runner into the gitlab instance.

You need to collect some information prior to executing the runner:

  • the URL of your gitlab instance. At LIRMM the URL of our gitlab server that coordinates runner is https://gite.lirmm.fr
  • go into Admin area, in menu Overview click on the Runners tab. The pages provide you the token to use got registering the runner.
  • the name of the docker image to use. This is the image generated at previous step, that we called pid-runner-x86_64 (use the good name depending on your server specification).
  • the platform you are currently targetting with the container. It depends on: the architecture type in use (x86, arm), the architecture bits (32 or 64), the OS (without greater customization by default linux is the only possible solution with docker), and the abi in use, depending on your compiler version (if your compiler is gcc with version greater than 5.0 or clang with version greater than 3.0, the ABI is abi11, otherwise it is abi98 (the legacy ABI for C/C++). From these information you can build a platform description string like: x86_64_linux_stdc++11 (put underscores between each term).

Now you can execute the runner by setting adequate information for options –url, –registration-token (given by gitlab administrator interface), –docker-image (pointing to the docker image we just created) and –tag-list (add a tag describing the host platform). For this later option keep the term pid into the list to tell the system it is configured adequately with PID dependencies.

sudo gitlab-ci-multi-runner register \
  --url "https://gite.lirmm.fr/" \
  --registration-token "-n45kpjhcHzSnza" \
  --description "runner for PID" \
  --executor "docker" \
  --docker-image pid-runner-x86_64 \
  --tag-list "pid,x86_64_linux_stdc++11"

The program prompts the user to enter values, simply press enter to keep default values passed as parameters.

Configuring the runner in gitlab (required admin rights in gitlab)

Go to the runners page of gitlab admin area: the new gitlab runner should appear.

One last configuration may be required depending on version of gitlab you are using:

  • Active option should be checked.
  • Protected must be unchecked.
  • Lock to current project option must be unchecked.
  • Run untagged jobs option should be unchecked, in order to avoid non PID build to interfere with PID jobs.

Final configuration of the build server

On the server edit the file /etc/gitlab-runner/config.toml. This file is used to configure the behavior of runners (those registered by root superuser). This file contains a section for each registered runner something that looks like.

...
[[runners]]
  name = "runner for PID"
  url = "https://gite.lirmm.fr/"
  token = "f5288e033998f2898127da2f2758cf"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "pid-runner-x86_64"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    # add the line below
    shm_size = 0
  [runners.cache]

If you have multiple runner registered just try to find the adequate section by simply looking at image = property. When done add to this section the parameter pull_policy = "if-not-present":

...
  [runners.docker]
    tls_verify = false
    image = "pid-runner-x86_64"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    # add the line below
    pull_policy = "if-not-present"
    shm_size = 0
  [runners.cache]

Then:

sudo gitlab-ci-multi-runner restart

Enable continuous integration for a package

From package developer perspective there are few steps to follow to enable continuous integration.

  • In gitlab go into your project general settings and activate pipelines. The CI/CD pipelines menu now appears in project settings.

  • In this menu choose the git clone strategy in the section Git strategy for pipelines, otherwise builds will fail (due to a BUG in gitlab-ci relative to the management of git LFS).

  • In order to optimize a bit the CI process you should also check the Auto-cancel redundant, pending pipelines option and uncheck the Public pipelines.

  • Finally add the available runners gitlab users to your gitlab project. Ask the PID administrators for the name of users to set as member. You can choose the Developper for these users (so that you are sure they will not push to master branch).

That’s it your project is ready to be automatically built and deployed.

These steps are the same wether the considered gitlab project is:

  • a package project
  • a framework project
  • a package static site project

Additional configuration for static sites projects

Static site project require one additionnal step : you have to set runners gitlab user that are member of the project as Master (instead of Developpers), so that they will be able to update their content.