TGLMAN

Setup Gitlab runner for run ci tests locally

Recently I stumbled upon a issue on a build for one of my side projects that showed up only on the gitlab ci and not locally, so for troubleshoot the problem I tried to setup the same environment on my machine.

Gitlab is quite open and is relatively easy to install it's components, I use debian on my machine and in the debian repository already exist the "gitlab-runner" package, so first step is installing that, together with docker used by the runner to actually execute the builds.

sudo apt install gitlab-runner
sudo apt install docker.io

Done that everything should be straight forward and is enough to go in your project folder and run:

gitlab-runner exec docker build-step-name

This command just read the .gitlab-ci.yaml file find the step with the name build-step-name and try to execute it, and here I stumbled on the first problems of the build, As soon i run this command I receiver a error that docker cannot pull the image gitlab-runner-helper:12.6.0, I tried to pull manually some docker images from the docker registry, with no success. Next step after some internet searches with no result, I listed the file present in the debian package for gitlab-runner and I noticed a sh script named mk-prebuilt-images.sh,this look like what I'm looking for,I tried my luck

sudo /usr/lib/gitlab-runner/mk-prebuilt-images.sh

Waited that to finish for some time I got the image gitlab-runner-helper:12.6.0 for docker installed on my machine.

Now that the image exists the steps actually run correctly, except I got another problem, my build use artifacts and the second step of the pipeline require artifacts produced by the first step, and this seems not supported by gitlab-runner without all gitlab setup, I tried to explore the runner to see if I could somehow reproduce the same beheavior with no success, so I worked around it using docker volumes and some additional steps in the build, So my command become:

gitlab-runner exec docker build-step-name --docker-volumes "/outside/docker/path:/data" 

and to my build I added to the step that produced the artifacts a copy to extract what is produced:

 - cp -r artifacts/path/* /data/

and to the step that use the artifacts a copy to put back the artifacts to the expected path:

 - mkdir -p artifacts/path
 - cp -r /data/* artifacts/path/

with this relative simple trick I could run all the steps I needed to try to reproduce the problem, but still didn't work, on my build I use kcov to run the tests and have a coverage report, it turn out that kcov uses some syscall not enabled in docker so to ran it is needed to pass some arguments to enable the support of the specific syscall, so finally my command become:

gitlab-runner exec docker build-step-name --docker-volumes "/outside/docker/path:/data" --docker-security-opt "seccomp=unconfined"

After all this testing I did notice that the build actually worked OK on my machine anyway, so I could not reproduce the problem, a lot of work and not much results, I will try other ways to track the problem

Posts

Setup Gitlab runner for run ci tests locally
18-01-2020
Tags: self_hosting

Check What You Run
23-12-2019
Tags: sh

Self Hosting Feed Reader Server
02-01-2018
Tags: self_hosting

Rust Lang Love
28-12-2017
Tags: coding, rust

Java Shell Script
13-05-2015
Tags: java, sh

Self Hosted Calendar Server With Radicale
02-05-2015
Tags: self_hosting

New Blog
02-05-2015
Tags: self_hosting

Primitive Boxing Problem
20-11-2012
Tags: java, coding

DDD Query Way
20-10-2012
Tags: java, ddd, coding

Refs

Twitter
Github
StackOverflow
GitLab

Projects

Object Query