Skip to content

Gitlab Runner

Some features of GitLab require a "runner" (in the sense of a "gopher" or a "minion"). A runner "registers" itself with a GitLab instance, and is given tasks to run. Tasks include running Continuous Integration (CI) builds, and building container images.

While a runner isn't strictly required to use GitLab, if you want to do CI, you'll need at least one. There are many ways to deploy a runner - this recipe focuses on the docker container model.

Ingredients

Ingredients

Existing:

1
2
3
4
1. [X] [Docker swarm cluster](/docker-swarm/design/) with [persistent shared storage](/docker-swarm/shared-storage-ceph/)
2. [X] [Traefik](/docker-swarm/traefik) configured per design
3. [X] DNS entry for the hostname you intend to use, pointed to your [keepalived](/docker-swarm/keepalived/) IP
4. [X] [GitLab](/recipes/gitlab) installation (see previous recipe)

Preparation

Setup data locations

We'll need several directories to bind-mount into our runner containers, so create them in /var/data/gitlab:

mkdir -p /var/data/gitlab/runners/{1,2}

{{ no such element: dict object['recipe'] }} Docker Swarm config

Create a docker swarm config file in docker-compose syntax (v3), something like the example below:

Fast-track with premix! 🚀

I automatically and instantly share (with my sponsors) a private "premix" git repository, which includes necessary docker-compose and env files for all published recipes. This means that sponsors can launch any recipe with just a git pull and a docker stack deploy 👍.

🚀 Update: Premix now includes an ansible playbook, so that sponsors can deploy an entire stack + recipes, with a single ansible command! (more here)

version: '3'

services:
  thing1:
    image: gitlab/gitlab-runner
    volumes:
    - /var/data/gitlab/runners/1:/etc/gitlab-runner
    networks:
    - internal

  thing2:
    image: gitlab/gitlab-runner
    volumes:
    - /var/data/gitlab/runners/2:/etc/gitlab-runner
    networks:
    - internal

networks:
  internal:
    driver: overlay
    ipam:
      config:
        - subnet: 172.16.23.0/24

Configure runners

From your GitLab UI, you can retrieve a "token" necessary to register a new runner. To register the runner, you can either create config.toml in each runner's bind-mounted folder (example below), or just docker exec into each runner container and execute gitlab-runner register to interactively generate config.toml.

Sample runner config.toml:

concurrent = 1
check_interval = 0

[[runners]]
  name = "myrunner1"
  url = "https://gitlab.example.com"
  token = "<long string here>"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "ruby:2.1"
    privileged = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
  [runners.cache]

Serving

Launch runners

Launch the GitLab Runner stack by running docker stack deploy gitlab-runner -c <path -to-docker-compose.yml>

Chef's notes 📓

  1. You'll note that I setup 2 runners. One is locked to a single project (this cookbook build), and the other is a shared runner. I wanted to ensure that one runner was always available to run CI for this project, even if I'd tied up another runner on something heavy-duty, like a container build. Customize this to your use case. ↩

  2. Originally I deployed runners in the same stack as GitLab, but I found that they would frequently fail to start properly when I launched the stack. I think that this was because the runners started so quickly (and GitLab starts sooo slowly!), that they always started up reporting that the GitLab instance was invalid or unavailable. I had issues with CI builds stuck permanently in a "pending" state, which were only resolved by restarting the runner. Having the runners deployed in a separate stack to GitLab avoids this problem. ↩

Tip your waiter (sponsor) 👏

Did you receive excellent service? Want to compliment the chef? (..and support development of current and future recipes!) Sponsor me on Github / Ko-Fi / Patreon, or see the contribute page for more (free or paid) ways to say thank you! 👏

Employ your chef (engage) 🤝

Is this too much of a geeky PITA? Do you just want results, stat? I do this for a living - I'm a full-time Kubernetes contractor, providing consulting and engineering expertise to businesses needing short-term, short-notice support in the cloud-native space, including AWS/Azure/GKE, Kubernetes, CI/CD and automation.

Learn more about working with me here.

Flirt with waiter (subscribe) 💌

Want to know now when this recipe gets updated, or when future recipes are added? Subscribe to the RSS feed, or leave your email address below, and we'll keep you updated.

Your comments? 💬