Run a CI with Docker in GitLab

[This is part 7/8 on GitLab Continuous Integration series]

We will build a Docker container from a Spring Boot Microservice and run it in our GitLab Registry

I. Create the ‘Java App’ code

The used Java example code dockerdemo is taken from a Spring Boot Guide. I’m listing the files and contents since there is no download link for just this files. I’m also omitting the gradle files since we will be building it with Maven.

If you don’t want to type so much code you can just use the hello world app from https://spring.io/quickstart and skip to Section II:

  1. Set an ‘App’ directory I’m naming it “testjava” in a local ci_cd directory:
$ mkdir ~/Desarrollo/ci_cd/testjava
$ cd ~/Desarrollo/ci_cd/testjava

$ mkdir src && \
  mkdir src/main && \
  mkdir src/main/java && \
  mkdir src/main/java/com && \
  mkdir src/main/java/com/example && \
  mkdir src/main/java/com/example/springboot
  1. This is the main file that will also list active elements.
$ vi src/main/java/com/example/springboot/Application.java

# Add
package com.example.springboot;

import java.util.Arrays;

import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;

@SpringBootApplication
public class Application {

	public static void main(String[] args) {
		SpringApplication.run(Application.class, args);
		System.exit(0);
	}

	@Bean
	public CommandLineRunner commandLineRunner(ApplicationContext ctx) {
		return args -> {

			System.out.println("Let's inspect the beans provided by Spring Boot:");

			String[] beanNames = ctx.getBeanDefinitionNames();
			int size = beanNames.length;
			int i = 0;

			Arrays.sort(beanNames);
			
			for (String beanName : beanNames) {
                i = i + 1;
				System.out.println(i + "/" + size + " " + beanName + " : " + ctx.getBean(beanName).getClass().toString());
			}

			System.out.println("Listing ending.");

		};
	}

}

This is the controller that handles the web request to root “/”

$ vi src/main/java/com/example/springboot/HelloController.java

# add
package com.example.springboot;

import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.bind.annotation.RequestMapping;

@RestController
public class HelloController {

	@RequestMapping("/")
	public String index() {
		return "Greetings from Spring Boot!";
	}

}
  1. The unit tests should be in:
$ mkdir src/test && \
  mkdir src/test/java && \
  mkdir src/test/java/com && \
  mkdir src/test/java/com/example && \
  mkdir src/test/java/com/example/springboot

To make “a simple unit test that mocks the servlet request and response through your endpoint”:

$ vi src\test\java\com\example\springboot\HelloControllerTest.java

# Add
package com.example.springboot;

import static org.hamcrest.Matchers.equalTo;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

import org.junit.jupiter.api.Test;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;

@SpringBootTest
@AutoConfigureMockMvc
public class HelloControllerTest {

	@Autowired
	private MockMvc mvc;

	@Test
	public void getHello() throws Exception {
		mvc.perform(MockMvcRequestBuilders.get("/").accept(MediaType.APPLICATION_JSON))
				.andExpect(status().isOk())
				.andExpect(content().string(equalTo("Greetings from Spring Boot!")));
	}
}

And a “use Spring Boot to write a simple full-stack integration test”:

$ vi src\test\java\com\example\springboot\HelloControllerIT.java

# Add
package com.example.springboot;

import static org.assertj.core.api.Assertions.*;

import java.net.URL;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.boot.web.server.LocalServerPort;
import org.springframework.http.ResponseEntity;

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class HelloControllerIT {

	@LocalServerPort
	private int port;

	private URL base;

	@Autowired
	private TestRestTemplate template;

    @BeforeEach
    public void setUp() throws Exception {
        this.base = new URL("http://localhost:" + port + "/");
    }

    @Test
    public void getHello() throws Exception {
        ResponseEntity<String> response = template.getForEntity(base.toString(),
                String.class);
        assertThat(response.getBody()).isEqualTo("Greetings from Spring Boot!");
    }
}
  1. The project and configuration file:
$ vi pom.xml

# Add
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.3.3.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.example</groupId>
	<artifactId>spring-boot</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>spring-boot</name>
	<description>Demo project for Spring Boot</description>

	<properties>
		<java.version>1.8</java.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>

		<!-- tag::actuator[] -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-actuator</artifactId>
		</dependency>
		<!-- end::actuator[] -->

		<!-- tag::tests[] -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
			<exclusions>
				<exclusion>
					<groupId>org.junit.vintage</groupId>
					<artifactId>junit-vintage-engine</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<!-- end::tests[] -->
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>

</project>

II. Create the ‘Docker File’ code

A Dockerfile with the instructions to download a clean a Linux image with the JDK Java (from DockerHub), will copy the .jar application to it, and set the starting app.jar in the container is:

$ vi Dockerfile

# Add
FROM openjdk:8-jdk-alpine
VOLUME /tmp
COPY target/*.jar app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

III. Create the ‘Pipeline Script’

Our Continuous Integration script includes:

Two docker images:

  • docker:latest: Connects to a docker daemon, i.e., to run docker build, docker run. It also contains the docker daemon but it’s not started as its entrypoint. It’s configured to connect to tcp://docker:2375 as a client.
  • docker:dind: builds on docker:latest and starts a docker daemon as its entrypoint. It is cleaner to write service: docker:dind instead of having a before_script to setup dockerd. Also you don’t have to figure out how to start & install dockerd properly in your base image (if you are not using docker:latest.). Declaring the service in your .gitlab-ci.yml also lets you swap out the docker-in-docker easily if you know that your runner is mounting its /var/run/docker.sock into your image. You can set the protected variable DOCKER_HOST to unix:///var/run/docker.sock to get faster builds.

Three variables necessary to work with ‘dind’:

  • Instruct docker to talk with the daemon inside the service with a network connection instead of the default /var/run/docker.sock socket. Try tcp://localhost:2375 instead for a kubernetes job.
  • The default overlay2 filesystem is used (Docker 17.09+).
  • Disable TLS certs as 18.09-dind+ will generate TLS certificates.

The script:

  1. There are two stages. The build stage uses maven to build and package our application into a .jar file. The package stage will build a docker image and run our container.
stages:
  - build
  - package

variables:
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""

build:
  stage: build
  image: maven
  script:
    - mvn clean
    - mvn package
  artifacts:
    paths:
      - target/*.jar

package:
  stage: package
  image: docker:latest
  services:
    - docker:dind
    - name: docker:dind
      alias: docker
  before_script:
    - echo 'Clave123' | docker login registry.example.com:5050 -u devguy --password-stdin
  script:
    - docker build -t registry.example.com:5050/devguy/dockerdemo:v1 .
    - docker run registry.example.com:5050/devguy/dockerdemo:v1
    - docker push registry.example.com:5050/devguy/dockerdemo:v1

Note: To log in to Docker Hub, leave $DOCKER_REGISTRY empty or remove it.


IV. Push the code into a GitLab Project

Since GitLab 10.5 a private project will be created automatically when we push the code from a local repository.

  1. Make a local GIT repository from our code
$ git init
$ git add .
$ git commit -a -m "GitLab CI Docker demo 1.0"
$ git remote add origin http://devguy:Clave123@gitlab.example.com/devguy/dockerdemo.git
$ git push origin master

The answer received from that push command is:

Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 4 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 588 bytes | 588.00 KiB/s, done.
Total 6 (delta 4), reused 0 (delta 0), pack-reused 0
To http://gitlab.example.com/devguy/dockerdemo.git
   34bfa69..4042119  master -> master

V. Result

Check the GitLab Project

The project will show our files:

Check the pipeline

To check the pipeline click: ‘Left Sidebar’ > CI/CD > Pipelines

See the result and the messages

If you click on the status or the steps, the log output will be shown (notice there is a raw icon to show large logs):

Getting source from Git repository
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/devguy/dockerdemo/.git/
...
Downloading artifacts from coordinator... ok        id=219 responseStatus=200 OK token=zdyczEG8
Executing "step_script" stage of the job script
$ docker build -t dockerdemo:v1 .
Step 1/4 : FROM openjdk:8-jdk-alpine
...
Status: Downloaded newer image for openjdk:8-jdk-alpine
Step 2/4 : VOLUME /tmp
Removing intermediate container e98ef82ae693
Step 3/4 : COPY target/*.jar app.jar
Step 4/4 : ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
Removing intermediate container f9aa7aa89e8b
Successfully built a5eff243a8bb
Successfully tagged dockerdemo:v1
$ docker run dockerdemo:v1
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.3.3.RELEASE)
2021-02-06 04:40:49.352  INFO 1 --- [           main] com.example.springboot.Application       : Starting Application v0.0.1-SNAPSHOT on 1fc6e70b2192 with PID 1 (/app.jar started by root in /)
...
2021-02-06 04:40:51.670  INFO 1 --- [extShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'
Job succeeded

The registry now holds our Docker image file. To see our tags click:

‘Left Sidebar’ > Packages & Registries > Container Registry


Reference

Continuous Integration (CI)

[This is part 6/8 on GitLab Continuous Integration series]

Continuous Integration (CI) functionality in servers like GitLab (programmed in Ruby), GitHub (programmed in Ruby), Bitbucket (programed in Python), Jenkins (programmed in Java), Travis (programmed in Ruby) or CircleCi provide us with the capability automating our code building, testing and even deploying tasks.

In CI/CD Examples at GitLab you can find examples for several use cases that illustrate usage with Webdriver, NPM, PHP, Python, Ruby.

We have so far:

  1. Set up our GitLab Server that work as a Controller.
  2. Set up our local GitLab Container Registry. [Optional]
  3. Installed and registered a GitLab runner or Executor.

We can now set up a CI pipeline that contains our automation script.


I. Setup CI Pipelines

In our first example we will just ‘echo’ commands. You can identify in the script:

  • Four jobs. Jobs “define what to do”. build-job:, test-job1:, test-job2:, deploy-prod:. Job1 and Job2 will run in parallel.
  • Three stages. Stages “define when to run the jobs”. stage:build, stage:test, stage:deploy.

Notice that it uses two predefined variables that in this case the GitLab Server creates and sets on each run: $GITLAB_USER_LOGIN and $CI_COMMIT_BRANCH.

build-job:
  stage: build
  script:
    - echo "Hello, $GITLAB_USER_LOGIN!"

test-job1:
  stage: test
  script:
    - echo "This job tests something"

test-job2:
  stage: test
  script:
    - echo "This job tests something, but takes more time than test-job1."
    - echo "After the echo commands complete, it runs the sleep command for 20 seconds"
    - echo "which simulates a test that runs 20 seconds longer than test-job1"
    - sleep 20

deploy-prod:
  stage: deploy
  script:
    - echo "This job deploys something from the $CI_COMMIT_BRANCH branch."

II. Create the CI file

Now create the Pipeline/CI file .gitlab-ci.yml using the terminal or the Web GitLab Server:

II.1 Using the terminal

To add the file we can use a terminal or as shown in the next section, use the Web app.

  1. Open a terminal, create a directory for your code root directory
$ mkdir ~/Desarrollo/ci_cd/gitlabtest
$ cd ~/Desarrollo/ci_cd/gitlabtest
  1. Initialize the local repository and associate the code with a repository (gitlabtest) that will exist in the GitLab Server.
$ git init
$ git remote add origin http://devguy:Clave123@gitlab.example.com/devguy/gitlabtest.git

Note the use of the developer credentials in the URL

  1. Create a ‘hidden’ file .gitlab-ci.yml
$ vi .gitlab-ci.yml
  1. Add the previous code and save it.
  2. Add the CI file, Commit and Push the code there:
$ git add .gitlab-ci.yml
$ git commit -m "Adding CI pipeline"
$ git push origin master

A repository will be created in the GitLab Server and the pipeline will starts when the changes are committed.

The pipeline starts when the changes are committed.

II.2 Use the GitLab Server

  1. Access our GitLab Server in http://example.gitlab.com log in as devguy/Clave123
  2. Go to Project overview > Details.
  3. Above the file list, select the branch you want to commit to (Master) /project/ Click ‘+’, select New file
  4. Type as name: .gitlab-ci.yml
  5. Add the previous script.
  6. Click Commit changes.

The pipeline starts when the changes are committed.


III. Check our CI job

Get to your project page in the server to see the result. Access the project page in the server with user with a maintainer rol (devguy).

The project would be like. You can see the project name, CI Yaml file:

Go to CI/CD -> Pipelines.

You will see the result on ran pipelines.

And into the first job:

Notes:If the project’s visibility it’s private (default if the project is created from a terminal) Edit the project. Change the switch from private to public. Its’ easier to work with these first projects and tools if we avoid asking for authorization.
If there is a Pipeeline Job Error, first check permissions, add your account to the project even if you are the Administrator or it will give this error.
To validate your .gitlab-ci.yml file, use the CI Lint tool, which is available in every project.
You can also use CI/CD configuration visualization to view a graphical representation of your .gitlab-ci.yml file.

Reference

GitLab Runners

[This is part 5/8 on GitLab Continuous Integration series]

The runners are the actual elements that do the work in a CI setup. They are assigned CI/CD jobs by the server according to a pipeline script that if all is set properly runs automatically when a commit is pushed to the server or manually if needed.


I. What we need

For automation using GitLab CI we need:

  1. To install an executor or GitLab runner that will do the actual work aka jobs. To distribute load they should be installed on a machines that are separated from the Server
  2. Then register that GitLab runner to the GitLab Server or coordinator. In the server the runners can be configured so they can be assigned all or specific pipelines.
  3. Verify that running ‘CI pipelines’ are enabled in our server and project.
  4. Add a pipeline script with stage our jobs. It is usually located in the project’s root directory. It is normally called .gitlab-ci.yml. Yes, a hidden file.

Executors

As the documentation states, there are several types of runners that can be used to run your builds for different scenarios.

ExecutorSSHShellVirtualboxDockerKubernetes
Clean build environment for every build
Reuse previous clone if it exists
Runner file system access protected
Complicated build environments✗ (1)✓ (2)
Debugging build problemseasyeasyhardmediummedium
Noteruns each build in an isolated containerCreates a Pod for each Job

And each one supports different features:

ExecutorSSHShellVirtualBoxDockerKubernetes
Secure Variables
GitLab Runner Exec command
gitlab-ci.yml: image
gitlab-ci.yml: services
gitlab-ci.yml: cache
gitlab-ci.yml: artifacts
Passing artifacts between stages
Use GitLab Container Registry private imagesn/an/an/a
Interactive Web terminal✓ (UNIX)✓ (Planned for Helm)
Notes: 1 – Manual install of dependencies. 2 – Vagrant. SSH is the least supported.

The recommended type is Docker as it will allow us to use Docker in Docker (DinD) services.


II. Install a GitLab Runner

To use GitLab Runner we will install it as a Docker container. We can use Linux’s apt or download it. In my case, to follow the recommendation to install it in another machine than the server, I installed it in my laptop computer.

$ cd ~/Descargas<br>
$ sudo apt update

This following is the correct command but it didn’t work at the end on 2020 in my Ubuntu/Debian 20.10 Linux Laptop. It might work by the time you are reading it:

$ sudo curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash

Note: Still by January 2021 there is no code for installing a gitlab-runner in ‘Groovy-Gorilla’ (Ubuntu 20.10). The error shown is:

Error as Groovy-Gorilla Ubuntu doesn't have a candidate.
Unable to download repo config from: https://packages.gitlab.com/install/repositories/runner/gitlab-runner/config_file.list?os=Ubuntu&dist=groovy&source=script
You can override the OS detection by setting 'os=' and 'dist=' prior to running this script.
You can find a list of supported OSes and distributions on our website: https://packages.gitlab.com/docs#os_distro_version

We can see in the list of official GitLab Runner repositories: Xenial (16.04), Bionic (18.04) and Focal (20.04). Considering tat the forums are full of reports for 19.03 we can assume that our 20.10 version will show them also. So we have Focal or Bionic as candidates. Now we check the availability of our the Docker we will use in our CI Pipeline. For that DinD, the hub.docker list contains (20.10.3-dind, 20.10-dind, 20-dind, dind, 20.10.3-dind-rootless, 20.10-dind-rootless, 20-dind-rootless, dind-rootless, 19.03.15-dind, 19.03-dind, 19-dind, 19.03.15-dind-rootless, 19.03-dind-rootless, 19-dind-rootles).

In that Docker Hub page you can read two warnings:

  • Although running Docker inside Docker is generally not recommended, there are some legitimate use cases, such as development of Docker itself. => We have to insist 🙂
  • Starting in 18.09+ the dind image will automatically generate TLS certificates in the directory specified by the DOCKER_TLS_CERTDIR environment variable. It is recommended to enable TLS by setting the variable to an appropriate value (-e DOCKER_TLS_CERTDIR=/certs) => So we have to remember to disable that in our runner in the next section

After updating the command with an available distribution (focal=Ubuntu 20.04):

$ sudo curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo os=ubuntu dist=focal bash

# We are advised to use this for Debian Buster (Ubuntu 20.10) so it can create empty folders:
# (-E preserves environment variables) 
$ export GITLAB_RUNNER_DISABLE_SKEL=true; sudo -E apt install gitlab-runner

Check the Runner

To check the runner we can list them:

$ sudo gitlab-runner list
# Answer
Runtime platform             arch=amd64 os=linux pid=21747 revision=8fa89735 version=13.6.0
Listing configured runners   ConfigFile=/etc/gitlab-runner/config.toml
docker-runner                Executor=docker Token=o63mEQt4q-RJnj3yQNhW URL=http://gitlab.example.com/

get its version:

$ sudo gitlab-runner --version
# Answer
Version:      13.7.0
Git revision: 943fc252
Git branch:   13-7-stable
GO version:   go1.13.8
Built:        2020-12-21T13:47:06+0000
OS/Arch:      linux/amd64

and ask for a quick verification:

$ sudo gitlab-runner verify
# Answer
Runtime platform               arch=amd64 os=linux pid=70447 revision=775dd39d version=13.8.0
Running in system-mode.
Verifying runner... is alive   runner=xmWbMf4w

II. Register the Runner

Registering a Runner will bind the GitLab Runner with the GitLab Instance. There are three types of bindings:

  • Shared that are available and runs jobs from all unassigned projects. In cloud services they are time limited. This is the one we will be using in the example.
  • Group or
  • Project-specific these can be more secure as environment variables are only shared with our code.

In the use the Docker executor Gitlab page we can find additional warnings:

  • Each job is in a clean environment without the past history, there’s no caching of layers.
  • A docker-privileged executor as --privileged is required for Docker-in-Docker. That is what we will use to package our app as a docker image and upload them to a repository. Bun if you enable --docker-privilegedyou are effectively disabling all of the security mechanisms of containers and exposing your host to privilege escalation.
  • By default, Docker 17.09 and higher uses --storage-driver overlay2 which is the recommended storage driver.
  • Since the docker:19.03.12-dind container and the runner container don’t share their root file system, the job’s working directory can be used as a mount point for child containers. For example, if you have files you want to share with a child container, you may create a subdirectory under /builds/$CI_PROJECT_PATH and use it as your mount point (check issue #41227):
variables:
  MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt

script:
  - mkdir -p "$MOUNT_POINT"
  - docker run -v "$MOUNT_POINT:/mnt" my-docker-image

Example project using this approach: https://gitlab.com/gitlab-examples/docker.

Back into the post. We are configuring an easy to use insecure CI Server. And we will be using the privileged attribute that can be exploited by running software to gain access to your machine. But we know what we are doing and love living dangerously. 🙂 (No thats thats a joke, we have our Server behind our home firewall).

To register the runner we need data from the ‘shared runner’ page. Log in as Admin user into the server and go to: ‘Admin area’ > Overview > Runners Copy url & registration-token, and paste those values in the following command:

$ sudo gitlab-runner register \
  --non-interactive \
  --url "http://gitlab.example.com/" \
  --registration-token "QR6jYXGZdd7xHpsy1K5b" \
  --executor "docker" \
  --docker-image docker:stable \
  --description "docker-runner" \
  --docker-privileged \
  --tag-list "master" \
  --run-untagged="true" \
  --locked="false"
# access-level was added for version 12
# Answer
Runtime platform      arch=amd64 os=linux pid=21631 revision=8fa89735 version=13.6.0
Running in system-mode.
Registering runner... succeeded      runner=7f4nVsxX
Runner registered successfully. Feel free to start it, but if its running already the config should be automatically reloaded! 

Note: You can use an interactive version, that is started with:

$ sudo gitlab-runner register
# Answers you will need to provide:
http://gitlab.example.com/
QR6jYXGZdd7xHpsy1K5b
gitlab-runner
<Enter>
docker
docker:stable

Check if the Runner is registered

To list runners registered and available in the GitLab Server:

  • As a user, go to Settings > CI/CD and expand the Runners section.
  • As an administrator (root), for a Shared Runner go to ‘Admin Area’ (click wrench icon on toolbar) > Overview > Runners

Fine Tuning the System

As it is, the runner can run common shell commands like maven clean or maven build, but it will break when using the Docker file system to send files (artifacts) between stages or posting files to the GitLab Docker repository. We need to make some configuration modifications so edit the config file to add a few lines and modify a Docker tag line:

  • add ‘clone_url’ that holds the IP of the GitLab server as a backup if it can’t find our ‘gitlab.example.com’ url:
clone_url = "http://192.168.1.221/"
  • add also extra_hosts with the GitLab Server and repository names and IPs so the runner can find them:
extra_hosts = ["gitlab.example.com:192.168.1.221","registry.example.com:192.168.1.221"]
  • Lets share the docker insecure registry configuration with our Docker that runs our jobs in the CI:
volumes = ["/etc/docker/daemon.json:/etc/docker/daemon.json","/cache"]
  • In the ‘image’ change “docker:latest” to:
image = "docker:stable"
  • Lets turn off TLS for our insecure setup:
  environment = ["GIT_SSL_NO_VERIFY=true", "DOCKER_TLS_CERTDIR="]

The resulting file will be:

$ sudo cat /etc/gitlab-runner/config.toml
# Content

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "docker-runner"
  url = "http://gitlab.example.com/"
  clone_url = "http://192.168.1.221/"
  token = "xmWbMf4wYFsPPGq7gJFR"
  executor = "docker"
  environment = ["DOCKER_TLS_CERTDIR=","GIT_SSL_NO_VERIFY=true"]
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
    Insecure = true
  [runners.docker]
    tls_verify = false
    image = "docker:stable"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/etc/docker/daemon.json:/etc/docker/daemon.json","/cache"]
    shm_size = 0
    extra_hosts = ["gitlab.example.com:192.168.1.221","registry.example.com:192.168.1.221"]

Don’t forget to restart the runner service to update it:

$ sudo service gitlab-runner restart

That’s it. You’ve got a CI server ready to work.


Remove a GitLab Runner

If later on you need to remove a runner you can click on the Runner page of the server or:

  1. First unregister it in the GitLab Server or in the CLI:
$ sudo gitlab-runner unregister --token ugNjkgTFHcndF4_btFuQ --url http://gitlab.example.com/
# Answer
Runtime platform                                    arch=arm os=linux pid=19719 revision=8fa89735 version=13.6.0
Running in system-mode.                                           
Unregistering runner from GitLab succeeded          runner=ugNjkgTF
Updated /etc/gitlab-runner/config.toml
  1. Then delete it so it won’t appear in the list:
$ sudo gitlab-runner list
# Answer
Runtime platform   arch=arm os=linux pid=19991 revision=8fa89735 version=13.6.0
Listing configured runners ConfigFile=/etc/gitlab-runner/config.toml

# To verify delete
$ sudo gitlab-runner verify --delete
$ sudo service gitlab-runner restart

# To uninstall the runner executable
# sudo apt remove gitlab-runner