[This is part 7/8 on GitLab Continuous Integration series]
CI stands for continuous integration. That is, build your app from the source code automatically upon pushing modification, then upload it to a repository. Some people include the testing part but in this post we don’t set that stage. We will:
- Set up a java app -the code is taken from a Spring Boot Microservice example– that contains a file directory structure, source for the app and the test, the pom file and its Dockerfile.
- Upload them to a GitLab project’s repository in our server.
- Set a gitlab-ci.yaml script that will take care of that and:
- Build a Docker image from the java application
- Push it to our private GitLab Container Registry
I. Get the ‘Java App’ code
The Java code example is the ‘gs-spring-boot' from the Spring Boot Guide. I initially just pasted the content of the files as its not my code and there is no longer a download link. But then I realized that after too many mistakes -with special characters and a minor fix- it is better to provide a download link to the code.
If you don’t want to use this code you also can use the hello world app from https://spring.io/quickstart and skip to Section II:
- Set an ‘App’ directory. I’m naming it “testjava” in a local ci_cd directory:
$ mkdir ~/Desarrollo/ci_cd/testjava
$ cd ~/Desarrollo/ci_cd/testjava
$ mkdir src && \
mkdir src/main && \
mkdir src/main/java && \
mkdir src/main/java/com && \
mkdir src/main/java/com/example && \
mkdir src/main/java/com/example/springboot
- This is the main file that will also list active elements.
$ vi src/main/java/com/example/springboot/Application.javaAdd
package com.example.springboot;
import java.util.Arrays;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
@SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
System.exit(0);
}
@Bean
public CommandLineRunner commandLineRunner(ApplicationContext ctx) {
return args -> {
System.out.println("Let's inspect the beans provided by Spring Boot:");
String[] beanNames = ctx.getBeanDefinitionNames();
int size = beanNames.length;
int i = 0;
Arrays.sort(beanNames);
for (String beanName : beanNames) {
i = i + 1;
System.out.println(i + "/" + size + " " + beanName + " : " + ctx.getBean(beanName).getClass().toString());
}
System.out.println("Listing ending.");
};
}
}
This is the controller that handles the web request to root “/”
Add
$ vi src/main/java/com/example/springboot/HelloController.java
package com.example.springboot;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.bind.annotation.RequestMapping;
@RestController
public class HelloController {
@RequestMapping("/")
public String index() {
return "Greetings from Spring Boot!";
}
}
- The unit tests should be in:
$ mkdir src/test && \
mkdir src/test/java && \
mkdir src/test/java/com && \
mkdir src/test/java/com/example && \
mkdir src/test/java/com/example/springboot
To make “a simple unit test that mocks the servlet request and response through your endpoint”:
$ vi src\test\java\com\example\springboot\HelloControllerTest.javaAdd
package com.example.springboot;
import static org.hamcrest.Matchers.equalTo;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
@SpringBootTest
@AutoConfigureMockMvc
public class HelloControllerTest {
@Autowired
private MockMvc mvc;
@Test
public void getHello() throws Exception {
mvc.perform(MockMvcRequestBuilders.get("/").accept(MediaType.APPLICATION_JSON))
.andExpect(status().isOk())
.andExpect(content().string(equalTo("Greetings from Spring Boot!")));
}
}
And a “use Spring Boot to write a simple full-stack integration test”:
$ vi src\test\java\com\example\springboot\HelloControllerIT.javaAdd
package com.example.springboot;
import static org.assertj.core.api.Assertions.*;
import java.net.URL;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.boot.web.server.LocalServerPort;
import org.springframework.http.ResponseEntity;
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class HelloControllerIT {
@LocalServerPort
private int port;
private URL base;
@Autowired
private TestRestTemplate template;
@BeforeEach
public void setUp() throws Exception {
this.base = new URL("http://localhost:" + port + "/");
}
@Test
public void getHello() throws Exception {
ResponseEntity<String> response = template.getForEntity(base.toString(),
String.class);
assertThat(response.getBody()).isEqualTo("Greetings from Spring Boot!");
}
}
- The project and configuration file:
$ vi pom.xmlAdd
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.3.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>spring-boot</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>spring-boot</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- tag::actuator[] -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- end::actuator[] -->
<!-- tag::tests[] -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- end::tests[] -->
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
II. Create the ‘Docker File’ code
A Dockerfile with the instructions to download a clean a Linux image with the JDK Java (from DockerHub), will copy the .jar application to it, and set the starting app.jar in the container is:
$ vi DockerfileAdd
FROM openjdk:8-jdk-alpine
VOLUME /tmp
COPY target/*.jar app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
III. Create the ‘Pipeline Script’
Our Continuous Integration script includes:
Two docker images:
docker:latest: Connects to a docker daemon, i.e., to rundocker build,docker run. It also contains the docker daemon but it’s not started as its entrypoint. It’s configured to connect totcp://docker:2375as a client.docker:dind: builds ondocker:latestand starts a docker daemon as its entrypoint. It is cleaner to writeservice: docker:dindinstead of having abefore_scriptto setupdockerd. Also you don’t have to figure out how to start & installdockerdproperly in your base image (if you are not usingdocker:latest.). Declaring the service in your.gitlab-ci.ymlalso lets you swap out the docker-in-docker easily if you know that your runner is mounting its/var/run/docker.sockinto your image. You can set the protected variableDOCKER_HOSTtounix:///var/run/docker.sockto get faster builds.
Three variables necessary to work with ‘dind’:
- Instruct docker to talk with the daemon inside the service with a network connection instead of the default /var/run/docker.sock socket. Try tcp://localhost:2375 instead for a kubernetes job.
- The default overlay2 filesystem is used (Docker 17.09+).
- Disable TLS certs as 18.09-dind+ will generate TLS certificates.
The script:
- There are two stages. The
buildstage uses maven to build and package our application into a.jarfile. Thepackagestage will build a docker image and run our container.
stages:
- build
- package
variables:
DOCKER_HOST: tcp://docker:2375/
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
build:
stage: build
image: maven
script:
- mvn clean
- mvn package
artifacts:
paths:
- target/*.jar
package:
stage: package
image: docker:latest
services:
- docker:dind
- name: docker:dind
alias: docker
before_script:
- echo 'Clave123' | docker login registry.example.com:5050 -u devguy --password-stdin
script:
- docker build -t registry.example.com:5050/devguy/dockerdemo:v1 .
- docker run registry.example.com:5050/devguy/dockerdemo:v1
- docker push registry.example.com:5050/devguy/dockerdemo:v1
Notes:
– Using the password in plain text in the script will show it in the logs, that is not right. But I’m just testing the build steps. In a following post I will use a variable and add a deployment.
– To log in to Docker Hub instead of our local repository, leave $DOCKER_REGISTRY empty or remove it.
IV. Push the code into a GitLab Project
Since GitLab 10.5 a private project will be created automatically when we push the code from a local repository.
- Make a local GIT repository from our code
$ git init
$ git add .
$ git commit -a -m "GitLab CI Docker demo 1.0"
$ git remote add origin http://devguy:Clave123@gitlab.example.com/devguy/dockerdemo.git
$ git push origin master
The answer received from that push command is:
Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 4 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 588 bytes | 588.00 KiB/s, done.
Total 6 (delta 4), reused 0 (delta 0), pack-reused 0
To http://gitlab.example.com/devguy/dockerdemo.git
34bfa69..4042119 master -> masterV. Result
Check the GitLab Project FOR THE IMAGE
The project will show our files:
Check the pipeline LOGS
To check the pipeline click:
‘Left Sidebar’ > CI/CD > Pipelines
See the result and the messages
If you click on the status or the steps, the log output will be shown (notice there is a raw icon to show large logs):
Getting source from Git repository
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/devguy/dockerdemo/.git/
...
Downloading artifacts from coordinator... ok id=219 responseStatus=200 OK token=zdyczEG8
Executing "step_script" stage of the job script
$ docker build -t dockerdemo:v1 .
Step 1/4 : FROM openjdk:8-jdk-alpine
...
Status: Downloaded newer image for openjdk:8-jdk-alpine
Step 2/4 : VOLUME /tmp
Removing intermediate container e98ef82ae693
Step 3/4 : COPY target/*.jar app.jar
Step 4/4 : ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
Removing intermediate container f9aa7aa89e8b
Successfully built a5eff243a8bb
Successfully tagged dockerdemo:v1
$ docker run dockerdemo:v1
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.3.RELEASE)
2021-02-06 04:40:49.352 INFO 1 --- [ main] com.example.springboot.Application : Starting Application v0.0.1-SNAPSHOT on 1fc6e70b2192 with PID 1 (/app.jar started by root in /)
...
2021-02-06 04:40:51.670 INFO 1 --- [extShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'applicationTaskExecutor'
Job succeededThe registry now holds our Docker image file. To see our tags click:
‘Left Sidebar’ > Packages & Registries > Container Registry
VI. Test the App
To download and run the app we can use in a terminal:
$ docker run --name testapp --rm -ti registry.example.com:5050/devguy/dockerdemo:v1As output you will see the list of all beans on your system:
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.3.RELEASE)
2021-12-12 22:14:50.858 INFO 1 --- [ main] com.example.springboot.Application : Starting Application v0.0.1-SNAPSHOT on 89f16eb558ed with PID 1 (/app.jar started by root in /)
2021-12-12 22:14:50.864 INFO 1 --- [ main] com.example.springboot.Application : No active profile set, falling back to default profiles: default
2021-12-12 22:14:52.515 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2021-12-12 22:14:52.540 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2021-12-12 22:14:52.540 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.37]
2021-12-12 22:14:52.652 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2021-12-12 22:14:52.652 INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1651 ms
2021-12-12 22:14:53.104 INFO 1 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
2021-12-12 22:14:53.365 INFO 1 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 2 endpoint(s) beneath base path '/actuator'
2021-12-12 22:14:53.421 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2021-12-12 22:14:53.454 INFO 1 --- [ main] com.example.springboot.Application : Started Application in 3.147 seconds (JVM running for 3.732)
Let's inspect the beans provided by Spring Boot:
1/216 application : class com.example.springboot.Application$$EnhancerBySpringCGLIB$$d7180133Reference
- The example code and explanation from: Setup CI/CD with GitLab by Mike Nöthiger [2019]. https://mikenoethiger.medium.com/setup-ci-cd-environment-with-gitlab-a00d7343fb1d
- Building Docker images with GitLab CI/CD https://docs.gitlab.com/ee/ci/docker/using_docker_build.html
- Changes to GitLab CI/CD and Docker in Docker with Docker 19.03 https://about.gitlab.com/blog/2019/07/31/docker-in-docker-with-docker-19-dot-03/
- Docker images explained in stack-overflow https://stackoverflow.com/questions/47280922/role-of-docker-in-docker-dind-service-in-gitlab-ci
Interesting IT Tip's 