Tag Archives: Java DEMO CI

Run a CI with Docker in GitLab

Posted on by
Reply

[This is part 7/8 on GitLab Continuous Integration series]

CI stands for continuous integration. That is, build your app from the source code automatically upon pushing modification, then upload it to a repository. Some people include the testing part but in this post we don’t set that stage. We will:

  1. Set up a java app -the code is taken from a Spring Boot Microservice example– that contains a file directory structure, source for the app and the test, the pom file and its Dockerfile.
  2. Upload them to a GitLab project’s repository in our server.
  3. Set a gitlab-ci.yaml script that will take care of that and:
    • Build a Docker image from the java application
    • Push it to our private GitLab Container Registry

I. Get the ‘Java App’ code

The Java code example is the ‘gs-spring-boot' from the Spring Boot Guide. I initially just pasted the content of the files as its not my code and there is no longer a download link. But then I realized that after too many mistakes -with special characters and a minor fix- it is better to provide a download link to the code.

If you don’t want to use this code you also can use the hello world app from https://spring.io/quickstart and skip to Section II:

  1. Set an ‘App’ directory. I’m naming it “testjava” in a local ci_cd directory:
$ mkdir ~/Desarrollo/ci_cd/testjava
$ cd ~/Desarrollo/ci_cd/testjava

$ mkdir src && \
  mkdir src/main && \
  mkdir src/main/java && \
  mkdir src/main/java/com && \
  mkdir src/main/java/com/example && \
  mkdir src/main/java/com/example/springboot
  1. This is the main file that will also list active elements.
$ vi src/main/java/com/example/springboot/Application.java

Add

package com.example.springboot;

import java.util.Arrays;

import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;

@SpringBootApplication
public class Application {

	public static void main(String[] args) {
		SpringApplication.run(Application.class, args);
		System.exit(0);
	}

	@Bean
	public CommandLineRunner commandLineRunner(ApplicationContext ctx) {
		return args -> {

			System.out.println("Let's inspect the beans provided by Spring Boot:");

			String[] beanNames = ctx.getBeanDefinitionNames();
			int size = beanNames.length;
			int i = 0;

			Arrays.sort(beanNames);
			
			for (String beanName : beanNames) {
                i = i + 1;
				System.out.println(i + "/" + size + " " + beanName + " : " + ctx.getBean(beanName).getClass().toString());
			}

			System.out.println("Listing ending.");

		};
	}

}

This is the controller that handles the web request to root “/”

Add

$ vi src/main/java/com/example/springboot/HelloController.java
package com.example.springboot;

import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.bind.annotation.RequestMapping;

@RestController
public class HelloController {

	@RequestMapping("/")
	public String index() {
		return "Greetings from Spring Boot!";
	}

}
  1. The unit tests should be in:
$ mkdir src/test && \
  mkdir src/test/java && \
  mkdir src/test/java/com && \
  mkdir src/test/java/com/example && \
  mkdir src/test/java/com/example/springboot

To make “a simple unit test that mocks the servlet request and response through your endpoint”:

$ vi src\test\java\com\example\springboot\HelloControllerTest.java

Add

package com.example.springboot;

import static org.hamcrest.Matchers.equalTo;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

import org.junit.jupiter.api.Test;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;

@SpringBootTest
@AutoConfigureMockMvc
public class HelloControllerTest {

	@Autowired
	private MockMvc mvc;

	@Test
	public void getHello() throws Exception {
		mvc.perform(MockMvcRequestBuilders.get("/").accept(MediaType.APPLICATION_JSON))
				.andExpect(status().isOk())
				.andExpect(content().string(equalTo("Greetings from Spring Boot!")));
	}
}

And a “use Spring Boot to write a simple full-stack integration test”:

$ vi src\test\java\com\example\springboot\HelloControllerIT.java

Add

package com.example.springboot;

import static org.assertj.core.api.Assertions.*;

import java.net.URL;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.boot.web.server.LocalServerPort;
import org.springframework.http.ResponseEntity;

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class HelloControllerIT {

	@LocalServerPort
	private int port;

	private URL base;

	@Autowired
	private TestRestTemplate template;

    @BeforeEach
    public void setUp() throws Exception {
        this.base = new URL("http://localhost:" + port + "/");
    }

    @Test
    public void getHello() throws Exception {
        ResponseEntity<String> response = template.getForEntity(base.toString(),
                String.class);
        assertThat(response.getBody()).isEqualTo("Greetings from Spring Boot!");
    }
}
  1. The project and configuration file:
$ vi pom.xml

Add

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.3.3.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.example</groupId>
	<artifactId>spring-boot</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>spring-boot</name>
	<description>Demo project for Spring Boot</description>

	<properties>
		<java.version>1.8</java.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>

		<!-- tag::actuator[] -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-actuator</artifactId>
		</dependency>
		<!-- end::actuator[] -->

		<!-- tag::tests[] -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
			<exclusions>
				<exclusion>
					<groupId>org.junit.vintage</groupId>
					<artifactId>junit-vintage-engine</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<!-- end::tests[] -->
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>

</project>

II. Create the ‘Docker File’ code

A Dockerfile with the instructions to download a clean a Linux image with the JDK Java (from DockerHub), will copy the .jar application to it, and set the starting app.jar in the container is:

$ vi Dockerfile

Add

FROM openjdk:8-jdk-alpine
VOLUME /tmp
COPY target/*.jar app.jar
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

III. Create the ‘Pipeline Script’

Our Continuous Integration script includes:

Two docker images:

  • docker:latest: Connects to a docker daemon, i.e., to run docker build, docker run. It also contains the docker daemon but it’s not started as its entrypoint. It’s configured to connect to tcp://docker:2375 as a client.
  • docker:dind: builds on docker:latest and starts a docker daemon as its entrypoint. It is cleaner to write service: docker:dind instead of having a before_script to setup dockerd. Also you don’t have to figure out how to start & install dockerd properly in your base image (if you are not using docker:latest.). Declaring the service in your .gitlab-ci.yml also lets you swap out the docker-in-docker easily if you know that your runner is mounting its /var/run/docker.sock into your image. You can set the protected variable DOCKER_HOST to unix:///var/run/docker.sock to get faster builds.

Three variables necessary to work with ‘dind’:

  • Instruct docker to talk with the daemon inside the service with a network connection instead of the default /var/run/docker.sock socket. Try tcp://localhost:2375 instead for a kubernetes job.
  • The default overlay2 filesystem is used (Docker 17.09+).
  • Disable TLS certs as 18.09-dind+ will generate TLS certificates.

The script:

  1. There are two stages. The build stage uses maven to build and package our application into a .jar file. The package stage will build a docker image and run our container.
stages:
  - build
  - package

variables:
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""

build:
  stage: build
  image: maven
  script:
    - mvn clean
    - mvn package
  artifacts:
    paths:
      - target/*.jar

package:
  stage: package
  image: docker:latest
  services:
    - docker:dind
    - name: docker:dind
      alias: docker
  before_script:
    - echo 'Clave123' | docker login registry.example.com:5050 -u devguy --password-stdin
  script:
    - docker build -t registry.example.com:5050/devguy/dockerdemo:v1 .
    - docker run registry.example.com:5050/devguy/dockerdemo:v1
    - docker push registry.example.com:5050/devguy/dockerdemo:v1

Notes:
– Using the password in plain text in the script will show it in the logs, that is not right. But I’m just testing the build steps. In a following post I will use a variable and add a deployment.
– To log in to Docker Hub instead of our local repository, leave $DOCKER_REGISTRY empty or remove it.

IV. Push the code into a GitLab Project

Since GitLab 10.5 a private project will be created automatically when we push the code from a local repository.

  1. Make a local GIT repository from our code
$ git init
$ git add .
$ git commit -a -m "GitLab CI Docker demo 1.0"
$ git remote add origin http://devguy:Clave123@gitlab.example.com/devguy/dockerdemo.git
$ git push origin master

The answer received from that push command is:

Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 4 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 588 bytes | 588.00 KiB/s, done.
Total 6 (delta 4), reused 0 (delta 0), pack-reused 0
To http://gitlab.example.com/devguy/dockerdemo.git
   34bfa69..4042119  master -> master

V. Result

Check the GitLab Project FOR THE IMAGE

The project will show our files:

Check the pipeline LOGS

To check the pipeline click:
‘Left Sidebar’ > CI/CD > Pipelines

See the result and the messages

If you click on the status or the steps, the log output will be shown (notice there is a raw icon to show large logs):

Getting source from Git repository
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/devguy/dockerdemo/.git/
...
Downloading artifacts from coordinator... ok        id=219 responseStatus=200 OK token=zdyczEG8
Executing "step_script" stage of the job script
$ docker build -t dockerdemo:v1 .
Step 1/4 : FROM openjdk:8-jdk-alpine
...
Status: Downloaded newer image for openjdk:8-jdk-alpine
Step 2/4 : VOLUME /tmp
Removing intermediate container e98ef82ae693
Step 3/4 : COPY target/*.jar app.jar
Step 4/4 : ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
Removing intermediate container f9aa7aa89e8b
Successfully built a5eff243a8bb
Successfully tagged dockerdemo:v1
$ docker run dockerdemo:v1
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.3.3.RELEASE)
2021-02-06 04:40:49.352  INFO 1 --- [           main] com.example.springboot.Application       : Starting Application v0.0.1-SNAPSHOT on 1fc6e70b2192 with PID 1 (/app.jar started by root in /)
...
2021-02-06 04:40:51.670  INFO 1 --- [extShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'
Job succeeded

The registry now holds our Docker image file. To see our tags click:

‘Left Sidebar’ > Packages & Registries > Container Registry

VI. Test the App

To download and run the app we can use in a terminal:

$ docker run --name testapp --rm -ti registry.example.com:5050/devguy/dockerdemo:v1

As output you will see the list of all beans on your system:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.3.3.RELEASE)

2021-12-12 22:14:50.858  INFO 1 --- [           main] com.example.springboot.Application       : Starting Application v0.0.1-SNAPSHOT on 89f16eb558ed with PID 1 (/app.jar started by root in /)
2021-12-12 22:14:50.864  INFO 1 --- [           main] com.example.springboot.Application       : No active profile set, falling back to default profiles: default
2021-12-12 22:14:52.515  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2021-12-12 22:14:52.540  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-12-12 22:14:52.540  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.37]
2021-12-12 22:14:52.652  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-12-12 22:14:52.652  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1651 ms
2021-12-12 22:14:53.104  INFO 1 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2021-12-12 22:14:53.365  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 2 endpoint(s) beneath base path '/actuator'
2021-12-12 22:14:53.421  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2021-12-12 22:14:53.454  INFO 1 --- [           main] com.example.springboot.Application       : Started Application in 3.147 seconds (JVM running for 3.732)
Let's inspect the beans provided by Spring Boot:
1/216 application : class com.example.springboot.Application$$EnhancerBySpringCGLIB$$d7180133

Reference

Advertisement