Run a CD pipeline for MariaDB and Web App.

This post is an add-on to the series as I tried to do a “LAMP automated deployment from a GitLab repository” using Ansible but the information I found online was too difficult to understand or was outdated.

Lets start with a LAMP Ansible playbook example from the official example repository and let’s modify it so it can upload it to a GitLab Repository and execute the deployment to set up a Web server in one machine and a MariaDB Server in other (two Raspberry Pi’s).

There are two things that we have to consider:

GitLab currently doesn’t have built-in support for managing SSH keys in a build environment (where the GitLab Runner runs). The most widely supported method is to inject an SSH key into our .gitlab-ci.yml.

  • A common way to make secrets available to containers is using a volume mounted into the container at runtime. Volumes can be added to the executor through the GitLab Runner configuration.
  • But the most widely supported method so GitLab can provide them is to inject the SSH key into your build environment (in the.gitlab-ci.yml). This solution works with any type of executor.
    We will supply the information from Secure Variables.

The docker image we configured in the runner it is a basic Docker image. The appropriate way is to use an image that has the necessary software, that is Ansible, Python and the MySql utilities to configure the database.

I browsed what was available in This one from mullnerz has Ansible 2.9 and Python 3.8. That is good enough for our project. As shown in:

$ docker run --rm --name ansible mullnerz/ansible-playbook ansible --version
ansible 2.9.6
python version = 3.8.2 (default, Feb 29 2020, 17:03:31) [GCC 9.2.0]

I. Create a GitLab Project

We need a repository for our code the GitLab Server. Lets create one:

  • Browse to our local
    or your GitLab, GitHub code service.
    We use devguy/Clave123 to log in (Credentials shown as reference as it is used in the url)
  • Click on right blue button: New project’ > Create blank project
    Project name: infra
    Visibility level: Public
    Create readme file: no
    Click on ‘Create project’ button
  • We can get access to it opening the url:

II. Check The Runner

We have set up previously a Docker executor to run our CI/CD jobs. To verify that the Docker Runner is still running as we configured it:

[On the PC]

$ sudo gitlab-runner verify
Runtime platform arch=amd64 os=linux pid=50507 revision=7a6612da version=13.12.0
Running in system-mode.
Verifying runner... is alive runner=ejJb1t1N

It seems ok. Let’s continue

III. Set The Log In Keys

Gitlab writes that: “When your CI/CD jobs run inside Docker containers and you want to deploy your code in a private server, you need a way to access it. In this case, you can use an SSH key pair.”

Create two new CI/CD variables.

  1. Access our GitLab Server in
    log in as devguy/Clave123
    (This are the url and credentials used in previous posts)
    Go to Project > infra
    Then Settings > CI/CD
    Expand Variables, and click ‘Add Variable’ button
  2. As Key enter the name SSH_PRIVATE_KEY
    and in the Value field paste the content of your private key
    To get the private key you can use the following command (copy the whole answer):
    $ cat ~/.ssh/id_rsa

    In all variables set as Type: Variable
    Environment scope: all
    In Flags uncheck Protect and Mask
    Click Create variable button
  3. The second will hosts a list of all your hosts:
    value: output of the command ssh-keyscan <IPS-list>
    (It will be a big string. Copy the answer):
    $ ssh-keyscan

III. Modifying the code

We need to create one file and modify an existing one.

Create the .gitlab-ci.yml pipeline file (the official usage example is here).

$ cd <your-root-dir>/Ansible/105_IaC_with_GitLab/Infra
$ vi .gitlab-ci.yml

Add as content:

    name: mullnerz/ansible-playbook
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y )'
  - eval $(ssh-agent -s)
  - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh
  - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
  - chmod 644 ~/.ssh/known_hosts
  stage: deploy
    - 'ansible-playbook -i hosts site.yml'

Notes to the before_script section:

  1. Install ssh-agent if not already installed
  2. Run ssh-agent (inside the build environment)
  3. Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store Use tr to fix line endings which makes ed25519 keys work without extra base64 encoding the ssh-add - command does not display the value
  4. Create the SSH directory and give it the right permissions

The executor runs Python 3.8 but the MySql utility used in the Ansible example uses Python 2.7 version (mysql-connector-python). So we need to replace it with python3-mysqldb
Edi the file: ./Ansible/IaC_with_GitLab/infra/roles/db/tasks/main.yml

- name: Install MariaBD package
    - 'python3-mysqldb'
    - 'mariadb-server'
    state: present

IV. Upload the Code Sources

[In the PC]

$ mkdir ~/Ansible/IaC_with_GitLab/infra
$ cd /Ansible/IaC_with_GitLab/infra
$ git config --global "devguy"
$ git config --global ""
# Start a politically correct init repository (no master)
$ git init
$ git remote add main
# Add the files
$ git add .
$ git commit -m "Initial commit"
$ git ls-tree -r main

Push the local repo branch under <local_branch_name> to the remote repo at <remote_name>.

$ git push main main
Enumerating objects: 38, done.
Counting objects: 100% (38/38), done.
Delta compression using up to 4 threads
Compressing objects: 100% (26/26), done.
Writing objects: 100% (38/38), 5.52 KiB | 807.00 KiB/s, done.
Total 38 (delta 0), reused 0 (delta 0), pack-reused 0
remote: To create a merge request for master, visit:
* [new branch] master -> master

$ git status
On branch main
nothing to commit, working tree clean

If a typo was made and a file needs updating the following sequence can be used:

# Make the editing
$ vi .gitlab-ci.yml
# Update the remote repository
$ git add .gitlab-ci.yml
$ git commit -m "Adding IaC CI pipeline"
$ git push main main

V. Automatic Pipeline Execution

The output in the GitLab Server is

Running with gitlab-runner 13.12.0 (7a6612da)
on docker-runner ejJb1t1N
Preparing the "docker" executor 00:02
Using Docker executor with image mullnerz/ansible-playbook …
Getting source from Git repository 00:02
$ which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y )
$ eval $(ssh-agent -s)
Agent pid 11
$ echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
Identity added: (stdin) (xxxxx@pcxxx)
$ mkdir -p ~/.ssh
$ chmod 700 ~/.ssh
$ echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
$ chmod 644 ~/.ssh/known_hosts
$ ansible-playbook -i hosts site.yml
PLAY [apply common configuration to all nodes] *********************************
TASK [Gathering Facts] *********************************************************
ok: []
ok: []
PLAY RECAP ********************************************************************* : ok=14 changed=4 unreachable=0 failed=0 skipped=0    rescued=0 ignored=0 : ok=16 changed=2 unreachable=0 failed=0 skipped=0    rescued=0 ignored=0   
Job succeeded

IV. Test

Test in a browser

Hello World! My App deployed via Ansible V6. 


Hello, World! I am a web server configured using Ansible and I am : kmaster
List of Databases:
foodb information_schema mysql performance_schema


There is an Ansible playbook to clean up our servers:

$ cd ~/Ansible/IaC_with_GitLab/infra
$ ansible-playbook -i hosts reset.yml

Notes to myself but hope they can be of use to other curious people.
The code pruned for the Raspberry Pi’s is here.