Run a CD pipeline for MariaDB and Web App.

This post is an add-on to the series as I tried to do a “LAMP automated deployment from a GitLab repository” using Ansible but the information I found online was too difficult to understand or was outdated.

Lets start with a LAMP Ansible playbook example from the official example repository and let’s modify it so it can upload it to a GitLab Repository and execute the deployment to set up a Web server in one machine and a MariaDB Server in other (two Raspberry Pi’s).

There are two things that we have to consider:

ONE
GitLab currently doesn’t have built-in support for managing SSH keys in a build environment (where the GitLab Runner runs). The most widely supported method is to inject an SSH key into our .gitlab-ci.yml.

  • A common way to make secrets available to containers is using a volume mounted into the container at runtime. Volumes can be added to the executor through the GitLab Runner configuration.
  • But the most widely supported method so GitLab can provide them is to inject the SSH key into your build environment (in the.gitlab-ci.yml). This solution works with any type of executor.
    We will supply the information from Secure Variables.

TWO
The docker image we configured in the runner it is a basic Docker image. The appropriate way is to use an image that has the necessary software, that is Ansible, Python and the MySql utilities to configure the database.

I browsed what was available in hub.docker.com. This one from mullnerz has Ansible 2.9 and Python 3.8. That is good enough for our project. As shown in:

$ docker run --rm --name ansible mullnerz/ansible-playbook ansible --version
ansible 2.9.6
python version = 3.8.2 (default, Feb 29 2020, 17:03:31) [GCC 9.2.0]

I. Create a GitLab Project

We need a repository for our code the GitLab Server. Lets create one:

  • Browse to our local
    http://gitlab.example.com
    or your GitLab, GitHub code service.
    We use devguy/Clave123 to log in (Credentials shown as reference as it is used in the url)
  • Click on right blue button: New project’ > Create blank project
    Project name: infra
    Visibility level: Public
    Create readme file: no
    Click on ‘Create project’ button
  • We can get access to it opening the url:
    http://gitlab.example.com/devguy/infra

II. Check The Runner

We have set up previously a Docker executor to run our CI/CD jobs. To verify that the Docker Runner is still running as we configured it:

[On the PC]

$ sudo gitlab-runner verify
Runtime platform arch=amd64 os=linux pid=50507 revision=7a6612da version=13.12.0
Running in system-mode.
Verifying runner... is alive runner=ejJb1t1N

It seems ok. Let’s continue


III. Set The Log In Keys

Gitlab writes that: “When your CI/CD jobs run inside Docker containers and you want to deploy your code in a private server, you need a way to access it. In this case, you can use an SSH key pair.”

Create two new CI/CD variables.

  1. Access our GitLab Server in
    http://example.gitlab.com
    log in as devguy/Clave123
    (This are the url and credentials used in previous posts)
    Go to Project > infra
    Then Settings > CI/CD
    Expand Variables, and click ‘Add Variable’ button
  2. As Key enter the name SSH_PRIVATE_KEY
    and in the Value field paste the content of your private key
    To get the private key you can use the following command (copy the whole answer):
    $ cat ~/.ssh/id_rsa
    —–BEGIN OPENSSH PRIVATE KEY—–

    —–END OPENSSH PRIVATE KEY—–
    In all variables set as Type: Variable
    Environment scope: all
    In Flags uncheck Protect and Mask
    Click Create variable button
  3. The second will hosts a list of all your hosts:
    key: SSH_KNOWN_HOSTS
    value: output of the command ssh-keyscan <IPS-list>
    (It will be a big string. Copy the answer):
    $ ssh-keyscan 192.168.1.223 192.168.1.224
    <<output>>

III. Modifying the code

We need to create one file and modify an existing one.

ONE
Create the .gitlab-ci.yml pipeline file (the official usage example is here).

$ cd <your-root-dir>/Ansible/105_IaC_with_GitLab/Infra
​
$ vi .gitlab-ci.yml

Add as content:

image:
    name: mullnerz/ansible-playbook
​
before_script:
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y )'
  - eval $(ssh-agent -s)
  - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh
  - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
  - chmod 644 ~/.ssh/known_hosts
​
deploy_app:
  stage: deploy
  script:
    - 'ansible-playbook -i hosts site.yml'

Notes to the before_script section:

  1. Install ssh-agent if not already installed
  2. Run ssh-agent (inside the build environment)
  3. Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store Use tr to fix line endings which makes ed25519 keys work without extra base64 encoding the ssh-add - command does not display the value https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
  4. Create the SSH directory and give it the right permissions

TWO
The executor runs Python 3.8 but the MySql utility used in the Ansible example uses Python 2.7 version (mysql-connector-python). So we need to replace it with python3-mysqldb
Edi the file: ./Ansible/IaC_with_GitLab/infra/roles/db/tasks/main.yml

- name: Install MariaBD package
  apt:
    name:
    - 'python3-mysqldb'
    - 'mariadb-server'
    state: present

IV. Upload the Code Sources

[In the PC]

$ mkdir ~/Ansible/IaC_with_GitLab/infra
​
$ cd /Ansible/IaC_with_GitLab/infra
​
$ git config --global user.name "devguy"
$ git config --global user.email "devguy@gitlab.example.com"
​
# Start a politically correct init repository (no master)
$ git init
$ git remote add main http://devguy:Clave123@gitlab.example.com/devguy/infra.git
​
# Add the files
$ git add .
$ git commit -m "Initial commit"
$ git ls-tree -r main

Push the local repo branch under <local_branch_name> to the remote repo at <remote_name>.

$ git push main main
Enumerating objects: 38, done.
Counting objects: 100% (38/38), done.
Delta compression using up to 4 threads
Compressing objects: 100% (26/26), done.
Writing objects: 100% (38/38), 5.52 KiB | 807.00 KiB/s, done.
Total 38 (delta 0), reused 0 (delta 0), pack-reused 0
remote:
remote: To create a merge request for master, visit:
remote: http://gitlab.example.com/devguy/infra/-/merge_requests/new?merge_request%5Bsource_branch%5D=master
remote:
To http://gitlab.example.com/devguy/infra.git
* [new branch] master -> master

$ git status
On branch main
nothing to commit, working tree clean

If a typo was made and a file needs updating the following sequence can be used:

# Make the editing
$ vi .gitlab-ci.yml
# Update the remote repository
$ git add .gitlab-ci.yml
$ git commit -m "Adding IaC CI pipeline"
$ git push main main

V. Automatic Pipeline Execution

The output in the GitLab Server is

Running with gitlab-runner 13.12.0 (7a6612da)
on docker-runner ejJb1t1N
Preparing the "docker" executor 00:02
Using Docker executor with image mullnerz/ansible-playbook …
...
Getting source from Git repository 00:02
...
$ which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y )
/usr/bin/ssh-agent
$ eval $(ssh-agent -s)
Agent pid 11
$ echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
Identity added: (stdin) (xxxxx@pcxxx)
$ mkdir -p ~/.ssh
$ chmod 700 ~/.ssh
$ echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
$ chmod 644 ~/.ssh/known_hosts
$ ansible-playbook -i hosts site.yml
PLAY [apply common configuration to all nodes] *********************************
TASK [Gathering Facts] *********************************************************
ok: [192.168.1.223]
ok: [192.168.1.224]
...
PLAY RECAP *********************************************************************
192.168.1.223 : ok=14 changed=4 unreachable=0 failed=0 skipped=0    rescued=0 ignored=0   
192.168.1.224 : ok=16 changed=2 unreachable=0 failed=0 skipped=0    rescued=0 ignored=0   
Job succeeded

IV. Test

Test in a browser

http://192.168.1.223/index.html

Hello World! My App deployed via Ansible V6. 

and

http://192.168.1.223/index.php

Homepage
Hello, World! I am a web server configured using Ansible and I am : kmaster
List of Databases:
foodb information_schema mysql performance_schema

Cleanup(Manual)

There is an Ansible playbook to clean up our servers:

$ cd ~/Ansible/IaC_with_GitLab/infra
​
$ ansible-playbook -i hosts reset.yml

Notes to myself but hope they can be of use to other curious people.
The code pruned for the Raspberry Pi’s is here.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s