This post is an add-on to the series as I tried to do a “LAMP automated deployment from a GitLab repository” using Ansible but the information I found online was too difficult to understand or was outdated.
Lets start with a LAMP Ansible playbook example from the official example repository and let’s modify it so it can upload it to a GitLab Repository and execute the deployment to set up a Web server in one machine and a MariaDB Server in other (two Raspberry Pi’s).
There are two things that we have to consider:
GitLab currently doesn’t have built-in support for managing SSH keys in a build environment (where the GitLab Runner runs). The most widely supported method is to inject an SSH key into our
- A common way to make secrets available to containers is using a volume mounted into the container at runtime. Volumes can be added to the executor through the GitLab Runner configuration.
- Another approach would be using GitLab’s
Secure Variablesto store the private key and configure the GitLab Runner with a
pre_build_scriptthat adds the SSH key to the ssh-agent.
- But the most widely supported method so GitLab can provide them is to inject the SSH key into your build environment (in the
.gitlab-ci.yml). This solution works with any type of executor.
We will supply the information from Secure Variables.
The docker image we configured in the runner it is a basic Docker image. The appropriate way is to use an image that has the necessary software, that is Ansible, Python and the MySql utilities to configure the database.
I browsed what was available in hub.docker.com. This one from mullnerz has Ansible 2.9 and Python 3.8. That is good enough for our project. As shown in:
$ docker run --rm --name ansible mullnerz/ansible-playbook ansible --version
python version = 3.8.2 (default, Feb 29 2020, 17:03:31) [GCC 9.2.0]
I. Create a GitLab Project
We need a repository for our code the GitLab Server. Lets create one:
- Browse to our local
or your GitLab, GitHub code service.
We use devguy/Clave123 to log in (Credentials shown as reference as it is used in the url)
- Click on right blue button: New project’ > Create blank project
Project name: infra
Visibility level: Public
Create readme file: no
Click on ‘Create project’ button
- We can get access to it opening the url:
II. Check The Runner
We have set up previously a Docker executor to run our CI/CD jobs. To verify that the
Docker Runner is still running as we configured it:
[On the PC]
$ sudo gitlab-runner verify
Runtime platform arch=amd64 os=linux pid=50507 revision=7a6612da version=13.12.0
Running in system-mode.
Verifying runner... is alive runner=ejJb1t1N
It seems ok. Let’s continue
III. Set The Log In Keys
Gitlab writes that: “When your CI/CD jobs run inside Docker containers and you want to deploy your code in a private server, you need a way to access it. In this case, you can use an SSH key pair.”
Create two new CI/CD variables.
- Access our GitLab Server in
log in as devguy/Clave123
(This are the url and credentials used in previous posts)
Go to Project > infra
Then Settings > CI/CD
Expand Variables, and click ‘Add Variable’ button
- As Key enter the name
and in the Value field paste the content of your private key
To get the private key you can use the following command (copy the whole answer):
$ cat ~/.ssh/id_rsa
—–BEGIN OPENSSH PRIVATE KEY—–
—–END OPENSSH PRIVATE KEY—–
In all variables set as Type:
Environment scope: all
In Flags uncheck
- The second will hosts a list of all your hosts:
value: output of the command ssh-keyscan <IPS-list>
(It will be a big string. Copy the answer):
$ ssh-keyscan 192.168.1.223 192.168.1.224
III. Modifying the code
We need to create one file and modify an existing one.
.gitlab-ci.yml pipeline file (the official usage example is here).
$ cd <your-root-dir>/Ansible/105_IaC_with_GitLab/Infra $ vi .gitlab-ci.yml
Add as content:
image: name: mullnerz/ansible-playbook before_script: - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y )' - eval $(ssh-agent -s) - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - - mkdir -p ~/.ssh - chmod 700 ~/.ssh - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts - chmod 644 ~/.ssh/known_hosts deploy_app: stage: deploy script: - 'ansible-playbook -i hosts site.yml'
Notes to the
- Install ssh-agent if not already installed
- Run ssh-agent (inside the build environment)
- Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store Use tr to fix line endings which makes ed25519 keys work without extra base64 encoding the
ssh-add -command does not display the value https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
- Create the SSH directory and give it the right permissions
The executor runs Python 3.8 but the MySql utility used in the Ansible example uses Python 2.7 version (mysql-connector-python). So we need to replace it with python3-mysqldb
Edi the file: ./Ansible/IaC_with_GitLab/infra/roles/db/tasks/main.yml
- name: Install MariaBD package apt: name: - 'python3-mysqldb' - 'mariadb-server' state: present
IV. Upload the Code Sources
[In the PC]
$ mkdir ~/Ansible/IaC_with_GitLab/infra $ cd /Ansible/IaC_with_GitLab/infra $ git config --global user.name "devguy" $ git config --global user.email "firstname.lastname@example.org" # Start a politically correct init repository (no master) $ git init $ git remote add main http://devguy:Clave123@gitlab.example.com/devguy/infra.git # Add the files $ git add . $ git commit -m "Initial commit" $ git ls-tree -r main
Push the local repo branch under
<local_branch_name> to the remote repo at
$ git push main main
Enumerating objects: 38, done.
Counting objects: 100% (38/38), done.
Delta compression using up to 4 threads
Compressing objects: 100% (26/26), done.
Writing objects: 100% (38/38), 5.52 KiB | 807.00 KiB/s, done.
Total 38 (delta 0), reused 0 (delta 0), pack-reused 0
remote: To create a merge request for master, visit:
* [new branch] master -> master
$ git status
On branch main
nothing to commit, working tree clean
If a typo was made and a file needs updating the following sequence can be used:
# Make the editing
$ vi .gitlab-ci.yml
# Update the remote repository
$ git add .gitlab-ci.yml
$ git commit -m "Adding IaC CI pipeline"
$ git push main main
V. Automatic Pipeline Execution
The output in the GitLab Server is
Running with gitlab-runner 13.12.0 (7a6612da) on docker-runner ejJb1t1N Preparing the "docker" executor 00:02 Using Docker executor with image mullnerz/ansible-playbook … ... Getting source from Git repository 00:02 ... $ which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y ) /usr/bin/ssh-agent $ eval $(ssh-agent -s) Agent pid 11 $ echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - Identity added: (stdin) (xxxxx@pcxxx) $ mkdir -p ~/.ssh $ chmod 700 ~/.ssh $ echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts $ chmod 644 ~/.ssh/known_hosts $ ansible-playbook -i hosts site.yml PLAY [apply common configuration to all nodes] ********************************* TASK [Gathering Facts] ********************************************************* ok: [192.168.1.223] ok: [192.168.1.224] ... PLAY RECAP ********************************************************************* 192.168.1.223 : ok=14 changed=4 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 192.168.1.224 : ok=16 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 Job succeeded
Test in a browser
Hello World! My App deployed via Ansible V6.
Hello, World! I am a web server configured using Ansible and I am : kmaster
List of Databases:
foodb information_schema mysql performance_schema
There is an Ansible playbook to clean up our servers:
$ cd ~/Ansible/IaC_with_GitLab/infra $ ansible-playbook -i hosts reset.yml
Notes to myself but hope they can be of use to other curious people.
The code pruned for the Raspberry Pi’s is here.