GitLab CI/CD pipelines running ansible

Ansible is one of many Configuration Management tools out there. It's written in python, has a vast variety of modules, a simple descriptive yaml syntax, and has an agentless style of handling infrastructure. Unlike Puppet there is no agent that run tasks in a regular manner. For running ansible playbooks regularly you can use tools like ansible tower. But this introduces further complexity to your infrastructure, which is depending on what you are doing unnecessary.

Hence the title GitLab CI/CD pipelines running ansible.

Ansible playbooks, roles, inventories and plugins should always be version controlled. Using GitLab as version control system already covers the aspect of "who has access to your playbooks". Add the power of GitLab CI/CD and the aspect of "when and how to run something" is covered as well. Failure notifications and status reports are covered too.

This example will cover the following:

• Building an ansible environment
• Testing playbooks
• Credential management
• Roles and Collections
• Junit Test Reports

Building an ansible container image

Since my gitlab-runners are using the docker executor it is necessary to use a container image containing ansible. There are many prepackaged docker images out there and they should do the job just fine. I need some additional libraries and a linter, so I'm building my own image and push it into the container registry, integrated in my self hosted GitLab instance.

Therefor I created a Repository (and group) called docker/ansible:

prerequirements.txt

lxml
junit-xml
openstacksdk
python-gitlab

requirements.txt

ansible
ansible-lint

Info

Requirements for python are split into 2 files to avoid dependency errors

Dockerfile

FROM python:latest

COPY prerequirements.txt /
COPY requirements.txt /

RUN pip install -r /prerequirements.txt && \
  pip install -r /requirements.txt

CMD ["ansible-playbook", "--help"]

.gitlab-ci.yml

---

include: { project: gitlab/ci-templates, file: docker-build.yml }

build:
before_script:
    - VERSION=$(docker run -i --rm curlimages/curl:latest https://pypi.org/simple/ansible/ | 
        awk -F '<|>' '/tar.gz/ {version=gensub(/.*-(.*).tar.gz/,"\\1","g",$3)}; END {print version}')

This builds an image called <myregistry>/docker/ansible:<VERSION_TAG> and <myregistry>/docker/ansible:latest

Info

I'm using Templates for the build process which I covered here The image's tag should contain the version of ansible, hence curling pypi for the version number. pip search does not work anymore. see this issue

Using the integrated container registry has a nice side effect:

It uses GitLab's auth mechanism and pipelines are using special tokens for this case so you don't have to bother with it.

Testing Playbooks

It is always a good idea to, at least, lint code before further actions. No matter how good you think your ansible writing is, Typos and other hiccups can happen.

Using CI templates I created a template called ansible-lint.yml

ansible-lint.yml

---

stages:
  - lint

lint:
  image: ${CI_REGISTRY}/docker/ansible:latest
  stage: lint

  script:
    - ansible-lint -v . *.yaml *.yml playbooks/*.yml playbooks/*.yaml

In my repository containing playbooks I added a .gitlab-ci.yml including the template:

.gitlab-ci.yml

---

include: { project: gitlab/ci-templates, file: ansible-lint.yml }

From now on a complaining linter produces pipeline results that looks like this:

Running Playbooks

Let's create a template called ansible-playbooks.yml:

ansible-playbooks.yml

---

stages:
  - play

variables:
  JUNIT_OUTPUT_DIR: '.junit'
  JUNIT_TASK_RELATIVE_PATH: 'true'

playbooks:
  image: ${CI_REGISTRY}/docker/ansible:latest
  stage: play
  rules:
    - if: $CI_PIPELINE_SOURCE =~ /web|pipeline|schedule/

  artifacts:
    reports:
      junit: .junit/*.xml
    paths:
      - .junit/*.xml

  before_script:
    - 'eval $(ssh-agent -s)'
    - 'echo "$SSH_PRIVATE_KEY" | tr -d "\r" | ssh-add -'
    - '[ -f requirements.yaml ] && ansible-galaxy install -r requirements.yaml'
    - '[ -f requirements.yml ] && ansible-galaxy install -r requirements.yml'

Info

rules:
  - if: $CI_PIPELINE_SOURCE =~ /web|pipeline|schedule/

The rules section limits the playbooks job to run only when triggered in webinterface, other pipelines or schedule.

Let's say you run the job on commits but only when particular files where changed. This would look something like this:

rules:
  - if: $CI_PIPELINE_SOURCE == "push"
    changes:
      - *.yaml

ansible-lint.yml and ansible-playbooks.yml templates in combined action.

---

include:
  - { project: gitlab/ci-templates, file: ansible-lint.yml }
  - { project: gitlab/ci-templates, file: ansible-playbooks.yml }

stages:
  - lint
  - play

playbooks:
  script:
    - ansible-playbook site.yaml

After including the templates I explicitly define stages. This ensures the order of job execution. The playbooks job gets a new script element, defining the playbook run.

Info

If you are using vaults for sensitive data, you can define something like this instead in your playbooks job section:

playbooks:
  script:
    - echo $VAULT_PASSWORD | ansible-playbook --vault-password-file=/bin/cat site.yaml

$VAULT_PASSWORD management

VAULT_PASSWORD should be defined as Gitlab environment variable in the UI. Explained in the Credential management section.

Credential management

Committing ssh private keys, passwords or any type of credentials in your repository is a massive security risk. Don't do it! Here I add the ssh key via Gitlab environment variables. These variables, when set in the UI, are stored encrypted in gitlab's Database and only users with developer access or higher can edit them.

The first two commands in the playbook job are starting a ssh-agent and reading/adding the private key defined in the SSH_PRIVATE_KEY environment variable.

before_script:
  - 'eval $(ssh-agent -s)'
  - 'echo "$SSH_PRIVATE_KEY" | tr -d "\r" | ssh-add -'

Roles and Collections

Reusability should be always in mind when using ansible. Don't reinvent the wheel, search for roles and collections on galaxy.ansible.com first and if nothing there fulfills your requirements, then "write your own stuff" is the goto option.

The template searches for requirements files dealing with dependencies and installs them for you.

- '[ -f requirements.yaml ] && ansible-galaxy install -r requirements.yaml'
- '[ -f requirements.yml ] && ansible-galaxy install -r requirements.yml'

Enable Unit Reports

Gitlab CI/CD is able to display junit reports in a nice way and ansible is capable of producing them. This is beneficial for finding errors fast. You don't have to scroll through your whole play to find any errors.

variables:
  JUNIT_OUTPUT_DIR: '.junit'
  JUNIT_TASK_RELATIVE_PATH: 'true'
[...]
artifacts:
  reports:
    junit: .junit/*.xml
  paths:
    - .junit/*.xml

The report will filter and highlight errors for you, thus increasing your productivity.

Further links

I'm not the first one, and hopefully not the last one, using ansible with the power of GitLab CI/CD. Here are some other examples:

• ansible gitlab ci at sopra steria
• ansible gitlab ci at gitlab

---

Last update: March 22, 2021