Using templates in gitlab-ci

GitLab CI/CD continuos integration is a powerful and great tool and I love using it on my selfhosted gitlab instance. Push some code and it will be build, tested and deployed thanks to the power of CI/CD.

How does it work¶

All you have to do is to write some instructions in a file called .gitlab-ci.yml. For each change pushed, these instructions aka pipelines are passed to one or more gitlab-runners. These pipelines are comprised of stages, which tells the runner when to do something and jobs, which defines what to do.

The runner execute these commands with its docker executor inside docker containers and cleans up afterwards. Since everything runs inside containers, the hostsystem stays neat and clean and isn't cluttered with tools, packages and stuff.

containers build with ci/cd¶

I use individual containers for linting and running ansible playbooks, testing puppet code, building and testing this site, building prometheus exporters and so on. All of these tools have their own repository and therefor .gitlab-ci.yml.

docker_group

So each and every repository contains the same following build steps:

.gitlab-ci.yml

---

image: docker:latest
variables:
  VERSION: $(cat VERSION)
build:
  script:
    - docker login -u ${CI_REGISTRY_USER} -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
    - docker build --pull -t ${CI_REGISTRY_IMAGE}:${VERSION} -t ${CI_REGISTRY_IMAGE}:latest .
    - docker push ${CI_REGISTRY_IMAGE}:${VERSION}
    - docker push ${CI_REGISTRY_IMAGE}:latest

This way changing the build workflow will become tedious.

Using includes ¶

The include keyword allows you to call external yaml files. These yaml files can be located in the same repository, in a remote repository or even via http(s).

Template repository¶

Knowing this, I build me an extra repository called gitlab/ci-templates:

gitlab/ci-templates

ci-templates
├── ansible-lint.yml
├── ansible-playbooks.yml
├── docker-build.yml
├── docusaurus.yml
├── mkdocs.yml
├── puppet-lint.yml
└── README.md

All these files are written like any other .gitlab-ci.yml file. Since these files function as templates you want them to be as generic as possible.

gitlab/ci-templates/docker-build.yml

---

image: docker:latest

build:
  script:
    - [ -n "$VERSION" ] && TAG_VERSION="-t ${CI_REGISTRY_IMAGE}:${VERSION}"
    - docker login -u ${CI_REGISTRY_USER} -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
    - docker build --pull ${TAG_VERSION} -t ${CI_REGISTRY_IMAGE}:latest .
    - [ -n "$VERSION" ] && docker push ${CI_REGISTRY_IMAGE}:${VERSION}
    - docker push ${CI_REGISTRY_IMAGE}:latest
  tags:
    - docker

docker-build.yml is using a couple of build in Variables to make it more generic:

CI_REGISTRY_IMAGE: the build in Registry URL + full path to the repository the pipeline is working on
CI_REGISTRY: the build in Registry URL
CI_REGISTRY_USER+CI_JOB_TOKEN: User and password for authentication
VERSION: is not build in and is meant to be set/overwritten by the pipeline including this template.

Including templates¶

Including these templates in one of the container repositories is very easy.

docker/ansible/.gitlab-ci.yml

---

include:
  - { project: gitlab/ci-templates, file: docker-build.yml }

build:
  before_script:
    - VERSION=$(docker run -i --rm curlimages/curl:latest https://pypi.org/simple/ansible/ | 
        awk -F '<|>' '/tar.gz/ {v=gensub(/.*-(.*).tar.gz/,"\\1","g",$3)}; END {print v}')

As you can see the only thing left after including docker-build.yml is to set the $VERSION variable.

Now if the build process needs optimization, this will be done once in the template. Every ci using the template will use the change instantaneously.

In case of breaking changes you could also use tags or commits to use a specific version of the template:

include:
  - { project: gitlab/ci-templates, file: docker-build.yml, ref: 1.0.0 }

Last update: March 22, 2021