Three ways of sharing ansible inventories

In order for Ansible to run against your infrastructure you need an inventory. Define your hosts put them into groups and add some variables. Playbooks runs are then bound to these groups, hosts or patterns.

When starting things of, it may be sufficient to store your inventory in /etc/ansible/hosts and write some simple playbooks. Without any intentions of sharing this is fine.

One repository

As soon as your playbooks and/or the number of hosts to manage starts growing or even when you want to run your playbooks on multiple machines you have to start organizing thinks. Ansible's user guide provides nice examples of how to structure your directory layout. I like the alternative layout a bit more, since it keeps you base directory a bit cleaner. But nothing is carved in stone. So this allows you to have one directory to to rule them all. Whilst organizing this way, feel free to start using version control as well.

Summary:

• flexible in regards of running from multiple hosts
• wildly accepted and easy to use directory layout

git submodules

At some point you might face following situation. You have a repository following best practices and your playbooks have become thematically distinguishable i.e. bunch of plays only for maintenance, bunch of plays for deployments, etc. It's perfectly fine to move them to separate repositories. But it's likely there is a overlaps in your inventory i.e. you want to maintain the same hosts as deploying them. Duplicating the inventory is a possibility but this would increase your effort keeping everything in sync and thus lower productivity to some extend.

Now there is a way of avoiding duplication by using git submodules. Git allows to bind a repository as subdirectory to your main repository, whilst keeping commits separate. Which, apart from ansible inventories, comes in handy for things like libraries.

In this example I'll split a repository called ansible/playbooks.

Create 3 repositories: ansible/maintenance, ansible/deployments and ansible/inventories. Move everything in ansible/playbooks/inventories into ansible/inventories. Move all roles, libraries and plays dedicated for maintenance into ansible/maintenance and everything dedicated to deployments into ansible/deployments.

Info

It is important that ansible/maintenance and ansible/deployments contains no directory called inventories.

Now let's add ansible/inventories as submodule.

cd ansible/inventories
git submodule add ../inventories.git inventories

Info

By specifying a relative git url, git submodule will resolve paths relative to the main repository. Using git submodule add https://example-gitlab.com/ansible/inventories.git inventories instead is perfectly fine. I'm using relative urls because GitLab's CI/CD wants it that way.

This will add 2 new files to your main repository .gitmodules and inventories. Commit and push them and you're done.

Using submodules requires a slight adjustment to your git-commands usage. For cloning and pulling you should add the argument --recurse-submodules. Instead of git clone git@example-gitlab.com:ansible/maintenance.git you should use git clone --recurse-submodules git@example-gitlab.com:ansible/maintenance.git You can read further details, like "what do I do, when I forgot the --recuse-submodules argument", in the git submodule book.

Summary:

• flexible in regards of running from multiple hosts
• wildly accepted and easy to use directory layout
• thematic playbook separation for better usability
• no duplication of inventory code

I personally avoid submodules, whenever I can

To be honest, I don't like submodules at all. I guess I don't use them right aka it's my fault/incompetence they break all the time. With submodules there is a ridiculously high tendency of unexpected behavior. All of a sudden you have to tread you main repository like a raw egg. A 5min change suddenly takes 1 hour, with 55min repairing the repository.

http inventory plugin

Ansible allows you to use dynamic inventories. Dynamic inventories are scripts and plugins that are using external resources like a CMDB, your CloudProvider's API or other application APIs as inventory source.

All these dynamic inventories are highly specific:

• gitlab_runners - inventory gitlab runners
• cobbler - inventory cobbler instances
• aws ec2 - inventory ec2 instances
• proxmox - inventory proxmox instances
• [...] - the list goes on and on

There was no plugin to generically fetch an inventory from a http(s) source, so I wrote a plugin my self and shared it on ansible-galaxy.

Take your inventory, convert it into json with ansible-inventory -i <your_inventory> --list --output <your_json_file> and push <your_json_file> to a webserver. The derjd.general.html plugin fetches that file as inventory.

Example:

Generate a small inventory.

hosts.yaml

---
test_group:
  vars:
    test_var1: test
  hosts:
    test1.example.com:
    test2.example.com:
      test_var2: foo

Convert the inventory into json.

command

ansible-inventory -i hosts.yaml --list --output example.json

output

{
    "_meta": {
        "hostvars": {
            "test1.example.com": {
                "test_var1": "test"
            },
            "test2.example.com": {
                "test_var1": "test",
                "test_var2": "foo"
            }
        }
    },
    "all": {
        "children": [
            "test_group",
            "ungrouped"
        ]
    },
    "test_group": {
        "hosts": [
            "test1.example.com",
            "test2.example.com"
        ]
    }
}

Start a webserver serving example.json

docker run -it --rm -d -v ./example.json:/usr/share/nginx/html/example.json -p 8080:80 -n webserv nginx:latest

Install derjd.general collection

Command

ansible-galaxy collection install derjd.general

Output

Starting galaxy collection install process
Process install dependency map
Starting collection install process
Installing 'derjd.general:1.0.1' to '/home/jd/.ansible/collections/ansible_collections/derjd/general'
Downloading https://galaxy.ansible.com/download/derjd-general-1.0.1.tar.gz to /home/jd/.ansible/tmp/ansible-local-356547qe2hcbn/tmpsdkv2ujk
derjd.general (1.0.1) was installed successfully

Add inventory plugin config.

cat > test.http_inventory.yml <<EOF
---
plugin: derjd.general.http
url: http://localhost:8080/example.json
EOF

Check if ansible can parse the webserved inventory.

Command

ansible-inventory -i test.http_inventory.yml --graph

Output

@all:
|--@test_group:
|  |--test1.example.com
|  |--test2.example.com
|--@ungrouped:

Using GitLab CI/CD you can generate artifacts.

.gitlab-ci.yml

---
convert:
  image: ${CI_REGISTRY}/docker/ansible:latest
  artifacts: { untracked: true }
  script: [ 'ansible-inventory -i . --list --output inventory.json' ]

This would be accessible via:

https://<your_gitlab_instance>/ansible/inventories/-/jobs/artifacts/master/file/inventory.json?job=convert`.

More details about Downloading artifacts.

Summary:

• flexible in regards of running from multiple hosts
• wildly accepted and easy to use directory layout
• thematic playbook separation for better usability
• no duplication of inventory code
• no add_host shenanigans
• no git submodules

---

Last update: March 22, 2021