Automation: Git and Ansible / Stage 5 of 6 · step 5 of 6

An unfinished playbook

~25 min

Not running Starting usually ~20 s

Situation

The shop servers need a status page. The status-board application is already installed (/usr/local/bin/status-board), but nobody wants to set it up by hand on every server. Timur started an Ansible playbook in ~/status-board and did not get to finish it: ansible-playbook site.yml fails right now.

Task

Finish site.yml so that it brings this machine (localhost from inventory.ini) to the needed state by itself:

The status user is a system user without a login shell (the playbook already has a task for it).
The /var/lib/status data directory: owner and group status, mode 0750.
The /etc/status/status.ini settings come from the templates/status.ini.j2 template with the variables from the playbook's vars:: owner root, group status, mode 0640.
The lab-status service (the unit is files/lab-status.service) is running, enabled at boot and answers: curl http://127.0.0.1:8095/.
Running the playbook again changes nothing: the recap says changed=0.

"Check" runs the playbook by itself. First it removes what the playbook must create: the lab-status service with its unit, /etc/status, /var/lib/status and the status user. Then it runs ansible-playbook site.yml from ~/status-board twice, as you. So the playbook must do everything — the check removes whatever you set up by hand.

Do not change the application, the files/lab-status.service file or the variables in vars:.

To run it: cd ~/status-board && ansible-playbook site.yml. Module help — ansible-doc file. There is no internet on the machine.

Lab goals

0 of 5

ansible-playbook site.yml runs without errors on a machine without the status page — not checked yet
The playbook creates the status system user and the /var/lib/status directory (status:status, 0750) — not checked yet
/etc/status/status.ini is built from the templates/status.ini.j2 template (root:status, 0640) — not checked yet
The lab-status service is running, enabled at boot and answers http://127.0.0.1:8095/ — not checked yet
A repeated run changes nothing — changed=0 — not checked yet

Hints

0 of 3 open · they open one at a time

Hint 1

Show the solution after 3 failed checks

Automation · picked for this lab

Ansible: inventory, modules and one-off commands

Cheat sheet

$_ — insert into the terminal

ansible -i hosts.ini all -m ping reach all machines
ansible -i hosts.ini all --list-hosts who a command will touch
ansible-inventory -i hosts.ini --graph groups and machines
ansible -i hosts.ini web -m command -a 'uptime' run a command
ansible -i hosts.ini web -m copy -a 'src=a.conf dest=/tmp/a.conf'
ansible -i hosts.ini web -b -m service -a 'name=nginx state=started'
ansible -i hosts.ini all -m ping -o one-line answers
ansible-playbook -i hosts.ini site.yml a playbook

Inventory

ini

[web]
node2

[db]
db1 ansible_host=10.0.0.21 ansible_user=deploy

[local]
localhost ansible_connection=local

[web:vars]
ansible_user=student

What	Where	Means
machine name	`node2`	how the machine is named in the output; Ansible also connects by it if there is no `ansible_host`
`ansible_host`	the machine's line	the address or name for SSH
`ansible_user`	the machine's line or `[group:vars]`	who to log in as; without it — your own name
`ansible_connection=local`	the machine's line	work on this machine without SSH
`[web]`	a header	a group; `all` is every machine in the inventory

text

student@node1:~/inv$ ansible-inventory -i hosts.ini --graph
@all:
  |--@ungrouped:
  |--@web:
  |  |--node2
  |--@local:
  |  |--localhost
student@node1:~/inv$ ansible -i hosts.ini all --list-hosts
  hosts (2):
    node2
    localhost

Without -i Ansible looks for /etc/ansible/hosts; the lab machine has none, and the command sees only localhost. Inventory errors in a playbook — ansible-playbook.

One-off commands

ansible <machines> -m <module> -a '<arguments>' runs one module on all machines of a group. The answer comes per machine:

text

student@node1:~/inv$ ansible -i hosts.ini web -m copy \
> -a 'content="hello\n" dest=/tmp/hello.txt'
node2 | CHANGED => {
    "changed": true,
    "dest": "/tmp/hello.txt",
    ...
}

The same command again gives node2 | SUCCESS and "changed": false: the file is already like that. The command module (also the default without -m) does not know what the command does and always reports CHANGED:

text

student@node1:~/inv$ ansible -i hosts.ini web -m command -a 'id -un'
node2 | CHANGED | rc=0 >>
student

In the output	Means
`SUCCESS`	the module ran, there was nothing to change
`CHANGED`	the module changed the machine (`command` always does)
`FAILED!`	the module ran with an error — the cause is in `msg`
`UNREACHABLE!`	cannot connect to the machine — ansible

-o prints each machine's answer on one line. service returns hundreds of unit fields — -o or a playbook is easier to read.

become: through sudo

Modules run as the user Ansible logged in as. Writing to /etc without the rights:

text

node2 | FAILED! => {
    ...
    "msg": "Destination /etc not writable"
}

-b (--become) runs the module through sudo:

text

student@node1:~/inv$ ansible -i hosts.ini web -b -m command -a 'id -un'
node2 | CHANGED | rc=0 >>
root

On the lab machines student's sudo needs no password. Where sudo asks for one, add -K (--ask-become-pass). In a playbook it is the same — become: true.

Playbooks and roles

yaml

# site.yml
- hosts: web
  become: true
  vars:
    pkg: nginx
  tasks:
    - name: Install {{ pkg }}
      apt: { name: "{{ pkg }}", state: present, update_cache: true }
    - name: Config from a template
      template: { src: nginx.conf.j2, dest: /etc/nginx/nginx.conf }
      notify: reload nginx        # triggers the handler on change
    - name: Start and enable
      service: { name: "{{ pkg }}", state: started, enabled: true }
  handlers:
    - name: reload nginx
      service: { name: nginx, state: reloaded }

bash

ansible-playbook -i inventory.ini site.yml
ansible-playbook -i inventory.ini site.yml --check --diff
ansible-playbook -i inventory.ini site.yml --limit web1 --tags config
ansible-galaxy init roles/myrole       # role skeleton

Variables go in group_vars/, host_vars/; secrets in ansible-vault (ansible-vault encrypt secrets.yml, run with --ask-vault-pass). The lab machine has no network to the outside: update_cache and packages from the network will not work.

How to read a run, changed=0 on the second run, handlers, --check --diff and playbook errors — ansible-playbook.

UNREACHABLE!

Ansible could not connect over SSH; the module never ran, the exit code is 4. The cause is in msg, which is the error text of ssh itself (lines wrapped):

text

node3 | UNREACHABLE! => {
    "msg": "Failed to connect to the host via ssh: ssh: Could not
      resolve hostname node3: Temporary failure in name resolution",
node2 | UNREACHABLE! => {
    "msg": "Failed to connect to the host via ssh: admin@node2:
      Permission denied (publickey).",
web1 | UNREACHABLE! => {
    "msg": "Failed to connect to the host via ssh: Host key
      verification failed.",

Text	Cause	What to do
Could not resolve hostname	the name is neither in DNS nor in `/etc/hosts`	fix the name or set `ansible_host`
Permission denied (publickey)	this user does not have your key	the right `ansible_user`, or the key in their `authorized_keys`
Host key verification failed	the host key is not in `known_hosts`, or it changed	`ssh` to that address by hand and accept the key; a changed one — `ssh-keygen -R <address>`

On the lab machines host key checking is off for the names node1 and node2 (/etc/ssh/ssh_config.d/99-lab.conf), but not for an IP address in ansible_host: the same node2 by address gives Host key verification failed. Check by hand as the same user: ssh -o BatchMode=yes node2 true; details — ansible … -vvv. Keys — ssh-keys.

Destination /etc not writable

The module runs without root rights: use -b or become: true — ansible.

sudo: a password is required

You need --become and working sudo. With a password — --ask-become-pass (-K) or NOPASSWD, see sudo.

A task is "changed" every time

You're using command/shell instead of an idempotent module. Use modules (apt, copy, template, lineinfile) — they only change things when needed. For command, use creates=/changed_when:, more in ansible-playbook.

Vault: can't decrypt anything

You need the vault password (--ask-vault-pass or --vault-password-file). Without it, encrypted variables are unavailable.

An unfinished playbook

Situation

Task

Signing in is needed to start the machine

Inventory

One-off commands

become: through sudo

Playbooks and roles

UNREACHABLE!

Destination /etc not writable

sudo: a password is required

A task is "changed" every time

Vault: can't decrypt anything