node_exporter: machine metrics for Prometheus

cheat sheet · Ubuntu 24.04 · node_exporter 1.7 https://lab.dimsanych.ru/en/wiki/node-exporter/

Cheat sheet

sudo systemctl enable --now prometheus-node-exporter turn it on
systemctl status prometheus-node-exporter is the service alive
ss -tlnp | grep 9100 where it listens
curl -s localhost:9100/metrics | less everything it serves
curl -s localhost:9100/metrics | grep -c '^node_' value count
curl -s localhost:9100/metrics | grep MemAvailable one metric
curl -s localhost:9100/metrics | promtool check metrics format
sudo systemctl restart prometheus-node-exporter after editing ARGS

Configuration

Startup flags

The exporter has no settings file, only command-line flags. The service takes them from the ARGS variable in /etc/default/prometheus-node-exporter; the package leaves it empty (ARGS=""), so everything is at its default:

bash

# /etc/default/prometheus-node-exporter
ARGS="--web.listen-address=127.0.0.1:9100 --no-collector.zfs"

After an edit, run sudo systemctl restart prometheus-node-exporter. reload does not fit here: see node-exporter. All flags with descriptions: prometheus-node-exporter --help.

Address and port

By default the exporter listens on port 9100 on every address of the machine:

text

$ ss -tlnp | grep 9100
LISTEN 0  4096  *:9100  *:*  users:(("prometheus-node",pid=1178,fd=3))

The --web.listen-address flag sets the address and port. It goes into ARGS in /etc/default/prometheus-node-exporter or into a service drop-in (sudo systemctl edit prometheus-node-exporter → the file /etc/systemd/system/prometheus-node-exporter.service.d/override.conf, where an ExecStart= line replaces the start command). Where the flag is actually set:

systemctl cat prometheus-node-exporter shows the unit with its drop-ins, each file under a # /path line;
systemctl status prometheus-node-exporter shows the whole command with its flags in the CGroup line under the process number;
ss -tlnp | grep prometheus-node shows which port it listens on now.

There is no password and no encryption: whoever reaches the port sees the metrics. --web.listen-address=127.0.0.1:9100 leaves access from the machine itself only, and Prometheus on another machine then gets connection refused. When Prometheus lives on another machine, the exporter listens on its external address, and the firewall opens the port to the Prometheus address only.

Metrics are served only on the /metrics path. On any other path the exporter answers with an HTML page and code 200, and Prometheus with a wrong metrics_path writes expected a valid start token, got "<" into lastError (prometheus).