Metrics, exporters and scraping

The exporter only knows “now”. History is kept by Prometheus — it fetches the metrics itself.

The node_exporter service is an exporter: it turns the machine's figures into this text format. Every scrape_interval Prometheus requests /metrics from each target — an exporter or an application from its config — and stores the values with a timestamp. Such a request is called a scrape. Targets are grouped into jobs — the scrape_configs list in /etc/prometheus/prometheus.yml (in the Wiki):

scrape_configs:
  - job_name: node                       # job name
    static_configs:
      - targets: ['localhost:9100']      # target address:port

Prometheus adds two labels to every series it scrapes: job, the job name, and instance, the target address. And for every target it writes its own up series: 1 — the last scrape worked, 0 — the target did not answer.

The Prometheus web interface lives on port 9090, but there is no browser to this machine. The state of the targets comes from the /api/v1/targets API — handy to read through jq (in the Wiki) — and promtool query instant runs a query against the collected data.