Alertmanager: to whom and how many times

Planned maintenance is known in advance: its notifications are muted for the right time without touching the configuration.

A silence is a condition on labels plus a time window: until the window ends, alerts with those labels produce no notifications. The alerts themselves stay in Alertmanager, but amtool alert does not show them; amtool alert query --silenced does. Silences are created with amtool, the Alertmanager client; it takes the service address from /etc/amtool/config.yml:

amtool silence add alertname=InstanceDown \
  instance=db01:9100 --duration=30m \
  --comment='disk replacement on db01'
# active silences and their IDs
amtool silence query
# silenced alerts
amtool alert query --silenced
# end a silence early
amtool silence expire <ID>

The condition is labels separated by spaces, and all of them must match. The more precise the condition, the less gets silenced by accident: alertname=InstanceDown without a host would silence all 50 servers.