Postmortem — a blameless outage review
cheat sheet · Ubuntu 24.04 · systemd 255 https://lab.dimsanych.ru/en/wiki/postmortem/Cheat sheet
-
cd ~/incident - outage window: -S from when, -U until when (today, UTC)
-
sudo journalctl -S 12:00 -U 12:10 -o short-iso -t sudo > raw.txt - the service in a separate call: -t with -u gives empty output
-
sudo journalctl -S 12:00 -U 12:10 -o short-iso -u nginx >> raw.txt - by time, into the "HH:MM:SS event" format
-
sort raw.txt | cut -c12-19,26- > timeline.txt - the first and the last error users saw
-
awk '$9 >= 500' /var/log/nginx/access.log | sed -n '1p;$p' - unix time from $msec into clock time
-
date -u -d @1792677901 +%T