A postmortem looks for what let the outage happen; who pressed the button does not matter to it.
Once the timeline is ready, the team writes a postmortem — a review of the outage: what happened, why and what to do so that it does not happen again. It is written after every noticeable outage, even if the site was fixed within a minute.
The main rule of a postmortem is to be blameless. The answer “deploy released a broken version” is true but useless: tomorrow another person will release another one. The useful question is what let the broken version reach users. Here there are several answers: release 1.5.0 ran nowhere before the production machine, the release did not check that the service came up, and systemd gave up on the service after five tries, so the site waited for a person.
People in a postmortem are named by role — “the release”, “the on-call engineer” — and the text describes what they saw and why their action looked reasonable at the time. Whoever made the change knows best how things went. If mistakes are punished, next time they will keep quiet, and the postmortem will be left without facts.
Which root cause wording fits a blameless postmortem?
-
It is true, but it names a person and stops there. A postmortem asks what let the version reach the users.
-
This is a judgement of a person. The site waited for the on-call engineer because systemd gave up restarting — that is what goes into the root cause.
-
The import failed at every start — the journal shows five ModuleNotFoundError lines. A check in the release catches such an error.
The root cause in a postmortem is the conditions that let the outage happen and drag on. Each of them can be closed by an action, while next time a different person stands in.
Key points
- A postmortem is written after a noticeable outage: what happened, why and what to do.
- Blameless: the question is “what let this happen”, people are named by role.
- Whoever made the change is the main source of facts about the outage.