- The first minutes of an incident follow the runbook: confirm the symptom, start the timeline, find the last change, bring the service back, check.
- The service goes back to users first — by a rollback, a restart, a switch; the cause is looked for later, from the saved copy and the timeline.
- What changed last shows in the sudo journal, in file modification times (
ls -lt,find -newermt) and in the machine's journal for the minutes before the alert. - A timeline is “time event” lines in UTC from the first minute; the time of events before you comes from the logs.
- The site at the start of the lesson went down with no failure in the application: deploy moved it to port 8001 while nginx stayed on 8000 — rolling the port back returned the site before anyone worked out the cause.
Step 7 of 7 · summary · 2 min
Summary
Found an inaccuracy? Report an error in the lesson