Linux Labs

SRE practices: SLOs, incidents, redundancy / Stage 2 of 8

SLOs and error budgets

Not running Starting usually ~20 s
Leave the lesson

Step 1 of 6 · on a machine · 6 min

How much more can go wrong

What you will learn

  • Turn an SLO into an error budget — how many bad answers a window allows.
  • Count from the logs how much of the budget is left after an outage.
  • Count the budget's burn rate and estimate when it runs out.

Yesterday afternoon the shop answered with errors for two and a half hours; now it works again. The team has a target: over any 30 days, at least 99.5 % good answers. Before rolling out a new version, your manager asks whether you still meet the target and how many errors are left in reserve. The shop's logs for 30 days are in /var/log/shop.

Exercise · look

Count the errors over 30 days

Look at the shop's logs in /var/log/shop and count all requests and 5xx answers across all the files, then the 5xx answers for yesterday.

First, predict

Yesterday's outage lasted two and a half hours out of 30 days. What share of the 30 days' 5xx answers fell on yesterday?

Now start the exercise, run the command and compare your prediction with the output.

Sign in and start The machine is available after signing in
Check yourself 1 question

What is true by the numbers from the shop's logs over 30 days?

  • Bad answers over 30 days are about 0.4 % — less than the allowed 0.5 %, so the target still holds.

  • The other way round: awk over access.log.1 counted about three quarters of all the month's 5xx answers.

  • For yesterday it is about 280 bad out of ~2900 requests, almost 10 %. 0.4 % is the share for all 30 days.

Over 30 days about 0.4 % are bad, and the 99.5 % target still holds.

Ubuntu 24.04
Loading…