Where to start First steps track · 15 stages
Linux Labs

Inside the system: resources, kernel, processes

Load average and the CPU, memory and the OOM killer, process limits and kernel parameters, disk load, strace and /proc, kernel messages, the system boot and time synchronisation — up to triaging a slow server.

12 stages7 labs~7 hours medium medium

Progress Stages: 0 of 12

The track has not been started. The first stage is «Load average and CPU load», ~30 min.

Start stage 1

Module 1

CPU, memory and OOM

Who waits for the CPU and who for the disk; how used memory differs from available and why the OOM killer killed a process.

3 stages · ~2 hours

  1. Stage 1 of 12

    1 · Load average and CPU loadLoad average and CPU load

    7 steps · ~30 min 7 steps · ~30 min
    1. An alarm with no load exercises 0 of 1 on a machine 4 min
    2. 2 What the load average counts theory 5 min
    3. 3 Processes in the D state on a machine 6 min
    4. 4 The queue for the CPU on a machine 6 min
    5. 5 The nice priority theory 4 min
    6. 6 The CPU goes to the reports on a machine 3 min
    7. 7 Summary summary 2 min
  2. Stage 2 of 12

    2 · Memory, swap and the OOM killerMemory, swap and the OOM killer

    9 steps · 2 labs · ~1 hour 9 steps · 2 labs · ~1 hour
    1. A service restarts by itself exercises 0 of 1 on a machine 4 min
    2. 2 Used and available on a machine 6 min
    3. 3 Swap on a machine 4 min
    4. 4 The service ceiling on a machine 7 min
    5. 5 A unit drop-in on a machine 4 min
    6. 6 A ceiling for the cache on a machine 3 min
    7. 7 A service restarts in a loop medium lab 25 min
    8. 8 A build runs out of memory medium lab 20 min
    9. 9 Summary summary 2 min
  3. 3 · Module test 8 questions · pass mark 70%

Module 2

Limits and kernel parameters

A process limit and a kernel parameter changed so that the change survives a restart; the process that holds the disk.

3 stages · ~2 hours

  1. Stage 4 of 12

    4 · ulimit and sysctl: system ceilingsulimit and sysctl: system ceilings

    10 steps · 2 labs · ~1 hour 10 steps · 2 labs · ~1 hour
    1. The shipper loses logs exercises 0 of 1 on a machine 4 min
    2. 2 Process limits on a machine 4 min
    3. 3 Soft and hard on a machine 3 min
    4. 4 A service's limit on a machine 5 min
    5. 5 Kernel parameters on a machine 5 min
    6. 6 A change for boot time theory 4 min
    7. 7 More watches on a machine 3 min
    8. 8 A service fails under load medium lab 25 min
    9. 9 A kernel setting disappears medium lab 20 min
    10. 10 Summary summary 2 min
  2. Stage 5 of 12

    5 · Who loads the diskWho loads the disk

    6 steps · ~30 min 6 steps · ~30 min
    1. The CPU is free, the machine is waiting exercises 0 of 1 on a machine 4 min
    2. 2 Waiting for the disk theory 6 min
    3. 3 Who writes to the disk on a machine 8 min
    4. 4 Giving way on the disk theory 6 min
    5. 5 The orders wait for the disk on a machine 3 min
    6. 6 Summary summary 3 min
  3. 6 · Module test 10 questions · pass mark 70%

Module 3

What a process is doing

The cause of a silent failure from system calls and /proc; kernel messages about OOM, filesystem errors and crashes.

3 stages · ~2 hours

  1. Stage 7 of 12

    7 · strace and /proc: what a process is doingstrace and /proc: what a process is doing

    8 steps · 1 lab · ~55 min 8 steps · 1 lab · ~55 min
    1. The export is silent exercises 0 of 1 on a machine 5 min
    2. 2 System calls on a machine 6 min
    3. 3 Misses and filters theory 4 min
    4. 4 A process that waits on a machine 4 min
    5. 5 /proc: a process from the inside theory 6 min
    6. 6 The mail does not go out on a machine 3 min
    7. 7 An application exits silently medium lab 25 min
    8. 8 Summary summary 2 min
  2. Stage 8 of 12

    8 · dmesg and the kernel logdmesg and the kernel log

    8 steps · ~30 min 8 steps · ~30 min
    1. The archive does not respond exercises 0 of 1 on a machine 4 min
    2. 2 The kernel buffer on a machine 4 min
    3. 3 The ext4 error line theory 5 min
    4. 4 A program crashed on a machine 3 min
    5. 5 The segfault line and importance theory 4 min
    6. 6 The kernel log in journald theory 5 min
    7. 7 The nightly export broke off on a machine 3 min
    8. 8 Summary summary 2 min
  3. 9 · Module test 10 questions · pass mark 70%

Module 4

Boot and time

The boot chain and what slows it down; UTC, the time zone and time synchronisation.

3 stages · ~2 hours

  1. Stage 10 of 12

    10 · From GRUB to a target: how the system bootsFrom GRUB to a target: how the system boots

    8 steps · 1 lab · ~55 min 8 steps · 1 lab · ~55 min
    1. How long the boot took exercises 0 of 1 on a machine 4 min
    2. 2 The boot relay on a machine 5 min
    3. 3 Who held the boot up on a machine 4 min
    4. 4 Ordering and dependency on a machine 6 min
    5. 5 Boot targets on a machine 6 min
    6. 6 A service in the boot on a machine 3 min
    7. 7 A service holds up the boot medium lab 25 min
    8. 8 Summary summary 2 min
  2. Stage 11 of 12

    11 · Time, time zone and synchronisationTime, time zone and synchronisation

    7 steps · 1 lab · ~55 min 7 steps · 1 lab · ~55 min
    1. The export arrives at noon exercises 0 of 1 on a machine 4 min
    2. 2 The clock and the zone on a machine 5 min
    3. 3 The schedule follows the machine's zone on a machine 5 min
    4. 4 Clocks drift on a machine 6 min
    5. 5 Slewing and stepping on a machine 8 min
    6. 6 The clocks of two machines drift apart medium lab 25 min
    7. 7 Summary summary 2 min
  3. 12 · Module test 10 questions · pass mark 70%
  1. Final test 20 questions · pass mark 80%
Loading…