Users, permissions, and processes / Stage 12 of 13 · step 8 of 9

A process that will not go away

~20 min

Not running Starting usually ~20 s

Situation

Monitoring sent an alert: a process on the server is in state Z, and it does not go away. A colleague tried kill -9, and the row stayed. Monitoring wrote the process numbers from the alert to /srv/zombie-lab/pids.

Next to it runs a training program, collector, as your user student, watched over by the lab-zombie.service service. The program's description is in /srv/zombie-lab/README.

Task

There is no longer a row in state Z in the process table.
collector is the same process that was started: the same PID and start time, running and not stopped.
The service's supervisor has written the outcome to /srv/zombie-lab/result.txt.
The program files, the lab-zombie.service service, and its limits are unchanged, and collector has no new children.

Constraints

Do not stop or restart collector or its service.

The tools you need are already installed. There is no internet on this machine.

Lab goals

0 of 4

The row in state Z is gone — not checked yet
collector is the same process, running and not stopped — not checked yet
The supervisor wrote the outcome to result.txt — not checked yet
The program, its limits and the task files are unchanged; no new children exist — not checked yet

Hints

0 of 3 open · they open one at a time

Hint 1

Show the solution after 3 failed checks

Processes · picked for this lab

ps — list processes

Cheat sheet

$_ — insert into the terminal

ps processes of this terminal
ps aux all processes, BSD style
ps -ef all processes, with the parent PPID
ps aux --sort=-%cpu | head most CPU
ps aux --sort=-%mem | head most memory
ps -u student a user's processes
ps -p 1234 -o pid,stat,etime,cmd custom columns for a PID
ps -C nginx by program name
ps -ef --forest a tree: who started whom
pgrep -a nginx PID and command line by name

Options with a dash (-e, -f, -u) and without (a, u, x) are different sets, and they can be mixed. Full list — man ps.

What the STAT letters mean

Letter	State
`R`	running or waiting for a CPU
`S`	sleeping, waiting for an event
`D`	waiting on disk or network, signals can't interrupt
`T`	stopped (Ctrl+Z or a STOP signal)
`Z`	zombie: exited, the parent hasn't collected it
`s`, `+`, `l`	session leader, foreground, multithreaded

Find a process by name without grep itself

bash

pgrep -a nginx                 # 1021 nginx: master process ...
ps aux | grep '[n]ginx'        # [n] doesn't match grep's own line

ps aux | grep nginx also finds the line of grep nginx itself — hence the "extra" process in the output.

Custom columns and the parent

text

student@ubuntu:~$ ps -o pid,ppid,user,stat,etime,cmd -p 1
    PID    PPID USER     STAT     ELAPSED CMD
      1       0 root     Ss         18:42 /sbin/init

etime is how long the process has run, ppid is the parent's PID: use it to find who started a process or who isn't reaping a zombie.

A service's process tree

bash

ps -ef --forest | grep -A3 '[n]ginx: master'
systemctl status nginx         # the same tree in the CGroup section

A process eats CPU, but ps shows little %CPU

%CPU in ps is the average over the process's lifetime, not "right now". top shows the current load (P sorts by CPU).

error: user name does not exist

ps -u expects an existing name or UID. Check the name: id name.

The process is listed, but kill doesn't help

State Z — the process has already exited and can't be killed: wait for the parent to collect it, or end the parent. State D — the process waits on I/O, and the signal takes effect when the operation finishes. More in kill and processes.

The command line is cut off

ps fits output to the window width. For the full line: ps auxww or ps -o args= -p PID.