First steps / Stage 12 of 15 · step 6 of 7

Count the errors in the logs

~20 min

Not running Starting usually ~20 s

Situation

Last night an online shop kept answering its customers with errors. Its logs live in /var/log/shop, one file per part of the site. In the morning a colleague asks for a summary: which error codes there were and how many times each. "If you can't read one of the logs, that's fine — just keep the message about it separately, and I'll sort out the access."

The colleague keeps the summaries of earlier nights in ~/report/history.txt, one line per night.

Task

Make it so that:

~/report/errors-by-code.txt has, for each code found in ERROR lines, the number of such lines and the code itself — one line per code, separated by a space, the most frequent first;
~/report/failed.txt holds the error messages of the command that read the logs, and nothing else;
a line for this night appears at the end of ~/report/history.txt, in the same form as the earlier ones: 2026-10-19 and the total number of ERROR lines. The earlier lines must stay where they are.

Read the logs as yourself, without sudo: the summary covers what is open to you.

Lab goals

0 of 3

~/report/errors-by-code.txt lists the count and code of each error, most frequent first — not checked yet
~/report/failed.txt holds only the command's error messages — not checked yet
~/report/history.txt keeps the earlier lines and ends with a line for 2026-10-19 — not checked yet

Hints

0 of 3 open · they open one at a time

Hint 1

Show the solution after 3 failed checks

Basic commands · picked for this lab

Output redirection and pipelines

Cheat sheet

$_ — insert into the terminal

cmd > out.txt output to a file (overwrite)
cmd >> out.txt output appended to a file
cmd 2> err.txt errors to a file
cmd > all.txt 2>&1 output and errors in one file
cmd &> all.txt the same, shorter (bash)
cmd 2>/dev/null throw errors away
cmd < in.txt input from a file
cmd1 | cmd2 cmd1's output into cmd2's input
cmd 2>&1 | less into a pipe together with errors
cmd | tee out.txt both to the screen and to a file

Output and errors in separate files

bash

./backup.sh > backup.log 2> backup.err

The order of 2>&1 matters

bash

cmd > all.txt 2>&1     # both streams into the file
cmd 2>&1 > all.txt     # only output to the file, errors on screen

The shell reads redirections left to right. In the second line 2>&1 copies where stdout pointed at that moment — the screen.

Process output with a chain of commands

bash

sudo grep 'Failed password' /var/log/auth.log | wc -l
ps -eo rss,comm | sort -rn | head -n 5   # five processes by memory
du -sh /var/log/* 2>/dev/null | sort -rh | head

Each stage does one thing: grep selects, sort orders, wc counts, head trims. Commands like these — they read input and print processed text — are called filters.

Write to a file that needs root

bash

echo '127.0.0.1 app.local' | sudo tee -a /etc/hosts

sudo cmd > /etc/file won't work: your shell does the redirection without root rights. tee runs under sudo and writes the file itself.

Multi-line input right into a command

bash

cat > app.conf <<'EOF'
port = 8080
log = /var/log/app.log
EOF

Empty a log without deleting the file

bash

: > /var/log/app.log         # length 0, file and owner stay
sudo truncate -s 0 /var/log/app.log   # the same for a root-owned file

If a process keeps the log open, rm won't free the space, but emptying will.

Find out which stage of a pipeline failed

bash

false | true
echo "${PIPESTATUS[@]}"      # 1 0 — each stage's exit code
set -o pipefail              # in scripts: any failing stage shows

Permission denied with sudo … > file

Your shell opens the file, not sudo. Use | sudo tee file (or tee -a to append).

bash: $F: ambiguous redirect

The file name in the variable has a space, or the variable is empty. Quote the variable — > "$F" — and check that it's set.

bash: out.txt: cannot overwrite existing file

noclobber mode is on (set -o noclobber); it forbids > onto an existing file. Overwrite explicitly with >|, or turn the mode off with set +o noclobber.

Errors didn't end up in the file

> redirects stdout only. Add 2>&1 after > file, or use &>.

grep in a pipeline doesn't find the error text

The error went to stderr, and a pipe passes only stdout. Pass both streams: cmd 2>&1 | grep text.

The file is empty after sort file > file

The shell truncated file for writing before sort read it. Write to a new file and rename it with mv; sort also has -o file for this.

`>` overwrites without asking

One arrow instead of two, and a log or config is replaced by a single line. Always append with >>. An interactive shell can be protected with set -o noclobber.

`2>/dev/null` hides the cause

Suppressing errors is handy for the output of find /, but in scripts and while troubleshooting the errors are the answer. Read the error first, then decide whether to silence it.

Count the errors in the logs

Situation

Task

Signing in is needed to start the machine

Output and errors in separate files

The order of 2>&1 matters

Process output with a chain of commands

Write to a file that needs root

Multi-line input right into a command

Empty a log without deleting the file

Find out which stage of a pipeline failed

Permission denied with sudo … > file

bash: $F: ambiguous redirect

bash: out.txt: cannot overwrite existing file

Errors didn't end up in the file

grep in a pipeline doesn't find the error text

The file is empty after sort file > file

> overwrites without asking

2>/dev/null hides the cause

`>` overwrites without asking

`2>/dev/null` hides the cause