Linux Labs

Observability: metrics, alerts, logs / Stage 7 of 9

Logs in one place

Not running Starting usually ~20 s
Leave the lesson

Step 1 of 7 · on a machine · 4 min

No lines on the collector

What you will learn

  • Read rsyslog rules and work out which files a line will end up in.
  • Forward a machine's lines to a collector and check delivery with a test logger line.
  • Turn on reception on the collector with the imtcp and imudp modules and sort lines by machine with a template.

The lab-orders application has just crashed. A colleague looks for its errors on the log collector — the server where machines send their journal lines — and cannot find them. On the training machine the collector runs next to the application: it is the lab-collector service, a second rsyslog instance on address 127.0.0.1, port 514. It keeps the machines' lines in /var/log/remote.

Exercise · look

Look at the collector

Look at which machines' lines are on the collector in /var/log/remote, where this machine keeps the lab-orders errors and what this machine calls itself.

First, predict

The lab-orders errors are in this machine's syslog. Will they be on the collector?

Now start the exercise, run the command and compare your prediction with the output.

Sign in and start The machine is available after signing in
Check yourself 1 question

The collector has lines from web1 and db1 but none from this machine, although the lab-orders errors are in its /var/log/syslog. What follows from this?

  • The lab-orders lines are in /var/log/syslog — this machine's rsyslog wrote them.

  • The collector has lines from web1 and db1, and systemctl status lab-collector shows the service as active. It works — nobody sent it anything.

  • The lab-orders errors are fresh, and the web1 lines on the collector are tens of minutes older — nobody removed them.

This machine's rsyslog writes lines to its own files and does not send them to the collector.

Ubuntu 24.04
Loading…