A journal record has more fields than its line shows, and jq picks the ones you need.
A journal line shows the time, the machine, the program with its process number, and the text. The record itself keeps dozens of fields, and journalctl -o json prints them all (in the Wiki) — one JSON object per record, on one line. These are the fields needed most often:
| Field | What it holds |
|---|---|
MESSAGE |
the record's text |
PRIORITY |
the level as a number: 3 — err, 4 — warning, 6 — info |
_PID |
the process number |
_SYSTEMD_UNIT |
the unit of the service that wrote it |
__REALTIME_TIMESTAMP |
the time in microseconds since 1970, UTC |
The line python3[1076] names the program: if a machine runs several Python services, the line cannot tell them apart, but the _SYSTEMD_UNIT field can. jq picks the fields: jq -r '._SYSTEMD_UNIT' prints one field without quotes, and select(.PRIORITY == "3") keeps records at level err. Field values in the journal's JSON are strings, so the number 3 is in quotes.
What will sudo journalctl -k -o json -n 1 | jq -r '._SYSTEMD_UNIT' print — the unit field of the kernel's latest record?
-
journald only receives and stores the record. A record's unit is the service that wrote it, and a kernel record was written by the kernel.
-
jq does not treat a missing field as an error: to it the value is null, and that is what it prints.
-
The word kernel is in the
_TRANSPORTfield, and a kernel record has no_SYSTEMD_UNITfield: no service sent it.
A kernel record has no _SYSTEMD_UNIT field — journald sets it on service records — so jq prints null. That is why kernel errors gather under null when you count by unit.
Fields are handy for counting: jq -r prints a field of every record on its own line, and uniq -c folds identical lines into one with a count in front — for example, how many errors each service wrote. uniq folds only neighbouring lines, so sort goes before it, and sort -rn after it lifts the biggest number to the top. Kernel records gather under null in such a count.
Find the noisiest service
The machine runs several Python services, and all of them log under the name python3. Find out which service wrote the most records at level err and above, and put its unit name in ~/noisy.txt.
Goals · 0 of 1
- Put the unit of the service with the most errors in ~/noisy.txt not done yet