Linux Labs

Bash: text processing and scripts / Stage 2 of 9

sed and awk: editing and fields of a stream

Not running Starting usually ~20 s
Leave the lesson

Step 5 of 9 · on a machine · 4 min

A sum for every client

A single total for the whole file is not the limit. awk builds a "client — total" table in one pass.

Instead of one variable you take an array — a variable with cells, each cell with a name of its own. s[$2] += $3 adds a payment to the cell named after the client in the second field: an acme payment goes to s["acme"], a hooli payment to s["hooli"]. In the END block, the for (k in s) loop walks every cell, and print k, s[k] prints the cell's name and its total.

bash
awk -F',' '{s[$2] += $3} END {for (k in s) print k, s[k]}' payments.csv
Check yourself 1 question

The accountant needs this table with the clients in alphabetical order. What will you do?

  • Look at the output: umbrella comes before acme. awk walks the array cells in an order of its own.

  • That changes the order of the fields within a line, while the order of the lines stays the same: awk walks the cells in an order of its own.

  • awk picks the order in which it walks the cells, and the order of the file's lines does not set it. What you can order is the output itself — | sort.

for (k in s) walks the array cells in an order awk picks itself — it has nothing to do with the alphabet or the order of the file's lines. When you need an order, pass the output to sort.

The client 0 line in the output is the file's header: awk opened a cell for it too. You can put a condition before the action, and the action runs on the lines where it holds. awk keeps the line number in NR, so NR > 1 { … } skips the header. A condition is any comparison: $3 > 1000 { … } takes the payments over a thousand.

Key points

  • The array s[$2] += $3 keeps a separate total for every value of a field.
  • for (k in s) in the END block prints the cells in an order awk picks; sort after awk sets the order.
  • A condition before the action, such as NR > 1, leaves the action the lines it needs.
Ubuntu 24.04
Loading…