Linux Labs
Where to start First steps track · 15 stages
Linux Labs
EN

Regular expressions — patterns for grep, sed and awk

6 min Ubuntu 24.04 · GNU grep 3.11 · GNU sed 4.9

A regular expression is a pattern that many strings match; grep searches by it, sed replaces by it, awk selects lines.

Cheat sheet

8 commands Printable version
  • grep -i 'error' app.log ignore case
  • grep -E 'error|fail' app.log either word
  • grep 'error\|fail' app.log the same in GNU basic syntax
  • grep -w 'err' app.log a whole word: not terror
  • grep -oE '\berr\b' app.log the same with word boundaries
  • grep -E '^[0-9]+$' file a line of digits only
  • grep -F '10.20.0.15' file literally, no special characters
  • sed -E 's/^(\w+) *= */\1=/' app.ini group \1 in a replacement

Overview

A regular expression is a pattern that many strings match: err(or)? matches both err and error. grep searches lines by a pattern, sed replaces text, and awk picks the lines to act on. Put the pattern in single quotes so the shell leaves *, $ and \ alone.

GNU grep and sed have two syntaxes. They can express the same things; the difference is the backslash:

basic (BRE) extended (ERE)
where grep, sed by default grep -E, sed -E, awk
+ ? { } | ( ) with \: \+, \|, \(…\) without \

The third dialect is PCRE in grep -P: it has \d, lazy repeats *? and (?:…).

Pattern elements

Pattern Meaning
. any single character
[abc], [^abc] one of these, anything but these
[0-9], [[:digit:]] a digit
*, +, ? 0 or more, 1 or more, 0 or 1 time
{2,5} 2 to 5 times
^, $ start and end of the line
\b, \<, \> a word boundary
\w, \s a letter, digit or _; a whitespace character
a|b a or b
(…) a group: repeat it whole or use \1 in a replacement
\., \* a literal dot, a literal asterisk

GNU grep and sed understand \b, \<, \>, \w and \s without -P. A word boundary is the spot between a letter, digit or _ and any other character:

text 
student@ubuntu:~$ grep -oE '\berr\b' app.log
err
student@ubuntu:~$ grep -w err app.log
err=5 retry

terror did not match — a letter stands before err. ERROR did not match because of case; -i catches it.

Greediness

* and + take as many characters as they can:

text 
student@ubuntu:~$ echo '<a><b>' | grep -Eo '<.*>'
<a><b>
student@ubuntu:~$ echo '<a><b>' | grep -Eo '<[^>]*>'
<a>
<b>

[^>]* means "any characters except >", so the match ends at the first >. Lazy .*? exists only in grep -P.

Recipes

4

A word in any case and spelling

text 
student@ubuntu:~$ grep -iE '^(error|err)\b' app.log
ERROR disk full
Error: timeout
err=5 retry

An IP address

bash 
grep -oE '[0-9]{1,3}(\.[0-9]{1,3}){3}' app.log   # any address
grep -oE '10\.20\.0\.15\b' app.log               # only this one

A config without comments and empty lines

bash 
grep -vE '^\s*(#|$)' /etc/ssh/sshd_config

Swap parts of a line

bash 
echo 'key = "va lue"' | sed -E 's/^(\w+) *= *"(.*)"$/\1=\2/'
text 
key=va lue

Groups ( ) remember parts of the match; \1 and \2 put them into the replacement.

Troubleshooting

5

grep 'a|b' finds nothing

Without -E, | is a plain character and grep looks for the string a|b literally. Code 1, no output. Write grep -E 'a|b' or grep 'a\|b'.

\d matches the letter d

\d exists only in PCRE. In grep -E it is just the letter d:

text 
student@ubuntu:~$ grep -E '\d+' app.log
ERROR disk full
ip 10.20.0.15 and 10.20.0.150

disk and and matched. For digits write [0-9] or [[:digit:]], or use grep -P '\d+'.

Unmatched ( or (

A bracket in the pattern is not closed: grep -E '(err' exits with code 2. For [ the message is Unmatched [, [^, [:, [., or [=. To match the bracket itself, escape it: \(.

The dot matches too much

grep -o '10.20.0.15' also finds the start of 10.20.0.150: a dot is any character, and the end of the address is open. Escape the dots and close the pattern with a boundary: '10\.20\.0\.15\b'.

The shell expanded the asterisk

Unquoted, grep error.* app.log may let the shell replace error.* with matching file names. Always put the pattern in single quotes.

See also

  • grep — searching lines by a pattern.
  • sed-awk — replacing by a pattern and processing fields.
  • globbing — file name patterns: they look alike but mean other things.
  • test-conditions — [[ $x =~ … ]] in bash.
  • text-tools — sort, uniq, cut to make sense of the matches.
  • find — finding files by name and -regex.

Found an inaccuracy? Report an error in the article

Practice on this topic

Lessons on this topic

Topics
Loading…