Linux Labs
Where to start First steps track · 15 stages
Linux Labs
EN

Boot targets: default, multi-user, rescue and emergency

11 min Ubuntu 24.04 · systemd 255

A target is a group of units systemd brings the boot up to; rescue and emergency are the targets for repairs.

Cheat sheet

8 commands Printable version
  • systemctl get-default the default boot target
  • sudo systemctl set-default multi-user.target
  • systemctl list-units --type=target active targets
  • systemctl list-dependencies multi-user.target what it pulls in
  • systemctl list-dependencies --reverse network-online.target
  • systemctl is-system-running running, degraded, starting
  • sudo systemctl rescue switch to rescue now
  • sudo systemctl default back to the default target

Overview

A target is a unit with no work of its own: a name for a group of units and a point in the boot order. systemd boots the system up to a target: it takes default.target, starts everything it needs and their dependencies. The server targets form a chain:

text 
sysinit.target → basic.target → multi-user.target → graphical.target
Target What is ready
sysinit.target file systems mounted, udev and the journal running
basic.target sockets, timers and slices in place
multi-user.target server services, login over SSH and on the console
graphical.target graphical login on top of multi-user.target
rescue.target file systems mounted, no services, a root shell on the console
emergency.target almost nothing, a root shell on the console

A target itself is a short unit. Here is multi-user.target:

ini 
[Unit]
Description=Multi-User System
Requires=basic.target
Conflicts=rescue.service rescue.target
After=basic.target rescue.service rescue.target
AllowIsolate=yes

AllowIsolate=yes allows switching the whole machine to this target — systemctl isolate: systemd stops everything the target does not need.

The default target

default.target is a symbolic link to a real target. The systemd package installs a link to graphical.target, and the lab machine has its own on top of it, in /etc:

text 
student@ubuntu:~$ systemctl get-default
multi-user.target
student@ubuntu:~$ readlink /etc/systemd/system/default.target
/usr/lib/systemd/system/multi-user.target

On a server without graphics graphical.target adds nothing to multi-user.target, so both give the same boot.

How a service gets into the boot

A service starts at boot if some target wants it. The line WantedBy=multi-user.target in the [Install] section does nothing by itself: systemctl enable turns it into a link in the target's directory.

text 
student@ubuntu:~$ ls /etc/systemd/system/multi-user.target.wants/ \
> | grep -E '^(cron|nginx|ssh)\.'
cron.service
nginx.service
ssh.service

systemctl list-dependencies shows the same as a tree: ● means the unit is running, ○ means it is not:

text 
student@ubuntu:~$ systemctl list-dependencies multi-user.target
multi-user.target
● ├─cron.service
○ ├─dmesg.service
  …
● ├─nginx.service
  …
● ├─ssh.service
  …
● ├─basic.target

A unit without an [Install] section cannot be enabled: systemctl status shows it as static. Such a unit starts only as a dependency of another one.

Wants and After

A target pulls in units like any unit does: Wants= and Requires= say what to start together, After= and Before= in which order. After= starts nothing by itself, so a dependency is written as a pair — Wants=x and After=x. The table of lines and what happens without the pair — systemd.

network-online.target

network.target means "network services are started" — an interface may still have no address. network-online.target means the network is configured: the waiting service systemd-networkd-wait-online holds it. A service that needs the network at start — a client of a database on another machine, a program that listens on a specific address — writes:

ini 
[Unit]
Wants=network-online.target
After=network-online.target

Who waits for the network on the lab machine:

text 
student@ubuntu:~$ systemctl list-dependencies --reverse \
> network-online.target
network-online.target
○ ├─docker.service
○ ├─fwupd-refresh.service
○ ├─iscsid.service
● ├─nginx.service
○ └─open-iscsi.service

The nginx unit has exactly that pair: Wants=network-online.target and After=network-online.target remote-fs.target nss-lookup.target.

On the lab machine the waiting service is turned off (masked): the machine has no network to the outside, and waiting would hold the boot until its timeout. So here network-online.target comes right after network.target.

Rescue and emergency

Both targets are for when a normal boot does not finish. Instead of services they start a root shell on the console and print:

text 
You are in emergency mode. After logging in, type "journalctl -xb"
to view system logs, "systemctl reboot" to reboot, or "exit"
to continue bootup.
rescue.target emergency.target
file systems mounted, including those from /etc/fstab only the root, often read-only
services and network none none
when a service or a setting is broken, file systems are fine a mistake in /etc/fstab, a file system does not mount

In emergency the root stays as the kernel mounted it: with ro on the kernel command line (cat /proc/cmdline) it is read-only. You can edit files after mount -o remount,rw /.

Before the shell systemd asks for the root password. On the lab machine root is locked (sudo passwd -S root → root L …), and /etc/systemd/system/emergency.service.d/ and rescue.service.d/ hold Environment=SYSTEMD_SULOGIN_FORCE=1: with it the shell opens without a password. On a server without this line and with a locked root you cannot get in — set the root password beforehand (sudo passwd root).

Targets and runlevels

The old SysV runlevel numbers are links to targets:

Runlevel Target
0 poweroff.target
1 rescue.target
2, 3, 4 multi-user.target
5 graphical.target
6 reboot.target

runlevel on the lab machine prints N 3: there is no previous level, the current one is 3, that is multi-user.target.

Recipes

4

Change the default target

text 
student@ubuntu:~$ sudo systemctl set-default graphical.target
Removed "/etc/systemd/system/default.target".
Created symlink /etc/systemd/system/default.target
  → /usr/lib/systemd/system/graphical.target.

The target changes at the next boot. To switch to it now — sudo systemctl isolate <target>.

Who starts a unit at boot

bash 
systemctl is-enabled nginx                  # enabled, disabled, static
systemctl show -p WantedBy,RequiredBy nginx  # which targets want it
systemctl list-dependencies --reverse nginx.service

Boot into rescue or emergency

  1. In the GRUB menu select an entry and press e.
  2. At the end of the line that starts with linux, add systemd.unit=rescue.target or systemd.unit=emergency.target.
  3. Ctrl+X boots with this line, once.

On the lab machine the GRUB menu is hidden (GRUB_TIMEOUT=0) and goes to the VM screen (GRUB_TERMINAL=console), while the lesson terminal is the ttyS0 serial console: kernel and systemd messages go there, GRUB's do not. More about the boot loader — boot-grub.

Fix things from emergency and boot on

bash 
journalctl -xb | grep -i -E 'failed|timed out'  # what broke
mount -o remount,rw /           # root writable
nano /etc/fstab                 # fix the line
systemctl daemon-reload         # reread fstab
mount -a                        # mount everything from fstab
systemctl default               # continue the boot

mount -a without errors means fstab is fine, and systemctl default takes the boot to the default target.

Troubleshooting

5

You are in emergency mode

The boot was stopped by a dependency of local-fs.target: most often an /etc/fstab line with a device that does not exist or a wrong UUID. journalctl -xb shows Timed out waiting for device and Dependency failed for …. Fix the line using blkid; mount a disk the system can live without with nofail. More — mount.

The service works, but is not running after a reboot

It was not enabled: systemctl is-enabled <service> prints disabled, and no target wants it. sudo systemctl enable <service> creates the link in multi-user.target.wants. static means the unit has no [Install] section — add one.

The service starts before the network and fails

The unit has no After=network-online.target, or only After=network.target. It needs the pair of lines from boot-targets; Wants= without After= sets no order.

Dependency failed for …

A unit with Requires= (or BindsTo=) did not start because its dependency failed to start. The cause is in the dependency's journal: journalctl -b -u <dependency>.

is-system-running: degraded

The boot finished, but some unit failed. systemctl --failed shows which; after the fix — sudo systemctl reset-failed.

Danger zone

1

rescue and isolate over SSH

systemctl rescue, systemctl emergency and isolate into a target without the network stop ssh.service: the SSH session drops, and you can come back only from the console. Switch targets when the console is at hand.

Terms

1
target
A systemd unit with no work of its own — a name for a group of units and a system state that systemd brings the boot up to: multi-user.target on a server, rescue.target for repairs. Also appears in: System boot and GRUB, ln, systemd, Symbolic and hard links and 31 more

See also

  • systemd — units, drop-ins and socket activation.
  • systemd-analyze — boot time and the dependency chain.
  • boot-grub — the boot loader and the kernel command line.
  • systemctl — enabling units and their state.
  • mount — /etc/fstab and mounting.
  • journalctl — the journal of this boot.

Found an inaccuracy? Report an error in the article

Practice on this topic

Lessons on this topic

Topics
Loading…