unattended-upgrades: security updates with no one at the keyboard

unattended-upgrades installs updates with no one at the keyboard. apt timers start it:

student@ubuntu:~$ systemctl cat apt-daily-upgrade.timer \
> | grep -E 'OnCalendar|Randomized|Persistent'
OnCalendar=*-*-* 6:00
RandomizedDelaySec=60m
Persistent=true

Persistent=true means that if the machine was off at 6:00, the run happens after boot. The unattended-upgrades.service unit ("Unattended Upgrades Shutdown") does not install anything: it waits for shutdown so as not to cut an installation short.

Steps 1 and 2 are switched on by 20auto-upgrades; the number is the interval in days:

student@ubuntu:~$ cat /etc/apt/apt.conf.d/20auto-upgrades
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

On the lab machine (unattended-upgrades 2.9.1) 20auto-upgrades turns on both steps, but the timers and the service are off (disabled), and there is no network. The old package lists hold about a hundred and fifty updates, over a hundred of them from noble-security, and a trial run fails at the download:

student@ubuntu:~$ sudo unattended-upgrade --dry-run -v
Could not figure out development release: Distribution data outdated.
  Please check for an update for distro-info-data. See
  /usr/share/doc/distro-info-data/README.Debian for details.
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=noble, o=Ubuntu,a=noble-security,
  o=UbuntuESMApps,a=noble-apps-security,
  o=UbuntuESM,a=noble-infra-security
Initial blacklist:
Initial whitelist (not strict):
The URI http://security.ubuntu.com/ubuntu/pool/main/u/util-linux/
  bsdutils_2.39.3-9ubuntu6.6_amd64.deb failed to download, aborting

The first line is about a development release and has nothing to do with an LTS server. The trial run takes about a minute: the program goes through every candidate.

What gets installed

The origins are the Allowed-Origins list in 50unattended-upgrades. A package is installed if its origin (o= — who publishes it, a= — the archive suite) matches one of the lines:

Unattended-Upgrade::Allowed-Origins {
	"${distro_id}:${distro_codename}";
	"${distro_id}:${distro_codename}-security";
	"${distro_id}ESMApps:${distro_codename}-apps-security";
	"${distro_id}ESM:${distro_codename}-infra-security";
};

${distro_id}:${distro_codename} is the noble release itself, which does not change after release. The -updates line in the file is commented out, so by default only security updates arrive. The ESM suites work with an Ubuntu Pro subscription attached.

Restart after an update

An update changes files on disk. A running program keeps executing the old ones: its file is deleted but still open — an open unlinked file. Until the service is restarted, the fix does not apply to it. The demo-sleep binary was replaced, and the demo-old service keeps running:

student@ubuntu:~$ sudo lsof +L1
COMMAND    PID USER  FD   TYPE DEVICE SIZE/OFF NLINK NODE NAME
demo-slee 4214 root txt    REG  253,4    35336     0 4464
  /usr/local/bin/demo-sleep (deleted)
student@ubuntu:~$ sudo readlink /proc/4214/exe
/usr/local/bin/demo-sleep (deleted)

An old shared library stays in memory the same way — marked (deleted) in /proc/<PID>/maps (see proc-filesystem). After systemctl restart demo-old the process starts from the new file, and the mark is gone.

Restarting services does not update the kernel. A package that needs a reboot creates /var/run/reboot-required; the machine reboots by itself only if Automatic-Reboot is on.

needrestart

needrestart finds processes running old files and names their services. -r sets the mode: l only lists, a restarts, i asks. -b prints lines for scripts:

student@ubuntu:~$ sudo needrestart -b
NEEDRESTART-VER: 3.6
NEEDRESTART-KCUR: 6.8.0-124-generic
NEEDRESTART-KEXP: 6.8.0-124-generic
NEEDRESTART-KSTA: 1
NEEDRESTART-SVC: demo-old.service
student@ubuntu:~$ sudo needrestart -r l
Running kernel seems to be up-to-date.

Services to be restarted:
 systemctl restart demo-old.service

KCUR is the running kernel, KEXP the newest installed one, KSTA: 1 means they match and no reboot is needed. SVC is a service to restart.

On Ubuntu the apt hook calls needrestart after every installation, and by default it restarts services by itself — /etc/needrestart/needrestart.conf says so:

# UBUNTU: the default restart mode when running as part of the APT
# hook is 'a', unless a specific UI is configured (see below).

Some services it leaves alone even in mode a: their list is the $nrconf{override_rc} key. Such services go to a separate list:

student@ubuntu:~$ sudo needrestart -r a
Running kernel seems to be up-to-date.
Restarting services...
Service restarts being deferred:
 systemctl restart demo-lib.service

Here demo-lib is deferred by a line in /etc/needrestart/conf.d/demo.conf:

$nrconf{override_rc}{qr(^demo-lib)} = 0;

A deferred service is restarted by hand when it is safe.

Logs

/var/log/unattended-upgrades/ is readable by root and the adm group:

On the lab machine the directory is empty: the upgrade has never run, and unattended-upgrades.service is off. Timer runs show in journalctl -u apt-daily-upgrade.

• apt — installing and upgrading packages by hand.
• systemd-timers — how the apt-daily* timers work.
• lsof — lsof +L1 and open unlinked files.
• proc-filesystem — /proc/<PID>/maps and exe.
• systemctl — restarting services.
• kernel — a new kernel and the reboot.