Storage / Stage 1 of 11

Out of space — which resource?

Not running Starting usually ~20 s

Step 7 of 8 · lab · 20 min

Find the actual space consumer

The file list says a huge image fills the data volume, yet df disagrees. Find out what really occupies the blocks and write a report without deleting anything.

Situation

A separate data volume is attached at /mnt/lab-data (its device is recorded in /srv/storage-lab/device.txt). It holds a cache, an archive and a large image. A colleague looked at ls -lh and was about to clean the volume "starting from the biggest file", but the df figures don't match that picture. Before anything gets deleted, the team needs an exact report of what really occupies the space on the volume.

Task

Create the report /home/student/storage-report.txt — exactly four key=value lines, in any order:

largest_allocated — the absolute path of the file with the most blocks allocated on the volume;
sparse_file — the absolute path of the sparse file;
sparse_apparent_bytes — that file's logical length in bytes;
sparse_allocated_bytes — how many bytes are allocated to it on disk.

Numbers are integers in bytes, with no units or spaces. Every line ends with a line feed (LF). The report is owned by you (student).

Constraints

Delete or change nothing on the volume: the files, their contents and the space they occupy must stay as they are.
Don't reformat or remount the volume, and leave /etc/fstab alone.

The tools you need are already installed. There's no internet on this machine.

Lab · on your own

No commands are given in this lab: type them in the terminal yourself. Goals are checked on the machine after every command.

Goals

0 of 4

largest_allocated names the volume's largest block consumer — not done yet
sparse_file and its sizes in bytes are correct — not done yet
The report is yours, has exactly four keys and unrounded numbers — not done yet
The volume, its files, their contents and allocated space are unchanged — not done yet

Ctrl+Enter

Starts the machine node1 · Ubuntu 24.04

The machine is starting

Hints

0 of 3

Report a problem

Lab goals

0 of 4

largest_allocated names the volume's largest block consumer — not done yet
sparse_file and its sizes in bytes are correct — not done yet
The report is yours, has exactly four keys and unrounded numbers — not done yet
The volume, its files, their contents and allocated space are unchanged — not done yet

node1 Ubuntu 24.04

Lab goals

0 of 4

largest_allocated names the volume's largest block consumer — not done yet
sparse_file and its sizes in bytes are correct — not done yet
The report is yours, has exactly four keys and unrounded numbers — not done yet
The volume, its files, their contents and allocated space are unchanged — not done yet

Storage and file systems · picked for this lab

Disk space: df, du, inode

Cheat sheet

$_ — insert into the terminal

df -h free space per partition
df -i free inodes per partition
du -xh --max-depth=1 / | sort -h top directories (within one filesystem)
ncdu / interactive overview
sudo lsof +L1 deleted-but-open files
sudo journalctl --vacuum-time=7d trim the system journal
sudo apt clean clear the package cache

df / du / ncdu

bash

df -h                             # free space by partition (human-readable)
df -i                             # free inodes by partition
df -h /var                        # a specific path (which partition it's on)
df -hT                            # + filesystem type

du -sh /var/*                     # size of each subdirectory of /var
du -h --max-depth=1 / | sort -h   # top "heavy" top-level directories
du -xh --max-depth=1 /            # -x: stay within one filesystem (skip /proc, mounts)
du -sh /var/log                   # how much space logs take up
ncdu /                            # interactive overview (the most convenient; apt install ncdu)

ls -lhS | head                    # the largest files in a directory
find / -xdev -type f -size +500M -exec ls -lh {} + 2>/dev/null   # files over 500 MB

-x/-xdev is the key to a clean result: count only the current filesystem, without descending into mounted and virtual ones (/proc, /sys).

What to clean up

bash

sudo journalctl --vacuum-time=7d  # trim the system journal to 7 days
sudo journalctl --disk-usage      # how much space the journal takes up
sudo apt clean                    # cache of downloaded packages
sudo apt autoremove --purge       # old kernels and dependencies
docker system df ; docker system prune -af   # docker images/layers (often huge)
du -sh ~/.cache/*                 # per-user caches

df says "full", but du can't find that much

A file was deleted but is still open by a process — the space isn't freed until it closes:

bash

sudo lsof +L1                     # files with 0 links but still open (see lsof)

Restart the process holding it (often a log that got rotated incorrectly).

No space left on device, even though `df -h` shows free space

You're out of inodes: df -i. Find the directory full of tiny files and clean it up:

bash

for d in /*; do printf '%8s %s\n' "$(find "$d" -xdev 2>/dev/null | wc -l)" "$d"; done | sort -rn

Logs filled up the disk

du -sh /var/log/*; set up rotation (logrotate) and the journal (journalctl --vacuum-time=, see journalctl).

Deleted files, but the space didn't come back

Either a deleted-but-open file (see lsof +L1 above), or it's a different partition — check df -h path.

Out of space — which resource?

Situation

Task

Constraints

Signing in is needed to start the machine

df / du / ncdu

What to clean up

df says "full", but du can't find that much

No space left on device, even though df -h shows free space

Logs filled up the disk

Deleted files, but the space didn't come back

No space left on device, even though `df -h` shows free space