Where to start First steps track · 15 stages
Linux Labs

wget — downloading files over the network

4 min GNU wget

Downloads files with resume and recursion; -r without -np and -l has no brakes and can mirror a whole site.

Cheat sheet

6 commands Printable version
  • wget URL download it
  • wget -O name URL under your own name
  • wget -c URL resume it
  • wget -O- URL | cmd to stdout (into a pipeline)
  • wget -i urls.txt a list of URLs
  • wget -r -np -k -p URL mirror a directory

Overview

wget downloads files over HTTP/HTTPS/FTP. Unlike curl (a general API client), wget is built for downloading: resume, recursion, mirrors, retries.

bash
wget https://site/file.tar.gz     # download into the current directory
wget -O out.tar.gz https://site/x # save under your own name

Options

9

-r (recursion) without -np (don't go above the starting directory) and without -l N (depth limit) has no natural limit — the download will follow every link on the site, including external ones, until you stop it manually. -c resumes an interrupted download only if the server supports Range — otherwise it starts over from zero regardless of the flag.

Option What it does Example
-O FILE Save under a different name; -O- sends the content to stdout instead. wget -O out.tar.gz https://site/x
-c Resume an interrupted download; only works if the server supports Range. wget -c https://site/big.iso
-P DIR Directory to save the file into. wget -P /downloads https://site/x
-q Quiet mode — no progress output, for scripts. wget -q https://site/x
-i FILE Download a list of URLs from a file, one per line. wget -i urls.txt
--limit-rate RATE Limit download speed to avoid saturating the link. wget --limit-rate=2m https://site/x
-r --recursive Recursive download; without -np and -l it follows every link with no limit. dangerous wget -r -np -l2 https://site/docs/
-np --no-parent During recursion, never ascend above the starting directory. wget -r -np https://site/docs/
-k --convert-links After mirroring, rewrite links so the copy browses correctly offline. wget -r -np -k https://site/docs/

Nothing found. Try a shorter query.

Recipes

1

Commands

bash
wget -c https://site/big.iso              # resume an interrupted download
wget -nv -P /downloads https://site/x     # quiet, into a directory
wget -O - https://site/script.sh | less   # to stdout (view without saving)
wget -i urls.txt                          # download a list of URLs from a file
wget --limit-rate=2m https://site/x       # limit speed to 2 MB/s
wget -t5 --retry-connrefused https://site/x   # persistent retries (a flaky network)
wget --header='Authorization: Bearer TOK' https://api/x   # with a token
wget --user=u --password=p ftp://host/file

# mirror a site directory (watch the volume):
wget -r -np -k -p https://site/docs/      # recursive, no parent, fix links, +assets
wget -r -l2 -np https://site/             # limit depth with -l

Troubleshooting

5

ERROR: cannot verify ... certificate

A TLS certificate/chain problem. The right fix is to repair the CA (ca-certificates, see openssl-tls); for one-off debugging, --no-check-certificate (NEVER in production).

The download broke off

wget -c URL resumes from where it stopped, if the server supports Range. For reliability in scripts, use -c -t0 --retry-connrefused.

Got HTML instead of the file

The link points to a page/redirect/login, not the file. Check the URL and redirects (wget -S shows the response headers). For APIs, curl is better.

It downloads endlessly / mirrors too much

Recursion -r without -np/-l has no brakes. Limit it: -np (no parent directory), -l N (depth), -A/-R (extension filters).

Slow / saturating the link

--limit-rate= to avoid saturating the link; -q to keep a script's log clean.

Danger zone

--no-check-certificate disables TLS certificate verification entirely — traffic can be intercepted along the way with no warning at all. Use it only for one-off debugging on a trusted network, never in production: the right fix is to repair the certificate chain (ca-certificates). Recursive download without -np/-l can drag down a whole site, including external links — before running -r in production, limit the depth and the domain.

See also

  • curl — HTTP/HTTPS requests from the CLI — debugging APIs and web services, headers, methods.
  • tar — Creating and extracting tar/gz/xz archives; backing up directories.
Loading…