Linux Labs
Where to start First steps track · 15 stages
Linux Labs
EN

nginx — web server and reverse proxy

13 min Ubuntu 24.04 · nginx 1.24

nginx serves sites from disk and forwards requests to applications; it picks a site by Host, nginx -t checks the config.

Cheat sheet

8 commands Printable version
  • sudo nginx -t check the config (sudo only)
  • sudo systemctl reload nginx apply, keep connections
  • sudo nginx -T | grep -n server_name which sites nginx sees
  • ls -l /etc/nginx/sites-enabled/ enabled sites
  • curl -i -H 'Host: docs.lab.test' http://127.0.0.1/ who answers
  • sudo tail -n 5 /var/log/nginx/error.log why 404, 403, 502
  • sudo tail -n 5 /var/log/nginx/access.log what was asked, the code
  • sudo ss -tlnp | grep nginx which ports it listens on

Overview

nginx is a web server: it serves site files from disk and works as a reverse proxy — it forwards a request to an application and returns its response to the client. The nginx service runs under systemd: the master process reads the config as root, worker processes serve requests as www-data. Everything nginx opens on disk or in a socket is checked against www-data's permissions.

How nginx picks a site

Each site is a server block. A request first goes to the blocks that listen on its address and port (listen), then nginx compares the Host header with their server_name.

nginx 
server {
    listen 80;
    server_name docs.lab.test;
    root /var/www/docs;
}

If a name matches, that block answers. If none matches, the default server answers: the block with listen 80 default_server, or the first block for that port if none is marked. On Ubuntu the default server on port 80 is the default site with the "Welcome to nginx!" page. So a typo in the name or a forgotten site gives someone else's page, not an error.

File path: root and location

nginx looks for a file at root + the request path. With root /var/www/docs a request for /guide/a.html means the file /var/www/docs/guide/a.html. For a directory request (/) it serves a file from the index list (index.html by default).

A location block sets rules for some paths. Of the prefixes the path starts with, the longest wins: location /api/ beats location / for /api/users.

Directive Path for the request /files/x.txt
location /files/ { root /srv/data; } /srv/data/files/x.txt
location /files/ { alias /srv/data/; } /srv/data/x.txt

root appends the whole path, alias replaces the matched prefix.

Proxy: proxy_pass

nginx 
location / {
    proxy_pass http://127.0.0.1:3000;
}

nginx opens a connection to the application — the upstream — and forwards the request. If it cannot connect, the client gets 502 Bad Gateway; if it gives up waiting, 504 Gateway Time-out. These are nginx's own responses: such requests never reach the application log. The backend can also listen on a unix socket: proxy_pass http://unix:/run/app/app.sock;.

Configuration

17

The Debian and Ubuntu layout:

  • /etc/nginx/nginx.conf — common settings, with an include of sites at the bottom;
  • /etc/nginx/sites-available/ — site files;
  • /etc/nginx/sites-enabled/ — links to enabled sites; nginx reads only these;
  • /var/log/nginx/access.log and error.log — the logs.

Enable a site

bash 
sudo ln -s /etc/nginx/sites-available/docs /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

A file in sites-available without a link in sites-enabled is invisible to nginx. To disable a site, remove the link; the file stays.

nginx -t checks the whole config and names the file and line of an error. reload re-reads the config without dropping connections; with an error in the config nginx keeps the old one. systemctl reload returns at once, while the new worker processes take connections a moment later: a curl on the same line may still see the old state.

Logs

access.log has a line for every request, error.log says why a request failed and what broke at start. The default access.log format is combined:

text 
127.0.0.1 - - [20/Oct/2026:10:03:58 +0000] "POST / HTTP/1.1" 405 166 "-"

Field by field: $remote_addr (client address), -, $remote_user, [$time_local], "$request" (method, path, version), $status (code), $body_bytes_sent (body size), "$http_referer", "$http_user_agent" — the user agent "curl/8.5.0" did not fit in the example. log_format defines your own format — in a site file outside the server block:

nginx 
log_format timed '$remote_addr [$time_local] "$request" '
                 '$status $body_bytes_sent $request_time';
access_log /var/log/nginx/app.log timed;

$request_time is how many seconds nginx spent on the request. Files with .1 and .2.gz next to a log are older ones that logrotate moves aside and compresses on schedule (/etc/logrotate.d/nginx).

Proxy: path and headers

Whether there is a URI after the address in proxy_pass decides which path the application receives:

location /api/ and proxy_pass Request /api/x goes out as
http://127.0.0.1:3003 /api/x — path unchanged
http://127.0.0.1:3003/ /x — /api/ replaced with /

By default the application sees Host: 127.0.0.1:3003 — the address from proxy_pass. The site name and the client address are passed in headers:

nginx 
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

HTTPS and the redirect from HTTP

nginx 
server {
    listen 443 ssl;
    server_name shop.lab.test;
    ssl_certificate     /etc/nginx/tls/fullchain.pem;
    ssl_certificate_key /etc/nginx/tls/shop.key;
    root /var/www/shop;
}
server {
    listen 80;
    server_name shop.lab.test;
    return 301 https://$host$request_uri;
}

ssl_certificate is the chain file: the site certificate first, the intermediate after it. $request_uri is the path with the query after ?: http://shop.lab.test/x?y=1 redirects to https://shop.lab.test/x?y=1. Certificates and how they are checked — openssl-tls.

Directive What it does Example
listen server The block's address and port; default_server makes it the default server, ssl turns on TLS. listen 443 ssl;
server_name server Site names nginx compares the Host header against. server_name docs.lab.test www.docs.lab.test;
root server, location The site directory; file path = root + request path. root /var/www/docs;
index server, location Which file to serve when a directory is requested. index index.html;
location server Rules for paths with this prefix; the longest matching prefix wins. location /api/ { … }
alias location A directory that replaces the matched location prefix (root appends the whole path). alias /srv/data/;
try_files server, location Check files in order and serve the first that exists, otherwise the last option. try_files $uri $uri/ =404;
proxy_pass location Forward the request to a backend; a URI after the address replaces the matched location prefix. proxy_pass http://unix:/run/app/app.sock;
proxy_set_header server, location A request header for the backend; without it Host is the address from proxy_pass. proxy_set_header Host $host;
proxy_read_timeout server, location How long to wait for the backend's response before 504. proxy_read_timeout 30s;
ssl_certificate server The certificate file; with an intermediate CA, the site certificate first, then the intermediate. ssl_certificate /etc/nginx/tls/fullchain.pem;
ssl_certificate_key server The private key for the first certificate in the ssl_certificate file. ssl_certificate_key /etc/nginx/tls/site.key;
return server, location Answer immediately with a code; 301 with a URL is a redirect. return 301 https://$host$request_uri;
client_max_body_size http, server, location The largest request body; anything bigger gets 413. client_max_body_size 20m;
error_log http, server Where to write errors; level warn, error, crit and above. error_log /var/log/nginx/docs.error.log;
access_log http, server, location Where and in which format to log a line per request. access_log /var/log/nginx/app.log timed;
log_format http A custom log format built from variables — $status, $request_time and others. log_format timed '$remote_addr "$request" $status $request_time';

Nothing found. Try a shorter query.

Recipes

2

Who will answer a name

bash 
curl -i -H 'Host: docs.lab.test' http://127.0.0.1/
sudo nginx -T | grep -n 'server_name\|listen'

The first command shows the answer as a client sees it, the second which listen and server_name nginx actually read, in file order.

Find where the proxy breaks

bash 
curl -s -o /dev/null -w '%{http_code}\n' http://app.lab.test/
sudo tail -n 1 /var/log/nginx/error.log
curl -i http://127.0.0.1:3000/        # the application directly

The error.log line names the upstream — where nginx went. If the application answers directly, the problem is in proxy_pass or permissions; if not, it is in the application.

Troubleshooting

11

Someone else's site answers

The request's Host matched no server_name, so the default server answered. Check the name in server_name and that the site's link exists in sites-enabled: sudo nginx -T | grep server_name.

404 Not Found

error.log has the path nginx built: open() "/var/www/portal/nope.html" failed (2: No such file or directory). Compare it with the real path of the file: usually root or alias is wrong.

403 Forbidden

(13: Permission denied) — www-data cannot read the file or enter a directory on the path; namei -l shows permissions along the whole path. directory index of "…" is forbidden — the directory has no file from index.

502 Bad Gateway

nginx did not open a connection to the backend. The error.log line names the reason:

In error.log Meaning
(111: Connection refused) nothing listens on the address or socket
(13: Permission denied) www-data has no right to the socket
(2: No such file or directory) no socket file at that path

In full: connect() failed (111: Connection refused) while connecting to upstream, … upstream: "http://127.0.0.1:3001/". For a socket — connect() to unix:/run/app/app.sock failed (13: Permission denied). nginx logs errors 13 and 2 at level [crit] and 111 at [error]: a search for [error] alone misses them. A socket file left behind after the application stopped also gives 111. Socket permissions and paths — unix-sockets.

504 Gateway Time-out

There is a connection but no answer: upstream timed out (110: Connection timed out) while reading response header from upstream. The application hangs or answers slower than proxy_read_timeout (60 s by default).

nginx -t: test failed

  • unexpected "}" in …:5 — the line before } has no ;;
  • a duplicate default server for 0.0.0.0:80 — default_server is set in two blocks on one port;
  • open() "/run/nginx.pid" failed (13: Permission denied) — nginx -t was run without sudo.

conflicting server name

[warn] conflicting server name "docs.lab.test" on 0.0.0.0:80, ignored — one name sits in two blocks on one port. The test still passes, but requests for that name go to the block loaded first, and the second never gets them. Find both:

bash 
grep -n 'docs.lab.test' /etc/nginx/sites-enabled/*

grep -r on the directory does not work here: it skips the links in sites-enabled.

key values mismatch

SSL_CTX_use_PrivateKey(…) failed (… key values mismatch) — the first certificate in the ssl_certificate file does not match the key. Either the key is wrong, or the chain file has the intermediate first. The site certificate goes first.

The new certificate is not served

nginx reads certificates at start and on reload. After replacing the file — sudo nginx -t && sudo systemctl reload nginx, then check a second later.

The service does not start: port in use

sudo ss -tlnp | grep ':80 ' shows who holds the port. Stop it or change listen.

A new listen did not apply

Changing the address on the same port (listen 80 → listen 127.0.0.1:80) does not work through reload: systemctl reload returns 0, nginx stays on the old sockets, and error.log says bind() to 127.0.0.1:80 failed (98: Address already in use). Only sudo systemctl restart nginx applies such a change; sudo ss -tlnp | grep nginx shows what really listens.

Danger zone

1

restart with an error in the config

On reload with an error the old config keeps working; restart stops nginx and cannot start it again: every site on the machine is down. Before applying anything — sudo nginx -t.

Terms

6
HTTP virtual host
A name-selected site sharing an address and port with other sites. Also appears in: HTTP
reverse proxy
A server that accepts client requests, forwards them to an application behind it and returns the application's response. nginx with proxy_pass works this way. Also appears in: HTTP, AppArmor, Grafana, HAProxy and 2 more
upstream (backend)
The application behind a reverse proxy that nginx forwards requests to — an address and port or a unix socket from proxy_pass. nginx error.log names it in the upstream field. Also appears in: docker compose, HAProxy, fail2ban, restic and 5 more
default server
The server block nginx picks when the request's Host matches no server_name on that address and port: the one marked default_server, or the first one if none is marked. Also appears in: HTTP, TLS and certificates
document root
The directory nginx builds file paths from: the request path is appended to root — root /var/www/docs and /a.html give /var/www/docs/a.html. Also appears in: Capabilities: root split into parts, PAM: login checks, password policy and lockout, Service hardening: User=, ProtectSystem, NoNewPrivileges, Docker and 70 more
location block
A block inside server with rules for part of the paths: location /api/ catches requests whose path starts with /api/. Of the matching prefixes the longest wins.

See also

  • http — request, response, status codes and the Host header.
  • unix-sockets — a backend on a socket: permissions, paths, errors 13 and 2.
  • openssl-tls — certificates, the chain, verification errors.
  • curl — requests to a site from the console.
  • systemctl — reload, restart, status.
  • journalctl — the service log when nginx does not start.
  • ss — who listens on ports 80 and 443.
  • Fix nginx — lab: nginx does not start.
  • Set up an nginx reverse proxy — lab: a proxy to an application.

Found an inaccuracy? Report an error in the article

Practice on this topic

Lessons on this topic

Tools on this topic

Topics
Loading…