Networking / Stage 5 of 14 · step 8 of 9

Ports: start the app service

~20 min

Not running Starting usually ~20 s

Situation

The app web service is down: systemctl status app shows failed. The service must answer at http://localhost:8080/, where its clients expect it.

Task

The app service is active and answers http://localhost:8080/ with HTTP 200.
Whatever was in app's way is stopped and will not start again after a reboot.

Do not change the port of app: its clients expect it on 8080.

The tools you need are already installed. There's no internet on this machine.

Lab goals

0 of 3

Whatever was in app's way is stopped and will not start after a reboot — not checked yet
The app service is active — not checked yet
The app service answers http://localhost:8080/ with HTTP 200 — not checked yet

Hints

0 of 3 open · they open one at a time

Hint 1

Show the solution after 3 failed checks

Network · picked for this lab

ss — sockets and ports

Cheat sheet

$_ — insert into the terminal

ss -tlnp listening ports + who
ss -tnp active connections
ss -tulnp TCP+UDP listeners
ss -tlnp 'sport = :80' who's on port 80
ss -tan state close-wait a socket leak?
ss -s statistics

Mnemonic for the common combo -tlnp: tcp listening numeric process. -n matters on slow networks: without it ss tries to resolve every address into a name, and the output noticeably slows down. -p needs root to show other users' processes — without sudo you'll only see your own.

Commands

bash

ss -tlnp                          # listening TCP ports + processes
ss -ulnp                          # listening UDP ports
ss -tnp                           # established TCP connections
ss -tanp                          # all TCP: both listening and established
ss -s                             # summary: total sockets by type
ss -tnp state established         # established only
ss -tnp state time-wait          # "stuck" in TIME-WAIT
ss -tlnp '( sport = :80 or sport = :443 )'   # who's listening on 80/443
ss -tnp dst 10.0.0.5             # connections to a specific host
ss -tnp 'dport = :5432'          # connections to the postgres port
ss -tnp dst 10.0.0.0/24          # to a whole subnet
ss -tonp                          # with keepalive/retransmit timers
ss -tnp | wc -l                   # roughly how many active TCP connections

States and filters

Filter by TCP state — state <name> (or state connected/listening):

bash

ss -tan state established         # working connections
ss -tan state syn-sent            # client waiting for a reply (a stuck connect?)
ss -tan state close-wait          # app isn't closing sockets (fd leak!)
ss -tan state time-wait | wc -l   # lots of TIME-WAIT — frequent short connections

Address filters combine with and/or: src/dst (address), sport/dport (port).

bash

ss -tnp 'dst 10.0.0.5 and dport = :443'
ss -tnp 'sport >= :1024'         # ephemeral outgoing ports

"Address already in use" when starting a service

The port is already taken. Find who and stop/change the port:

bash

ss -tlnp 'sport = :8000'         # users:(("python",pid=1234,fd=3))

The service is running but unreachable from outside

Is it listening at all: ss -tlnp | grep :PORT.
Bound to 127.0.0.1 instead of 0.0.0.0/*? Then it's local only — fix the bind address in the config.
Is the port open in the firewall? (nft list ruleset / ufw status — see firewall).

The number of CLOSE-WAIT sockets keeps growing

The app isn't closing connections (a descriptor leak) — sooner or later you'll hit ulimit. ss -tanp state close-wait shows the culprit process.

Lots of TIME-WAIT

Normal with many short-lived client connections; usually not a problem. Fixed by keep-alive/connection pooling on the application side, not by randomly tuning the kernel.

Not sure which process holds a connection

ss -tnp shows users:(("nginx",pid=1234,...)). For more depth, lsof.

Ports: start the app service

Situation

Task

Signing in is needed to start the machine

Commands

States and filters

"Address already in use" when starting a service

The service is running but unreachable from outside

The number of CLOSE-WAIT sockets keeps growing

Lots of TIME-WAIT

Not sure which process holds a connection