a service — use systemd; a long interactive task — use tmux
Overview
The shell can run tasks in the background and manage them. But "background" in
bash does not mean "survives logging out of the session" — for that you need
nohup, disown, or, better, tmux/systemd.
Recipes
3
Shell commands
bash
long-cmd & # run in the background
jobs # list the shell's background jobs
jobs -l # + PID
fg %1 # bring job 1 to the foreground
bg %1 # resume a stopped job in the background
Ctrl+Z # suspend the current one (→ background with bg)
kill %1 # kill a shell job (by job spec)
wait # wait for all background jobs to finish
wait %1 # wait for a specific one
Surviving logout
bash
nohup long-cmd > out.log 2>&1 & # ignore SIGHUP, log to a file
disown -h %1 # detach an already-running job from the shell
setsid long-cmd >out.log 2>&1 # run in a new session (fully detached)
systemd-run --user --unit=job long-cmd # a one-off transient unit (logs in the journal)
For services, use a systemd unit: it gives you autostart, restart
on failure, and logs in the journal — nohup does none of that. For long
interactive tasks, use tmux.
Running in parallel
bash
cmd1 & cmd2 & cmd3 & wait # run in parallel and wait for all of them
for h in h1 h2 h3; do ssh "$h" uptime & done; wait # poll hosts in parallel
cmd1 && cmd2 # cmd2 only if cmd1 succeeded (NOT background)
cmd1 || cmd2 # cmd2 only if cmd1 failed
( cd /tmp && long ) & # a background subshell with its own directory
For serious parallelism, use xargs -P N (see xargs) or GNU parallel.
Troubleshooting
4
SymptomCause and fix
The process died after closing SSH
Launched with &, it is tied to the session and gets SIGHUP on logout. Run
it with nohup ... &, disown, tmux, or turn it into a systemd
service.
A background job's output clutters the terminal
Redirect stdout and stderr: cmd > out.log 2>&1 &.
Ctrl+Z "froze" a process
It wasn't killed — the process is suspended (stopped, state T). fg
resumes it in the foreground, bg resumes it in the background, kill %1
terminates it (see processes).
The script finished, but background jobs are still running/orphaned
Add wait at the end, or trap 'kill 0' EXIT to kill the process group on exit.
The mode in which the shell waits for a command to finish: the terminal belongs to it, and Ctrl+C and Ctrl+Z act on it. fg brings a job back here.Also appears in:
Dockerfile, sleep, Docker, Processes and 1 more
The mode in which a command keeps running while the shell prints the prompt at once: start it with & at the end or resume it with bg.Also appears in:
wait, sleep, kill, tmux and 9 more
See also
processes — Viewing, signaling, and prioritizing processes; finding CPU/memory hogs.
tmux — Sessions that survive an SSH disconnect; windows and panes.