Process lifecycle

Every new process is born from fork() — a clone of the parent.

fork() is a system call that creates an exact copy of the calling process. The child gets:

• A new PID
• A copy-on-write copy of the parent’s memory
• Copies of all open file descriptors
• The same program counter (both parent and child resume from after fork())

fork() returns twice: it returns the child’s PID in the parent, and 0 in the child. This is how both processes know which role they play.

# Observe fork in action: your shell forks for every command
# Before the fork: one bash process
echo $$    # print the current shell's PID, e.g. 1234

# The shell forks, then the child execs 'ls'
ls /tmp

# After ls exits, control returns to the parent (bash)
# The child process no longer exists

The key insight: the child is not a blank slate — it is a clone. That clone-first, replace-second model is why spawning a child is cheap: only pages that are actually written get copied (copy-on-write).

exec() replaces the child’s program image with a new one.

After fork(), the child typically calls one of the exec() family of syscalls (execve, execvp, etc.). This replaces the child’s memory, code, and stack with a new program. File descriptors are inherited unless marked close-on-exec.

# strace lets you see the fork+exec sequence for any command
strace -e trace=execve ls /tmp 2>&1 | head -5
# execve("/usr/bin/ls", ["ls", "/tmp"], 0x... /* env */) = 0
# The kernel replaces the process image with /usr/bin/ls
# The return value 0 means success

# You can also see it for a shell command:
strace -e trace=clone,execve bash -c 'echo hello' 2>&1 | grep -E 'clone|execve'
# clone(...) = <child-pid>       -- this is the fork()
# execve("/usr/bin/bash", [...]) -- child execs bash with -c echo hello

The shell does this thousands of times a day. Every pipeline stage, every subshell, every $(command) substitution — each is a fork followed by an exec.

Process states: what the STAT column in ps really means.

Every process is in one of these kernel scheduler states at any moment:

ps aux
# USER       PID %CPU %MEM    VSZ   RSS TTY      STAT ...
# root         1  0.0  0.2  22596  9320 ?        Ss   ...  ← systemd: sleeping, session leader
# www-data   987  0.1  1.1 123456 44000 ?        S    ...  ← nginx worker: interruptible sleep
# root       123  0.0  0.0      0     0 ?        D    ...  ← in uninterruptible I/O wait
# root      3456  0.0  0.0      0     0 ?        Z    ...  ← zombie: exited, not yet reaped

# STAT letter meanings:
# R  Running or runnable (on the run queue or actually on a CPU)
# S  Interruptible sleep (waiting for an event; can be woken by a signal)
# D  Uninterruptible sleep (waiting for I/O; CANNOT be killed — not even by SIGKILL)
# Z  Zombie (exited but parent has not called wait() yet)
# T  Stopped (SIGSTOP, Ctrl-Z, or being traced by a debugger)
# +  Foreground process group (shown as S+ or R+)
# s  Session leader
# l  Multi-threaded

The D state is the one that surprises operators: a process stuck in D state does not respond to kill -9. It is inside a kernel operation (usually an I/O syscall) and the kernel will not interrupt it. If a process is stuck in D for minutes, the problem is the underlying I/O (hung NFS mount, dying disk) not the process itself.

wait(): the parent must reap the child, or it becomes a zombie.

When a process exits, the kernel keeps a small record (the exit status, the PID, resource usage). This record stays until the parent calls wait() or waitpid() to collect it. A child that has exited but not been reaped is a zombie (state Z).

# Zombies show up in ps as:
# 1234 Z+  0:00 [defunct]

# How many zombies do you have right now?
ps aux | awk '$8 ~ /Z/ { print $0 }'

# Zombies are NOT dangerous by themselves:
# They consume a PID slot and a small kernel struct — nothing else.
# No CPU, no memory, no file descriptors.

# Zombies DO matter if there are thousands of them:
# PIDs are a finite resource (cat /proc/sys/kernel/pid_max — usually 4194304)
# A parent with a bug that never calls wait() accumulates zombies over time.
# The fix is to fix the parent (or restart it), not to kill the zombies.

# Finding the parent of a zombie:
# Get the zombie's PID (e.g. 1234), then:
ps -o ppid= -p 1234    # prints the parent PID
# That parent is the one that needs to call wait()

Orphans and PID namespaces: what happens when a parent dies first.

If a parent exits while its children are still running, those children become orphans. The kernel immediately reparents orphans to PID 1 (systemd). PID 1 continuously calls wait() in a loop, so orphan zombies are immediately reaped.

# Demonstrate orphan adoption:
# In one terminal, start a process that spawns a child and exits:
bash -c 'sleep 300 & echo "child PID: $!"' &
# Now kill the parent bash immediately
# The sleep 300 continues running — check its parent:
sleep 1
pstree -p | grep sleep
# The sleep is now a child of PID 1 (systemd), not of the original bash

# PID namespace preview (container bridge):
# A container is (among other things) a new PID namespace.
# Inside the namespace, processes see PIDs starting from 1.
# The first process in that namespace becomes its PID 1.
# If that process exits without reaping its children, zombies accumulate.
# This is why container PID 1 must handle signals and call wait() — or use a tiny init.
unshare --pid --fork --mount-proc bash
# Inside: ps shows PID 1 = bash, PID 2 = ps
# Exit: all processes in the namespace are killed