The OOM killer

Why the OOM killer exists: memory overcommit.

Linux allocates memory optimistically. When a process calls malloc() or mmap(), the kernel grants virtual address space but does not back it with physical RAM until the memory is actually written (demand paging). This is overcommit: the sum of all processes’ virtual memory exceeds physical RAM plus swap. It works because most allocations are never fully used (think: a process that allocates 1 GB but only touches 100 MB).

# Check the overcommit policy:
cat /proc/sys/vm/overcommit_memory
# 0 = heuristic (default): allow some overcommit, refuse obviously impossible requests
# 1 = always overcommit: never refuse malloc, no matter how large
# 2 = never overcommit: total committed memory <= RAM + swap * ratio

cat /proc/sys/vm/overcommit_ratio
# 50  (used only when overcommit_memory=2: commit limit = RAM + 50% of swap)

# Check how much is currently committed:
grep -E 'MemTotal|MemAvailable|SwapTotal|Committed_AS|CommitLimit' /proc/meminfo
# MemTotal:       16384000 kB
# MemAvailable:    2048000 kB    ← actual free+reclaimable
# CommitLimit:    20971520 kB    ← maximum committal (RAM + swap)
# Committed_AS:   18500000 kB    ← how much has been promised to processes

# When Committed_AS approaches CommitLimit and pages are actually touched,
# the kernel runs out of physical pages. It must either:
# a) Get pages from swap (if swap is available and not also exhausted)
# b) Invoke the OOM killer

oom_score: how the kernel picks the victim.

Every process has an oom_score between 0 and 1000. The kernel kills the process with the highest score. The score is computed from:

• RSS (resident set size): how much physical RAM the process is using — the primary factor
• swap usage
• run time: longer-running processes get a slight discount (they are assumed more important)
• oom_score_adj: a bias added to the raw score, set by the user or systemd

# Read any process's current oom_score:
cat /proc/1234/oom_score
# 742   ← high: this process is a likely OOM victim

# Read the oom_score_adj (the bias):
cat /proc/1234/oom_score_adj
# 0     ← no bias applied (range: -1000 to +1000)

# oom_score_adj = -1000: NEVER kill this process (kernel skips it entirely)
# oom_score_adj = -500:  half as likely to be chosen
# oom_score_adj = 0:     neutral
# oom_score_adj = +500:  half again more likely
# oom_score_adj = +1000: kill this first (used for memory-hungry batch jobs)

# Protect a critical process (e.g., sshd — you need it for remote access):
echo -1000 | sudo tee /proc/$(pgrep sshd)/oom_score_adj

# Set high score for a disposable worker:
echo 500 | sudo tee /proc/$(pgrep batch-worker)/oom_score_adj

# Persistent: set in systemd unit file:
# [Service]
# OOMScoreAdjust=-900   (for critical services like databases)
# OOMScoreAdjust=200    (for batch jobs that should die first)

# Scan all processes by oom_score to understand current exposure:
for pid in /proc/[0-9]*/oom_score; do
  score=$(cat "$pid" 2>/dev/null)
  comm=$(cat "${pid%oom_score}comm" 2>/dev/null)
  echo "$score $comm"
done | sort -rn | head -10

Reading the OOM kill event in dmesg and journalctl.

The kernel logs a detailed kill report. Knowing how to read it is the first step in any OOM postmortem.

# Check dmesg for OOM events (most direct):
sudo dmesg | grep -A 20 'Out of memory'
# [123456.789] Out of memory: Killed process 8823 (java) total-vm:4194304kB,
#              anon-rss:3145728kB, file-rss:65536kB, shmem-rss:0kB,
#              UID:1000 pgtables:12288kB oom_score_adj:0
# [123456.790] oom_reaper: reaped process 8823 (java), now anon-rss:0kB,
#              file-rss:0kB, shmem-rss:0kB

# Decode the fields:
# total-vm:4194304kB  = 4 GB virtual address space (allocated but not all in RAM)
# anon-rss:3145728kB  = 3 GB of anonymous (heap/stack) pages in physical RAM
# file-rss:65536kB    = 64 MB of file-backed pages (mmap'd files, shared libs)
# oom_score_adj:0     = no bias was applied

# Also in journalctl (journald captures kernel messages):
sudo journalctl -k | grep -i 'oom\|killed process\|out of memory' | tail -20

# journald also records the systemd unit that was killed:
sudo journalctl -u your-service --since '1 hour ago' | grep -i 'kill\|oom'
# If systemd-oomd or the kernel OOM killer hit the service, you see:
# "Memory cgroup out of memory: Killed process ..."  (cgroup OOM)
# or the kernel log line above (system OOM)

# See the full memory snapshot at time of OOM kill:
sudo dmesg | grep -B 5 -A 50 'Out of memory' | head -80
# The kernel dumps a table of all processes with their RSS and oom_score
# before killing the victim — invaluable for understanding what consumed memory

Cgroup OOM vs system OOM: MemoryMax fires first.

When a systemd service has MemoryMax set, it gets a cgroup-local OOM kill the moment the cgroup exceeds that limit — even if the host has gigabytes of free memory. This is intentional: the cgroup limit contains the blast radius.

# Distinguish the two in logs:
sudo journalctl -k | grep 'oom'
# Kernel OOM (system-wide):
#   "Out of memory: Killed process 8823 (java) score 742..."
#   Fires when the entire system is out of memory.
#
# Cgroup OOM (MemoryMax exceeded):
#   "Memory cgroup out of memory: Killed process 8823 (java)..."
#   OR (on newer kernels):
#   "oom-kill:constraint=CONSTRAINT_MEMCG,..."
#   Fires when a single cgroup exceeds its memory.max.

# Check if a service has a MemoryMax set:
systemctl show your-service | grep -E 'MemoryMax|MemoryHigh|MemoryCurrent'
# MemoryMax=536870912    ← 512 MB hard cap
# MemoryHigh=infinity    ← no soft pressure point set
# MemoryCurrent=498073600  ← currently at 475 MB — close to the cap!

# The cgroup-local OOM kill appears in the service's journal:
sudo journalctl -u your-service | grep -i kill

# Tuning: if the service legitimately needs more memory, raise MemoryMax:
sudo systemctl set-property your-service.service MemoryMax=1G

# If the service has a memory leak (grows without bound), MemoryMax is a
# safety net — but the real fix is the leak.