Dockerfile mechanics: which instructions become layers, how the cache keys each step, and why order matters

Two kinds of instruction: layers vs config

Walk a Dockerfile top to bottom and every instruction does one of two things. A few of them — RUN, COPY, ADD — execute against the filesystem and produce a new read-only layer: a tarball of exactly the files that changed, named by a content digest. Everything else — ENV, WORKDIR, EXPOSE, ENTRYPOINT, CMD, LABEL, USER, ARG — touches no filesystem at all. It writes a field into the image’s JSON config and records a zero-byte history entry. Run docker history and you can see the split directly:

$ docker history myapp:1 --format '{{.Size}}\t{{.CreatedBy}}'
0B        CMD ["node" "server.js"]
0B        EXPOSE 3000
0B        ENV NODE_ENV=production
0B        WORKDIR /app
12.4MB    COPY . . # source
108MB     RUN npm ci
1.1kB     COPY package.json package-lock.json ./
0B        WORKDIR /app
77.8MB    RUN apt-get install ...

The 0B rows are the config mutations: they change the image (and therefore the image ID) but add no layer bytes. The sized rows are the real filesystem layers. This is not trivia — it tells you exactly which lines are expensive to rebuild and which are free.

The cache key of one step

Now the part that decides your rebuild time. When BuildKit considers reusing a cached layer for a step, it computes a cache key and looks for a match. The key for a step is, roughly: the digest of the parent layer (the entire image state so far) plus the literal instruction text — and, for COPY and ADD only, a checksum of the actual file contents being copied. So RUN npm ci is keyed purely on the text RUN npm ci and whatever state sits beneath it; change neither and it hits the cache. But COPY . . is keyed on a checksum of every copied file, so changing one byte of one source file changes its key and forces a miss.

# every step's key includes everything above it
FROM node:20-slim          # parent base
WORKDIR /app               # config only, free
COPY package.json package-lock.json ./   # key = checksum(these 2 files)
RUN npm ci                 # key = text "RUN npm ci" + state above
COPY . .                   # key = checksum(whole source tree)
CMD ["node","server.js"]   # config only, free

The consequence is a one-way cascade: the first step whose key changes invalidates every step below it. Cache is a prefix match from the top. If COPY package.json still matches but COPY . . does not, only COPY . . and what follows rebuild — npm ci above it stays cached. Flip the order so COPY . . comes first and the cascade reaches npm ci on every code edit. That is the entire war-story in one rule.

RUN-chaining and the rm-in-a-later-layer trap

Because each RUN is its own layer, a file you create in one RUN and delete in a later RUN does not shrink the image. Layers are stacked and immutable: the lower layer still carries the bytes, and the delete only writes a whiteout in the higher layer that hides the file at runtime. The bytes are still pulled, still stored, still extractable.

# WRONG: two layers — clean happens too late to shrink anything
RUN apt-get update && apt-get install -y build-essential
RUN apt-get clean && rm -rf /var/lib/apt/lists/*   # whiteout only

# RIGHT: one layer — the index and caches never persist into the layer
RUN apt-get update \
 && apt-get install -y --no-install-recommends build-essential \
 && rm -rf /var/lib/apt/lists/*

Chaining the install and the cleanup into one RUN means the apt package index (often 40–60 MB) and downloaded .deb caches are removed within the same layer, before its tarball is finalized, so they never land in the image at all. On a typical Debian build this single change saves roughly 40 to 100 MB, and --no-install-recommends trims another tranche of optional packages. The rule generalizes: anything you create only to delete must be created and deleted inside one RUN.

Busting the cache when you mean to

Sometimes the text-keyed cache is too sticky and you need a deliberate miss — for example to force a fresh git clone or a re-fetch. The clean lever is an ARG whose value you pass at build time: referencing it inside a RUN folds its value into that step’s cache key, so changing the arg busts from that point down. A content checksum (passing a known file hash as a build arg) does the same precisely. The blunt instrument is docker build --no-cache, which skips the cache for every step. And to share cache across machines or a fresh CI runner, docker build --cache-from seeds the local cache from a pushed image so a cold runner still hits warm layers. One more thing worth a single line: BuildKit is the default builder now, and it parallelizes independent stages of a multi-stage build rather than running them strictly top to bottom — the deep dive on multi-stage builds is the next unit.