Monorepo CI: affected-only builds and remote caching

Affected: from a git diff to a subset of the task graph

By the end of this lesson you’ll know exactly why the 40-minute build keeps happening and the three levers that eliminate it — or at least shrink it to match the diff, not the repo.

“Build and test everything on every PR” is correct and, in a large monorepo, ruinous. Its cost grows with the size of the repository while the value of any single PR depends only on what that PR changed. Affected-only builds break that coupling. The tooling — Nx, Turborepo, Bazel — models the repo not as a flat list of packages but as a project graph: nodes are projects, edges are import/dependency relationships. On top of it sits a task graph: build, test, and lint for each project, wired so a project’s build depends on its dependencies’ build.

Computing the affected set is two steps. First, the tool asks git which files changed between two commits — a --base and a --head — and maps each touched file back to the project that owns it. That gives the directly changed projects. Second, it walks the project graph in reverse: any project that transitively depends on a changed project is also affected, because a change upstream can break it. The union is the affected set, and the tool runs the requested tasks for exactly that subset.

# In CI, base = last successful main SHA, head = the PR tip.
# nrwl/nx-set-shas exports these so you don't diff against a stale point.
nx affected -t build,test,lint --base=origin/main --head=HEAD

# Turborepo expresses the same idea via the package graph + git:
turbo run build test lint --filter='...[origin/main]'

The base matters more than it looks. Set it to the last successful commit on main, not a fixed branch point — otherwise a long-lived PR diffs against an old base, recomputes a huge affected set, and you lose the win. In CI you wire this with a helper like nrwl/nx-set-shas.

Remote cache: hash the inputs, restore the outputs

Affected tells you what to run. The cache tells you whether you even need to. Before executing a task, the tool computes a hash over everything that can influence the output: the project’s source files, the resolved hashes of its dependencies, the task’s configuration, the tool versions, and any declared environment variables. That hash is the cache key. If a task with that exact key was run before — by anyone, on any machine — its recorded output (the dist/, the test result, the logs) is downloaded and replayed instead of recomputed.

The leverage is that the cache is shared and content-addressed. Turborepo splits this into a global hash and a per-task task hash; if either changes, the task misses. A teammate who already built main locally, or a previous CI run, populates a remote cache (Nx Cloud, Turborepo Remote Cache) that your run reads from. A cache hit that turns a multi-minute build into a sub-second artifact restore is the difference between a CI run that scales with the diff and one that scales with the repo.

Sharding: split what’s left across runners

Affected and caching shrink which tasks run. Sharding shrinks how long the survivors take by running them in parallel across N runners. The classic form is a CI matrix: split a 2,000-case test suite into shards 1/4 … 4/4, give each its own runner, and a 40-minute suite finishes in ~10 (four-way ≈ 75% faster); a small suite can drop from 3 minutes to 30 seconds. Nx Cloud and Turborepo go further and distribute the task graph itself across agents, respecting dependency order so a downstream build waits for its upstream.

# GitHub Actions matrix: 4 parallel test shards.
strategy:
  matrix:
    shard: [1, 2, 3, 4]
steps:
  - run: npx jest --shard=${{ matrix.shard }}/4   # or: nx affected -t test --parallel

One rule separates a fast matrix from a slow one: balance by execution time, not test count. Four shards of equal file count but lopsided runtime leave three runners idle while one grinds — the slowest shard sets your wall-clock, so the win is only as good as your worst shard.

The honest tradeoff: affected + remote cache is the highest-leverage move, but its entire value rests on the hash being complete and the affected computation being tight. Miss an input and you ship stale; let a root file balloon the affected set and you’ve paid the setup cost for nothing.

The failure modes that erase the win

Three failures recur, and each turns a scaling win into a liability. When you see any of them in your repo, the fix is always the same: tighten the input set, not the tooling.

The root-file blast radius. Change tsconfig.base.json, nx.json, or — the classic — the lockfile, and the tool marks everything affected. The lockfile case is a deliberate failsafe: a dependency bump could affect any project, so by default Nx plays it safe and rebuilds all. The result is that a routine package-lock.json update detonates the full 40-minute build, and a team that doesn’t understand why concludes “affected doesn’t work.” It works; you tripped the global-input wire. You tune it with namedInputs so only genuinely global files are global — but mis-scope a {workspaceRoot} glob and you re-create the blast radius yourself.

The stale hit from an unhashed input. Covered above: an env var, a tool version, or a config file that influences the output but isn’t in the hash. The cache returns a build that is wrong for the current inputs, and because a hit is silent and fast, it can pass review and reach production. This is the most dangerous failure because it doesn’t fail — it succeeds with the wrong answer.

Remote-cache poisoning / trust. A remote cache is a shared write surface. If an untrusted actor (a fork PR, a compromised token) can write artifacts, they can plant a malicious dist/ under a key a trusted build will later read, and your “cache hit” runs their code. This is why Turborepo signs artifacts with HMAC-SHA256 and verifies them on download, treating any artifact that fails the signature as a miss — and why you scope cache write access tightly and never let untrusted PRs populate the cache trusted runs consume.