Coverage, mutation testing, and what's worth testing

How coverage is collected, and why line ≠ branch

Before you trust that green percentage on your CI dashboard, you need to understand exactly what it measures — because the number that looks like assurance is hiding a gap that’s large enough to ship a one-character prod bug through.

Node’s built-in coverage (node --test --experimental-test-coverage, or c8 wrapping any runner) uses V8’s native coverage: the engine already tracks which byte ranges of compiled code execute, so the runtime hands back precise execution counts with near-zero overhead — no source rewriting, unlike the old Istanbul/babel-instrument approach. The report rolls that up into four metrics, and the gap between them is where bugs hide. Line/statement coverage asks “did this line run?” Function coverage asks “was this function called at least once?” Branch coverage asks the only question that matters: “for each decision point — every if, &&, ||, ternary, ?. — did tests exercise both outcomes?”

The trap is that 100% line coverage can sit on top of a completely untested branch. Consider a short-circuit:

export function fee(order) {
  const base = order.amount * 0.029;
  // one line, two branches: the `||` short-circuits
  return order.isPremium || base < 0.30 ? 0.30 : base;
}

// the only test
test("charges the percentage fee", () => {
  assert.equal(fee({ amount: 100, isPremium: false }), 2.9);
});

One test, and the line-coverage report says 100% — every line ran. But branch coverage tells the truth: the isPremium === true path never executed, and the base < 0.30 minimum-fee floor never executed. Two real behaviors — the premium waiver and the small-order floor — are completely untested, and a line-coverage gate would wave them through. This is why a senior reads the branch column first: a falsy short-circuit, a default parameter, a catch block, an early return guard — each is a branch that line coverage silently absorbs into a “covered” line.

The coverage trap: execution is not verification

Here is the deeper, more dangerous version of the same lie, and it is the heart of why coverage dashboards mislead. Coverage instruments execution. It cannot see assertions. A test that runs code without checking anything counts as 100% covered:

test("applyDiscount works", () => {
  const order = { total: 100, tier: "gold" };
  service.applyDiscount(order); // executes every line of applyDiscount...
  // ...and asserts nothing. Coverage: 100%. Bugs caught: 0.
});

This is not hypothetical — it is the most common way real suites hit a high number and verify nothing. The pattern shows up as: tests with no expect/assert at all; snapshot-only tests where the snapshot was generated from already-buggy output and just locks the bug in; tests that mock the very call they pretend to verify (vi.spyOn(svc, "charge").mockResolvedValue(true) and then assert the mock returned true — you tested the mock, not the code). I have seen a billing module sit at 91% line coverage with effectively zero meaningful assertions — every test executed the path and none inspected the result. The TRADEOFF is real and cuts both ways: a coverage gate is cheap and does catch genuinely untested files (a new module with 0% lights up instantly), but it is trivially gamed by assertion-free tests, and the marginal climb from 90% to 100% is usually the least valuable work in the codebase — error branches that can’t fire, defensive default: cases, generated code — so a hard 100% mandate spends your most expensive engineering hours fighting the report instead of finding bugs.

Mutation testing: does your suite actually verify anything?

Mutation testing answers the question coverage structurally cannot. A mutation tool — Stryker for JS/TS — takes your source and injects tiny, behavior-changing faults called mutants, one at a time, then reruns the test suite against each. The mutations are surgical and mimic real bugs: flip < to <=, swap + for -, replace a boolean literal with its opposite, delete a statement, change a return x to return null, turn a conditional’s body into an empty block. For each mutant there are two outcomes. A killed mutant means at least one test failed when the code changed — good, a test was watching that behavior. A surviving mutant means every test still passed despite the injected bug — which is a proof, not a guess, that no assertion constrains that behavior. The mutation score = killed ÷ (total non-equivalent mutants), and it measures suite quality far better than coverage because it is defined in terms of caught bugs, not executed lines.

// the function under test
export function isAdult(age) {
  return age >= 18;   // Stryker mutates >= to >, and to <=, and to true/false
}

// the only test
test("21 is an adult", () => {
  assert.equal(isAdult(21), true);
});

Coverage here is a clean 100% — the line ran. Run Stryker and the report is brutal: the >= → > mutant survives, because isAdult(21) is true under both >= and >; the test never probes the boundary at 18 where they differ. That surviving mutant is the exact class of the one-character prod bug from the hook. A whole module can post 100% line coverage and a 60% mutation score — 40% of injected bugs sail through untouched, and every one of those survivors is a behavior your suite would not have caught breaking. The TRADEOFF is cost: mutation testing reruns the (relevant subset of the) suite once per mutant, so a module with a few hundred mutants and a fast suite is seconds-to-minutes, but a large codebase is minutes-to-hours. So you don’t run it on every commit — you point it at the critical modules (billing, auth, pricing), run it in nightly CI or a pre-release gate, and use Stryker’s incremental mode and per-test coverage to mutate only what changed.

What’s actually worth testing

The fix is not “test more,” it’s “test the right seams and verify them.” The testing trophy (and the older pyramid) both point the same way: bias your effort toward integration tests that exercise real seams — a request hitting a real router, service, and an in-memory or test database — over a swarm of brittle unit tests of trivial code. Don’t test framework code, getters, or pass-through wrappers; testing getName() { return this.name } adds a covered line and zero confidence, and it freezes your implementation so every refactor breaks tests that assert nothing about behavior. Test behavior and the branches and boundaries that matter: the off-by-one at the threshold, the empty-list case, the timeout path, the malformed input — exactly the places mutants survive. The FAILURE MODE to avoid is the 100%-coverage mandate itself: chasing the number produces brittle, assertion-light, implementation-coupled tests that slow every refactor (you spend the afternoon fixing tests that never verified anything) while still missing the bug class that matters, because line coverage was never measuring verification. Tie your team’s quality target to mutation score on the modules that matter, treat coverage as a cheap floor that flags wholly-untested code, and you optimize for caught bugs instead of executed lines.