On-Stack Replacement: swapping the frame mid-loop

The problem normal tier-up can’t solve

Recall how tier-up installs optimised code (lesson 01): when a function’s budget trips, V8 compiles a faster version and swaps it in for the next entry to the function. That model has a blind spot. Consider a function that is called once and spends all its time in a single long loop:

function crunch(rows) {
  let acc = 0;
  for (let i = 0; i < rows.length; i++) {   // 10,000,000 iterations
    acc += score(rows[i]);
  }
  return acc;
}
crunch(tenMillionRows); // called exactly once

The loop is blazingly hot — millions of back-edges — so its back-edge counter trips the budget almost immediately (lesson 01 introduced this counter precisely for this case). V8 wants to optimise. But the normal mechanism is useless here: it would install optimised code for the next call to crunch, and crunch is mid-flight on the only call it will ever get. By the time crunch returns, the work is done. Waiting for re-entry means never optimising the one loop that matters.

OSR: replacing the frame in flight

On-Stack Replacement is the mechanism that breaks this deadlock. Instead of waiting for the function to return and be re-called, V8 replaces the executing frame in place, mid-loop. The sequence:

1. The loop’s back-edge counter trips the tier-up budget while the loop is running in the interpreter (or baseline tier).
2. V8 compiles a special OSR-entry version of the function — optimised machine code whose entry point is not the function’s top, but the loop header. It is specialised to begin executing as if control had just arrived at the start of a loop iteration, with all the loop’s live variables already in place.
3. V8 performs a live-state transfer: it reads the current values of every live variable from the interpreter frame (the loop index i, the accumulator acc, the rows reference) and writes them into the optimised frame’s registers and stack slots, in the representations the optimised code expects. This is the inverse of the deopt frame translation from lesson 05 — there we rebuilt an interpreter frame from an optimised one; here we build an optimised frame from an interpreter one.
4. Control transfers into the OSR-entry optimised code at the loop header, and the same loop continues — but now in optimised machine code. The remaining nine-and-a-half million iterations run fast.
5. After the loop finishes and crunch eventually returns, normal optimised code (a regular, top-entry compilation) takes over for any subsequent calls.

Why micro-benchmarks live or die by OSR

This is the mechanism behind a notorious benchmarking pitfall. If your benchmark fires up once and runs a single giant loop, ask yourself: has your function ever been called enough times to get a normal tier-up? The answer is almost certainly no — you are depending entirely on OSR. A micro-benchmark that wraps its work in a single huge loop — for (let i = 0; i < 1e9; i++) work() — does not warm up work through invocation-count tier-up in the usual way; it relies on OSR kicking in partway through to optimise the loop body live. That has two consequences for anyone measuring performance:

• The first chunk of iterations runs unoptimised (interpreter/baseline), then there is a discontinuity where OSR fires and the rest run fast. If you time the whole loop as one number, you blend cold and hot execution and get a meaningless average.
• OSR-entry code can be slightly worse than a normal top-entry optimisation, because it must accept the loop’s live state as it found it rather than from a clean function entry, which constrains some optimisations. So an OSR-warmed measurement can under-report the steady-state speed you would see if the function were called many times normally.

The fix is the standard one: warm up the function with many separate calls before timing (so it gets a normal, non-OSR optimisation), and measure steady-state iterations, not the warm-up transient. You can see OSR happen with --trace-osr; combined with --trace-opt it shows the OSR-entry compile distinct from the regular one.