From source text to AST

Two stages before anything runs

When the runtime hands V8 a string of JavaScript, two things happen before the first instruction executes, and they are distinct phases:

1. Scanning (lexing). The scanner reads the raw source — V8 stores it as UTF-16 internally — one character at a time and groups characters into tokens: the indivisible words of the language. const x = 42; becomes the token stream Keyword(const), Identifier(x), Punctuator(=), Number(42), Punctuator(;). Whitespace and comments are consumed and dropped. The scanner is the hot inner loop of parsing — V8 has rewritten it repeatedly for UTF-16 throughput because it touches every single byte.

2. Parsing. The parser pulls tokens from the scanner and assembles them into an Abstract Syntax Tree (AST) — a structured, nested representation of the program’s grammar. const x = 42; becomes a VariableDeclaration node holding a VariableDeclarator with an Identifier (x) and a NumericLiteral (42). The flat token stream becomes a tree that captures structure: which expression is nested in which statement, which block scopes which variable.

How the parser builds the tree: recursive descent + Pratt

V8’s parser is a hand-written recursive-descent parser: one function per grammar rule, each calling the others as the grammar nests. ParseStatement may call ParseExpression, which may call ParseBlock, which calls ParseStatement again — the call stack mirrors the nesting of your code. This is why a pathologically deeply-nested expression (thousands of nested parentheses) can throw a RangeError: Maximum call stack size exceeded at parse time, before execution.

Statements are straightforward to parse top-down, but expressions have precedence: a + b * c must bind b * c first. A naive recursive-descent parser handles this with one function per precedence level, which is verbose. V8 instead uses operator-precedence (Pratt) parsing for expressions: a single loop that consumes operators while their binding power is high enough, climbing precedence levels on demand. The result is the same correct tree, built with far less code and fewer function calls.

// a + b * c parses to this tree shape (the * binds tighter than +):
//        (+)
//       /   \
//      a    (*)
//          /   \
//         b     c

The AST is transient — it is NOT what runs

This is the single most important correction to the naive model. People imagine the engine “walking the AST” to execute the program, like a tree-walking interpreter from a compilers course. V8 does not do this. The AST’s only consumer is the bytecode generator: V8 walks the tree once, emits Ignition bytecode for it, and then the tree becomes unreachable and is reclaimed by the garbage collector. From that point on, the bytecode runs — the AST is gone. (The next lessons in this unit cover bytecode and the interpreter loop.)

So the AST is an intermediate representation: a convenient structured form that exists only to be lowered into something the interpreter can execute efficiently. Keeping it around would waste memory; tree-walking it would be far slower than running flat bytecode.

Parse cost is linear in source size

Why does this matter? Because every kilobyte you add to your bundle adds a fixed parse tax — even if the user’s machine cached the file. Scanning touches every byte; parsing visits roughly every token once. So parse time grows linearly with source size — double the bytes, roughly double the parse cost. This is why bundle size has a direct, mechanical effect on startup latency that is separate from download time: even a fully-cached bundle still has to be re-scanned and re-parsed on a cold start.

You can see the tree V8 actually builds: run d8 --print-ast script.js and it prints the AST node-by-node. It is the clearest way to confirm that, say, an arrow function and a function expression produce different node types, or that an await desugars the way the spec says.

Failure mode: a giant data literal parsed eagerly

Here is the production trap. A team inlines a large dataset directly into a JS module as an object or array literal — a multi-megabyte “JSON-in-JS”:

// data.js — 3 MB of this, inlined into the bundle
export const LOOKUP = { "0001": { name: "...", coords: [] }, /* 50k more */ };

Because it is JavaScript source, V8 must fully scan and parse all 3 MB as an expression on the cold path — building AST nodes for every key, value, and nested literal — before the module finishes evaluating. That is pure linear parse tax on the critical path, and it blocks first paint.

The fix is to move the data out of JS and into a real .json file, loaded as a string and handed to JSON.parse:

const LOOKUP = JSON.parse(await fetch("/data.json").then(r => r.text()));

JSON.parse has a dedicated, hand-tuned C++ fast path that is dramatically faster than the general JavaScript parser building AST nodes — it does not produce an AST, does not allocate scope objects, and parses a far simpler grammar. The same bytes cost a fraction of the time. The rule of thumb: data should travel as data, not as code. This is exactly what bundlers like webpack and esbuild do when they extract large constants, and what V8’s own team recommends for big payloads.