Projections, replay, and snapshots

Projections: read models derived from the event log

The event log is the source of truth for the write side. It is not designed for read access patterns. Answering “show all overdue orders for account manager Chen” by scanning and folding a 200,000-entry event log on every page load is structurally wrong.

A projection is a process that reads events from the log and builds a denormalized read model shaped for a specific query. It is the same concept as in CQRS (unit 07) — the key difference is that the source being projected is the event log rather than a normalized relational table.

Each projection subscribes to the event stream and handles each event type it cares about:

on OrderPlaced:    insert row into OrderDashboard with status "pending"
on OrderApproved:  update row, set status "approved", approved_by, approved_at
on InvoiceGenerated: update row, set status "invoiced", invoice_id

The result is an OrderDashboard table with one row per order, denormalized and shaped for the dashboard query. Queries hit the read model directly — no event folding at query time.

Rebuild by replay: the structural superpower

Because the event log is the source of truth and the read model is a derived artifact, the read model can be rebuilt from scratch at any time by replaying the full event log through the projection function. This is structurally different from a CRUD system where the read model and the write model are the same data — there is nothing to “replay.”

Rebuild-by-replay is valuable in multiple scenarios:

Schema change: The dashboard needs a new column days_to_first_payment. Add the column to the read model table, update the projection logic to compute it, then replay all events. Every historical row gets the correct value computed from the original event data. No write-side migration.

Bug fix: The projection had a bug — it was computing overdue status incorrectly, marking orders as overdue two days too early. Fix the projection logic, replay the event log, and the read model is now correct for all historical data.

New read model: The finance team needs a new view: all orders grouped by billing period. Create a new projection that builds a BillingPeriodSummary table. Replay the existing event log to build it from history. No waiting for future events to accumulate.

Snapshots: bounding the replay cost

Loading an aggregate by folding its full event log becomes expensive as the log grows. An order with 4,000 events requires 4,000 applyEvent calls before a command can even begin. This is the fold-performance problem from lesson 1.

A snapshot is a cached intermediate fold result. At some sequence number N, the application persists the aggregate’s derived state as a snapshot alongside the event log:

{
  aggregate_id: "order-48291",
  snapshot_seq: 3800,
  state: { status: "invoiced", total: 48200, line_items: [...], ... },
  created_at: "2024-06-01T12:00"
}

On the next load, instead of starting from event 1 and applying all 4,000 events, the application:

1. Loads the latest snapshot (state at seq 3800)
2. Loads only events after seq 3800 (events 3801–4000)
3. Folds only those 200 events onto the snapshot state

The fold cost drops from 4,000 to 200. As new events are appended, a new snapshot is taken periodically.

The critical invariant: a snapshot is not the source of truth. The event log is. A snapshot can be discarded at any time — the aggregate state can always be reconstructed by folding the full event log from event 1. Snapshots exist purely for fold performance; they do not replace events.

Temporal queries: replaying a prefix of the log

Lesson 1 introduced the temporal query: to get the state of order #48291 on March 3rd at 2 PM, filter the event log to events before that timestamp and fold the prefix. This is a structural consequence of the immutable, append-only log.

Temporal queries have a performance implication: they cannot use a snapshot taken after the target timestamp (that snapshot includes events you want to exclude). They must fold from the beginning of the log to the target timestamp. For aggregates with very long histories, temporal queries over old timestamps are expensive. This is a known structural cost of event sourcing — it is manageable for occasional compliance queries, but should not be on the hot path of real-time queries.

Event versioning and upcasting

The event log is immutable — events already stored cannot be changed. But the application’s understanding of what an event means evolves. Six months after the system launched, OrderApproved gains a new field approval_threshold. The 40,000 existing OrderApproved events in the log predate this field. When the projection replays them, it will encounter events missing the new field.

The two structural responses:

Weak schema / tolerant reader: The projection handles OrderApproved with a tolerance policy: if approval_threshold is absent, treat it as null or a default value. The projection code handles both old and new versions of the event. This works for additive changes (new optional fields) and is the simplest approach.

Upcasting: On load (or at read time), old event versions are transformed to the current format before reaching the aggregate’s applyEvent function. An upcast function for v1 → v2 OrderApproved might set approval_threshold: null for all v1 events. The aggregate’s fold logic sees only the current format. The event log retains the original v1 bytes — upcasting is applied at read time, not written back to the log.