The traps
Event sourcing's immutability and eventual consistency are features on paper and constraints in production — schema evolution, GDPR erasure, bad events, and projection lag all require deliberate structural answers. Knowing when not to event-source is as important as knowing how.
Eighteen months into the event sourced billing platform, three distinct fires were burning simultaneously. First: a customer had submitted a GDPR erasure request. Legal said the customer’s name, email, and tax ID had to be purged within 30 days. The legal team had assumed the engineers could just delete the relevant rows. The engineering team explained that the customer’s personal data was embedded in hundreds of events — OrderPlaced, InvoiceGenerated, PaymentReceived — spread across the immutable event log. Deleting them would corrupt the log. Second: a billing engineer had accidentally appended an OrderApproved event for the wrong order ID — a fat-finger error during a manual support operation. “Just delete it,” said the support manager. The engineering team explained again: events in the log are immutable. Deleting it would corrupt replay for every read model. Third: the finance projection had been showing stale totals for about two seconds after each order update — normally fine — but during a high-traffic period the projection pipeline had backed up and the lag had stretched to forty seconds. The CFO saw a dashboard showing incorrect overdue amounts and called the CEO. Three fires, one root cause: the team had adopted event sourcing without fully pricing in its structural constraints.
Trap 1: Eventual consistency between log and projections
In an event sourced system, the event log is updated synchronously with each command. The read models (projections) are updated asynchronously — they consume events from the log and update their materialized views in a separate process. This creates a staleness window: between when an event is appended and when the projection has processed it, queries against the read model return the previous state.
During normal operation this lag is milliseconds to low seconds. During high load, projection backpressure, or a restart, it can stretch further. The dashboard showing incorrect overdue amounts was not a bug — it was the projection lag exposed under load.
Managing projection lag requires deliberate design:
- Monitor lag as a first-class metric: alert when the projection-to-log offset exceeds a threshold (seconds, not minutes).
- Size the projection workers for peak event throughput, not average throughput.
- Design read models with staleness in mind: a dashboard showing “as of N seconds ago” is better than a dashboard that silently shows wrong data without acknowledging it.
- Do not use async projections for decisions on the write path: if the write side needs to read a value before appending an event, query the write-side aggregate state, not a potentially-stale read model.
The eventual consistency trap is not unique to event sourcing — it appears in any CQRS system with async projection (unit 07). In event sourcing it is amplified because the read model is the only query path; there is no normalized current-state table to fall back on.
Trap 2: Schema evolution — the immutable past, the changing future
Events are immutable facts about the past. The domain model is not static. These two truths create a recurring tension: the structure of an event that was correct in 2023 may not match what the domain model expects in 2025.
Lesson 2 introduced upcasting as the mechanism. The trap is the operational discipline required to make it work at scale:
- Every event type needs a version identifier (explicit or embedded in the type name:
OrderApproved.v1,OrderApproved.v2). - Every upcast function must be maintained indefinitely — as long as v1 events exist in the log, the upcast from v1 → v2 must be deployed.
- Additive changes (new optional fields) are safe: tolerant readers handle them with defaults.
- Destructive changes (renaming or removing a field, changing a field’s meaning) require a new version and a migration strategy.
- A renamed field that appears in thousands of projections creates a wide blast radius — every projection that reads the old field name must be updated.
The trap: teams that do not enforce event versioning from day one accumulate implicit v1 events everywhere, then face a painful untangling when the first breaking change arrives. Discipline in naming (OrderApproved.v1) and in maintaining a registry of upcast functions is maintenance overhead that must be staffed.
▸Why this works
The structural lesson: an event schema is a public API with the worst possible deprecation policy — you cannot remove old consumers (the event log contains them permanently). Treat event schema changes with more care than REST API changes. The event log is the one dependency you can never break.
Trap 3: GDPR and the right to be forgotten
Article 17 of the GDPR requires that personal data be erased upon request. The immutable event log is a structural obstacle: personal data embedded in event payloads cannot be deleted without corrupting the log.
The standard structural response is crypto-shredding (also called crypto-erasure):
- Personal data in event payloads is encrypted using a per-subject encryption key (one key per customer, or per data-subject boundary).
- The event log stores the encrypted personal data — the ciphertext, not the plaintext.
- When an erasure request arrives, the encryption key is destroyed. The event payloads in the log still exist, but the personal data is now computationally irretrievable — effectively erased.
Crypto-shredding has structural prerequisites:
- Personal data must be identified upfront in every event type. Ad-hoc fields added months later may be missed.
- The key store must be separate from the event log. Losing the key store before an erasure request is processed means the data is never erasable.
- Projections that materialized the plaintext must also be updated — crypto-shredding the event log does not automatically purge a read model that already extracted and stored the personal data in plaintext.
Teams that do not design for crypto-shredding from the beginning face a painful retrofit: identifying all personal data in hundreds of event types, adding encryption retroactively (not possible for existing events), and potentially accepting that pre-retrofit personal data cannot be erased from the log.
Trap 4: The bad event is set in stone
During a manual support operation, an engineer appended an OrderApproved event for the wrong order ID. In a CRUD system, the fix is an UPDATE. In an event sourced system, there is no UPDATE.
The correct response to a bad event is a compensating event: a new event appended to the log that reverses the business effect of the erroneous one. For the wrong-order approval, a OrderApprovalReverted event is appended with a reason explaining the error. The aggregate’s fold logic handles OrderApprovalReverted by returning the order to its previous state.
This is not a workaround — it is the intended mechanism. It preserves the full audit trail: the approval, the error, and the correction are all visible in the log. A compliance auditor can see exactly what happened, when, and why.
▸lesson.inset.note
Compensating events require forethought: the domain model must define correction events for each write that can go wrong operationally. Teams that discover this only when a bad event has already been appended face the worst possible moment to design a compensating event schema.
The structural implication: operator tooling that directly writes to the event log must be treated with the same discipline as production application code. Appending a wrong event via a maintenance script causes exactly the same problem as a code bug. Event logs in production should be access-controlled with the same strictness as production databases.
The billing platform receives a GDPR erasure request for customer #C-8821. The customer's data (name, email, tax ID) appears in 340 events across the OrderPlaced, InvoiceGenerated, and PaymentReceived event types. The system was NOT designed with crypto-shredding. What are the realistic options, and what does each cost?
Trap 5: Operational and complexity cost — when not to event-source
Event sourcing is not a default architectural choice. It imposes costs that must be justified by requirements:
- Write side complexity: instead of an
INSERT/UPDATE, every write requires loading the aggregate (fold or snapshot + tail), validating the command, producing an event, and appending it atomically. This is significantly more code than CRUD. - Read side complexity: there is no normalized current-state table to query. Every query needs a read model, which needs a projection, which needs a pipeline. For a system with 20 entity types, this means 20+ projections to build and maintain.
- Operational complexity: the event store, the projection workers, the snapshot store, and the key store are all production infrastructure. Each needs monitoring, capacity planning, and failure recovery. Projection lag must be monitored as a first-class metric.
- Testing complexity: event sourced aggregates are tested by asserting on emitted events (given these past events → when this command → expect these new events). This is learnable but unfamiliar to most backend teams.
An engineer proposes applying event sourcing to every entity in the B2B billing platform: orders, line items, invoices, payment records, user preferences, UI notification settings, and session tokens. What is the correct response, and how should the decision actually be made?
The honest guidance on when to event-source:
Event sourcing is a strong fit when:
- The entity’s history of state transitions is a compliance or legal requirement (financial transactions, healthcare records, regulated decisions)
- Temporal queries over past state are needed in production, not just for debugging
- The domain requires distinguishing between different business reasons for the same resulting state (OrderApproved by reviewer vs AutoApproved by rule)
- The engineering team has genuine experience with the pattern — event sourcing adopted by a team that has never maintained a projection pipeline is high-risk
Event sourcing is a poor fit when:
- The entity is CRUD-heavy with no meaningful history (preferences, settings, session data, cache entries)
- The primary requirement is fast, flexible querying of current state — there is no audit need
- The team is not yet experienced with projection infrastructure and lag management
- The system must be delivered on a tight timeline — event sourcing increases initial delivery time significantly
A developer argues: 'Event sourcing and CQRS are the same pattern — you can't use one without the other.' What is the correct position, and why does the distinction matter?
- 01What is crypto-shredding and why is it the standard GDPR response for event sourced systems?
- 02A bad event was appended to the event log due to a support engineer's mistake. Why can it not simply be deleted, and what is the correct response?
- 03Name three categories of entities for which event sourcing is a poor fit, and explain why.
Three fires, one root cause: the team adopted event sourcing without pricing in its structural constraints.
The GDPR fire was predictable. Personal data in an immutable event log cannot be deleted. Crypto-shredding — encrypting personal data per subject, destroying the key on erasure — is the structural solution, but it must be designed in from day one. A retrofit is painful or impossible for events already stored in plaintext.
The bad-event fire was also predictable. The event log is immutable. Operator tooling that writes directly to it must be treated with production discipline. When a bad event lands, the response is a compensating event — a new fact that reverses the business effect. History is preserved; the error and its correction are both visible.
The projection-lag fire was a monitoring failure. Eventual consistency between the event log and the projections is structural. Under load, the lag stretches. Projection lag must be monitored as a first-class metric, sized for peak throughput, and acknowledged in the UI where it matters.
Event sourcing and CQRS are independent patterns that compose well but are not the same thing. Event sourcing justifies its costs for aggregates with genuine audit, temporal-query, or intent-capture requirements. For preferences, session data, and simple CRUD entities, the costs are real and the benefits are absent.
The structural track ends here. The next unit (09) steps from event sourcing — how state is stored — to event-driven architecture: how modules communicate through events and how that shapes coupling.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.