Data & money: build a correct money mover
Hands-on project: build a small wallet/ledger service that survives retries and concurrency — a double-entry ledger, client idempotency keys, an atomic transfer that defeats the lost update, and a reconciliation pass — proven with adversarial tests.
Reading that idempotency keys and a double-entry ledger prevent double-charges and lost updates is not the same as building a service that survives a retry storm and a thousand concurrent transfers without losing a cent. Build a small money mover, then prove its correctness the only way that counts: with adversarial tests that fire duplicate requests and concurrent debits at it and check that the books still balance.
This project makes the unit operational: you’ll implement the two load-bearing patterns — idempotency and the double-entry ledger — defeat the lost update with an atomic transfer, and then attack your own system with duplicates and concurrency to prove the invariant that the books always balance.
Build a small wallet/ledger service (any language + a real transactional database) that moves money between accounts correctly under retries and concurrency. It must implement an append-only double-entry ledger, accept client idempotency keys, perform atomic transfers that cannot double-spend or lose updates, and include a reconciliation pass that verifies the books balance — then prove all of it with adversarial tests.
- An append-only ledger where balances are computed by summing immutable entries, with a seed script and a query that prints any account's balance and its entry history.
- A transfer that is atomic (both entries or neither) and idempotent (a repeated key returns the original result with no second movement; a conflicting key is rejected).
- A passing retry-storm test: N concurrent identical requests produce exactly one transfer's worth of entries and a single balance change.
- A passing concurrent-debit test: many over-committed concurrent transfers leave the account non-negative and the global debits = credits invariant intact.
- A reconciliation command that recomputes balances from entries and reports the books as balanced (and would flag an injected imbalance), plus the design note explaining the isolation/locking choice and one failure reconciliation catches.
- Add a cross-shard transfer as a saga: split accounts across two databases, implement debit→credit with a compensating action, model the in-flight pending state, and show reconciliation healing a transfer you deliberately interrupt between the two legs.
- Add a snapshot/materialized balance and serve reads from it (snapshot + fold the tail of recent entries), then prove the snapshot always equals the full re-fold — the event-sourcing read optimization without losing verifiability.
- Add an external 'PSP' stub whose response you can drop/delay, put a charge behind the idempotency key, and show that a lost response + client retry does not double-charge, while a daily reconciliation against the PSP's report still catches a divergence.
- Stress the hot account: make one account the target of most transfers and measure how your locking choice serializes it; compare pessimistic locking vs an atomic conditional write vs optimistic/CAS under high contention and report the throughput and retry rates.
- 01Why derive balances by summing immutable entries instead of storing a mutable balance field?
- 02How do the two adversarial tests prove the two core invariants?
- 03What does the reconciliation pass catch that the hot path alone cannot?
This project turns the unit’s two load-bearing patterns into working code. You build an append-only double-entry ledger where every transfer is two immutable entries summing to zero and a balance is a derived sum, never a mutable field — so money is conserved by construction and the books are verifiable. The transfer is one atomic transaction (both legs or neither) guarded by a client idempotency key (atomic check-and-store under a unique constraint), and the lost update is defeated by an atomic conditional check, a row lock, or a serializable transaction. Then you attack your own system: a retry storm (N identical concurrent requests must move money exactly once) proves idempotency, and concurrent over-committed debits (the account must never go negative, debits must still equal credits) prove the lost update is closed. Finally a reconciliation pass independently re-sums entries and asserts the global invariant — the layer that catches lost responses, interrupted sagas, and drift the hot path can’t see. An engineer who has built this once stops trusting a single response as the truth about money and starts trusting the reconciled, append-only ledger — and knows, with tests to prove it, that retries and concurrency can’t make the books lie.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.