The Distributed Monolith

What makes a distributed monolith

A distributed monolith is a set of services that appear independent but are tightly coupled at runtime and at deploy time. It combines the worst properties of both architectures:

• From the monolith: a change to one service requires changes and coordinated deploys across many services; the codebase is hard to understand as a whole because the coupling is implicit (network calls and shared schema, not visible imports).
• From distributed systems: network latency on every inter-service call, partial failure modes that are harder to debug than process-level exceptions, and the operational overhead of multiple deployable units.

The diagnostic question is not “how many services do we have?” — it is “can each service be deployed and failed independently?” If the answer is no, you have a distributed monolith regardless of how many services the architecture diagram shows.

Three structural symptoms define the pattern:

1. Lockstep deployment. Services must be deployed together, or in a specific sequence, because they share a data schema or API contract that is not versioned. A change to the orders table requires coordinating OrderService, BillingService, and InventoryService in the same deploy window.

2. Chatty synchronous call chains. A single client request fans out to N synchronous hops through services that cannot respond until downstream services respond. Each hop adds latency and failure probability. Under load, a slow service at the end of the chain produces timeouts that cascade upstream.

3. Shared database. Multiple services read and write the same tables via a shared ORM or shared connection pool. No service exclusively owns its data; schema changes affect every service that queries the changed table.

The dual-write problem

The shared database produces a structural defect that surfaces as data inconsistency: the dual-write problem.

Consider BillingService creating a charge and updating an order status in sequence:

1. INSERT INTO charges (order_id, amount) VALUES (123, 450.00)
2. UPDATE orders SET status = 'charged' WHERE id = 123

Both statements succeed locally. But orders is also written by OrderService. If OrderService concurrently writes status = 'cancelled' after step 1 but before step 2, the order ends up in the charged state even though it was cancelled. BillingService and OrderService each wrote their local view of the truth to the same table, with no owner enforcing consistency.

The dual-write problem cannot be fully solved by transactions when the writes span services: a transaction that wraps both services’ writes is a distributed transaction — which reintroduces 2PC and its failure modes (unit 09). The correct fix is to give each service exclusive ownership of its tables, eliminating the possibility of concurrent writes from different services to the same record.

Temporal coupling: the hidden coupling of synchronous chains

Temporal coupling is a form of runtime coupling that occurs when one service can only complete its work if another service is available at the same moment. Every synchronous service call creates temporal coupling.

In the billing platform’s five-service chain, OrderService is temporally coupled to InventoryService, BillingService, and NotificationService. If any one of them is unavailable when an order is placed — even briefly, during a deploy, or during a GC pause — the entire order-placement operation fails. The failure of a notification service (low business priority) causes order placement (high business priority) to fail. This is the inverted risk profile of temporal coupling: the most critical business operation fails when the least critical service is unavailable.

Temporal coupling also makes independent deployment harder. If OrderService must call InventoryService synchronously on every request, they cannot be tested independently. The behavior of OrderService under InventoryService failures must be tested with a live InventoryService or with carefully crafted test doubles. In practice, teams skip this testing because it is hard, producing the distributed monolith’s characteristic fragility.

Diagnosing a distributed monolith

The billing platform’s architecture review checklist for the distributed monolith:

Change one service, must change three. If adding a field to the Order domain object requires code changes in OrderService, BillingService, and InventoryService — even when only one of them introduces the field — the services are structurally coupled. A true service boundary means the field change is internal to one service’s codebase.

A single request fans out to N sync hops. If the system’s sequence diagram for a single user action shows a waterfall of synchronous calls through four services, each of which must complete before the previous can respond, the services are an implementation detail of one workflow, not autonomous capabilities.

Deploys require a coordination thread. If deploying BillingService requires a Slack thread to confirm with the OrderService team that the shared schema migration is compatible, the deployment coupling is explicit. Autonomous services can deploy any time without inter-team coordination.

No service can fail in isolation. If bringing InventoryService down for maintenance causes order placement to fail, order placement is not isolated from inventory. Autonomous services degrade gracefully when their dependencies fail — they do not propagate failure upstream.

How to fix a distributed monolith

The distributed monolith is fixed by the same approach that prevents it: decompose by business capability along bounded-context seams (lesson 02), enforce data ownership, and replace chatty synchronous chains with asynchronous event-driven communication where the business allows eventual consistency (unit 09).

The incremental fix:

1. Identify the chattiest synchronous chain. The call chain that fans out the furthest is the most urgent temporal coupling to break.
2. Determine which calls are blocking vs. which are notifications. Sending a confirmation notification does not need to block the order transaction. Reserving inventory does — the business requires knowing if stock is available before committing the order.
3. Replace notifications with events. OrderService publishes an OrderPlaced event; NotificationService subscribes and sends the email asynchronously. The order-placement transaction completes without waiting for notifications.
4. Address data ownership. Identify which tables each service actually owns. Move exclusive write access to the owning service. Replace all other access with API calls to the owning service.
5. Version APIs, not schemas. When two services must share a concept (the order status), they share it through an API with a versioned contract — not through a shared table that both write to.