The Go memory model, data races, and the -race detector
Go guarantees behavior only for race-free programs: happens-before comes from channels, mutexes, atomics. Any data race is undefined behavior; benign races do not exist. Run -race in CI (2–20x slower, 5–10x memory). Mutexes guard state; channels transfer ownership.
The shutdown handler set done = true; the worker loop checked for !done — a plain bool, written once, read in a loop. “It’s just a flag,” the review comment said, “worst case we do one extra iteration.” In production the worker never stopped. The compiler, seeing a loop that never writes done, hoisted the load out of the loop and spun on a register forever; the deploy hung on graceful shutdown until the orchestrator sent SIGKILL mid-write, and the team spent a day blaming Kubernetes. The same quarter, a different service crashed nightly with fatal error: concurrent map writes — two goroutines touching a cache map, a crash the runtime only catches sometimes, when its cheap sampling happens to notice. Both bugs had passed code review with the phrase “benign race.” The Go memory model has a blunt reply: a program with a data race has no defined behavior to reason about. There are no benign races — only races whose consequences you haven’t met yet.
Happens-before: the only ordering that exists
When you review concurrent code and something looks “obviously fine,” ask yourself: what creates the happens-before edge here? If you can’t name it, the compiler can’t see it either — and will act accordingly. The Go memory model (go.dev/ref/mem) defines when a read is guaranteed to observe a write: only when the write happens-before the read. Within one goroutine, program order gives you that for free. Between goroutines, happens-before is created exclusively by synchronization:
- a channel send happens-before the corresponding receive completes (and for unbuffered channels, the receive happens-before the send returns);
- the close of a channel happens-before a receive that returns a zero value because of it;
- an unlock of a
sync.Mutexhappens-before any laterLockthat acquires it; sync.WaitGroup.Donehappens-before theWaitit releases;sync.Once’s function happens-before anyDoreturning;- atomic operations in
sync/atomicbehave like tiny synchronized accesses — a goroutine that observes an atomic store also observes everything sequenced before that store.
Nothing else creates ordering. Not time.Sleep, not “the write obviously happened seconds earlier,” not a print statement that seems to fix it. Without an edge, the compiler may reorder and cache aggressively (hoisting the Hook’s flag load is legal), and each CPU core may serve reads from its own store buffer. The model is explicit: if two accesses to the same location from different goroutines are not ordered by happens-before and at least one is a write, that is a data race, and the behavior of the whole program is undefined — Go promises sequentially consistent semantics only to race-free programs. Multiword values make “undefined” concrete: an interface value or slice header is two/three words, so a racy write can be observed half-updated — a type pointer from the new value with data from the old — which is memory corruption, not a stale read.
The race detector: how it works and what it costs
Build or test with -race and the compiler instruments every memory access; the runtime (based on ThreadSanitizer) keeps vector clocks per goroutine and shadow memory recording the last accesses to each word. On every access it checks: is there a previous access to this address, from another goroutine, at least one a write, with no happens-before path between them? If yes, it prints both stack traces — the racing read and write, with goroutine creation sites. Two properties matter for how you use it:
- No false positives. It detects actual unsynchronized accesses against the formal model, not heuristics. Every report is a real bug; “we know about that one” is not a triage category.
- Only false negatives. It flags races that executed, not races that could. A race on a code path your test never runs, or an interleaving that never occurred, stays invisible. Coverage and realistic concurrency in tests are what make
-racestrong.
The price: typically 2–20× slower execution and 5–10× more memory (goroutines also start with bigger stacks). That is too heavy for most production fleets but trivial for CI — go test -race ./... belongs in every pipeline, and a canary instance built with -race in staging traffic is the senior move for catching interleavings tests never produce.
// The Hook's bug, minimally. `go test -race` reports it deterministically
// when both goroutines run; without -race it may "work" for months.
var done bool // shared, unsynchronized
func worker() {
for !done { // racy read — compiler may hoist, core may cache
work()
}
}
func shutdown() { done = true } // racy write
// Fix 1 (atomic flag): var done atomic.Bool … done.Store(true) / done.Load()
// Fix 2 (channel): done := make(chan struct{}) … close(done) /
// select { case <-done: return; default: }
// Fix 3 (ctx): pass a context.Context and select on ctx.Done()A goroutine loops on a plain bool flag set by another goroutine. A teammate argues the race is benign: worst case, one stale iteration. What does the memory model actually permit?
Mutexes, RWMutex, atomics: guarding state
sync.Mutex is the workhorse: Lock/Unlock create the happens-before edges that make everything done inside the critical section visible to the next locker. Go mutexes are cheap when uncontended (~tens of nanoseconds, a single atomic CAS) and unfair-but-bounded under contention (a starvation mode kicks in after ~1 ms of waiting, handing the lock to the longest waiter). They are not reentrant — locking twice from one goroutine deadlocks. Keep critical sections tiny and never hold a lock across I/O or a channel operation; the convention is to declare the mutex immediately above the fields it guards.
sync.RWMutex allows concurrent readers and exclusive writers. The honest tradeoff: its bookkeeping is heavier than Mutex, so it wins only when read sections are long or readers are many and writes are rare; for a nanosecond-scale read of a couple of fields, a plain Mutex is usually faster, and a frequently-written RWMutex is slower than Mutex while also letting readers starve writers’ arrival window. For read-mostly data that changes wholesale (config, routing tables), atomic.Pointer[T] — readers load a snapshot pointer, the writer builds a new immutable value and stores it — gives lock-free reads with one allocation per update. sync/atomic types (atomic.Bool, atomic.Int64, atomic.Pointer) cover flags and counters; reaching for atomics to build multi-variable invariants by hand is how lock-free bugs are born — if two fields must change together, that is a mutex’s job.
▸lesson.inset.deep-dive
Why the runtime catches concurrent map writes but not your flag race: maps already carry a hash-state word the runtime touches on every operation, so it cheaply sets and checks a writing bit there — a best-effort tripwire that costs nothing and catches only some interleavings, crashing with an unrecoverable fatal error rather than a panic precisely because the map’s internals may already be corrupt. Generalizing that check to every variable would be the race detector, with its 2–20× cost. The asymmetry explains production folklore: map races crash loudly (sometimes), all other races corrupt quietly (always-eventually). Neither is a synchronization strategy.
Channels or mutexes — the slogan, qualified
“Don’t communicate by sharing memory; share memory by communicating” is design advice, not a ban. The honest decision rule: mutexes protect state in place; channels transfer ownership of data or coordinate lifecycles. A counter, a cache, a set of in-place fields — that is mutex territory, and forcing it through a channel-owning goroutine adds latency, an extra goroutine to manage, and an API where every read is a round-trip. A pipeline stage handing off work items, a worker pool, cancellation, “exactly one goroutine may touch this buffer now” — that is channel territory, and emulating it with mutexes and condition variables reinvents what channels already are. The codebase smell test: a struct with a mutex and twelve methods that each lock-mutate-unlock is fine; a channel used as a mutex (buffer 1, send to lock, receive to unlock) is a sign someone applied the slogan instead of the rule. Either way, both tools compile to the same memory-model currency: happens-before edges. Pick the one that makes ownership obvious to the reader.
Your CI runs go test -race ./... and it has been green for a year. A race-detector report then fires in a staging canary. What does the year of green CI actually tell you?
You have a shared in-memory cache (map) read by 50 goroutines and written by one background refresher every 30 s. Which synchronization primitive fits best?
- 01What creates happens-before in Go, and why is a race on a plain bool flag not benign?
- 02How does the -race detector work, what does it cost, and what do its reports and silences mean?
The Go memory model defines exactly one mechanism of cross-goroutine visibility: happens-before edges, created by synchronization and nothing else. Channel sends happen-before their receives complete, close happens-before the zero-value receive it causes, mutex unlocks happen-before subsequent locks, WaitGroup and Once have their analogous guarantees, and sync/atomic operations act as synchronized accesses. Real time, sleeps, and apparent obviousness create no ordering — so a plain bool shutdown flag read in a loop is a data race, and the compiler may hoist the load so the loop never observes the write, which is how a graceful shutdown hangs until SIGKILL. The model promises sequentially consistent behavior only to race-free programs; a racy program has undefined behavior, including half-written interface values and slice headers — memory corruption, not just staleness. The runtime’s concurrent-map-writes crash is a cheap best-effort tripwire for maps only; everything else corrupts silently. The -race detector instruments every access and checks vector clocks against shadow memory: no false positives (every report is a real model violation), only false negatives (it sees only interleavings that ran), at 2–20x speed and 5–10x memory cost — which prices it into CI and staging canaries rather than production. For fixing races: sync.Mutex guards multi-field invariants with tiny critical sections and no I/O under the lock; RWMutex pays off only for long or numerous read sections with rare writes; atomic types handle flags and counters; atomic.Pointer snapshots serve read-mostly config. The slogan about sharing memory by communicating, honestly qualified: mutexes protect state in place, channels transfer ownership and coordinate lifecycles — both mint the same currency, happens-before, and the right choice is the one that makes ownership obvious. Now when you see a “benign” flag, a counter shared without a lock, or a cache touched from two goroutines — name the synchronization primitive before approving the review. If you can’t name it, add -race to the test run and let the detector answer for you.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.
Apply this
Put this lesson to work on a real build.