When Python is the wrong tool: the decision tree and the salary-versus-servers arithmetic
Triage the bottleneck: I/O-bound and C-library-bound workloads keep Python; interpreter-bound hot loops at scale cost 10-50x per core. Free-threaded 3.13 changes thread math, not single-thread speed. Strangler-extract the hot path; keep Python where dev speed beats compute.
The feature-scoring endpoint burned 35 ms of pure-Python CPU per request. At 4 000 RPS that meant twelve 8-core pods doing nothing but interpreting the same loop. One engineer rewrote that single endpoint in Go — same algorithm, line for line: 1.1 ms per request, and the fleet shrank from twelve pods to two. Compute bill down $14k a month. Leadership’s next move was predictable: an RFC titled “Migrate all services to Go”. The audit killed it in one slide — the other thirty Python services combined used fewer cores than the old scorer alone; they were CRUD glue, orchestration, and batch jobs idling on I/O, where Go would have bought nothing but a year of rewriting. The scorer was the bill. The rest were never the bill. The senior skill is not picking a language — it is locating the bill before proposing the invoice.
Classify the bottleneck before judging the language
“Python is slow” is not a diagnosis. The diagnosis comes from lesson 1 — py-spy against the real workload — and lands in one of three buckets:
- I/O-bound — samples sit in socket waits, the CPU is mostly idle. Python is fine: an idle core is idle in every language. The levers are asyncio, connection pooling, and concurrency, not a rewrite. A Go port of a service that waits on Postgres still waits on Postgres.
- C-library-bound — frames live inside numpy, torch, the DB driver. Python is fine: it is already a thin orchestrator over compiled kernels (lesson 3’s whole point). The levers are batching and feeding the C better, and a rewrite re-implements glue around the same kernels.
- Interpreter-bound — pure-Python frames are hot, at scale. Now the 10-50x per-core gap is real money. Escalate the ladder: algorithm first, vectorize or extend (lesson 3), and only when the path stays hot and stays the bill — rewrite that path.
Free-threaded 3.13 changes the thread math, not the speed
The free-threaded build (PEP 703) makes the GIL optional: pure-Python threads can finally run in parallel. What it does not change is single-thread speed — early free-threaded builds actually pay a single-digit-percent overhead, and the specializing interpreter took time to return. So it moves one boundary: “use processes or released-GIL C for CPU parallelism” relaxes toward threads. The per-core interpretive gap that drives the cases below survives intact: forty slow cores in parallel are still forty slow cores.
Where Python is the wrong tool
Three honest cases, each anchored in a number from this unit:
- Sustained CPU-bound services at scale. 10-50x slower per core means 10-50x the fleet for the same throughput. On 2 cores nobody cares; at the Hook’s twelve pods — or at hundreds of cores — the interpreter is the infrastructure bill.
- Hard-latency paths. Generational GC pauses (a gen-2 pass over a big heap is milliseconds), allocator jitter, and interpreter variance fight a single-digit-millisecond p99 SLO. You can mitigate —
gc.freeze, tuning thresholds — but you are negotiating with the platform, and the SLO does not negotiate. - Memory-constrained, high-cardinality data. Lesson 2’s arithmetic: 28-byte ints and 250-byte objects put 10 M small records into gigabytes that Go or Rust hold in hundreds of megabytes. If the working set must fit in RAM per node, object overhead picks the language.
All three cases share the same shape: the Python overhead is real, measurable, and already the bill — not a theoretical concern. Without case 1’s sustained core count, case 2’s hard latency target, or case 3’s cardinality-driven RAM cliff, the arithmetic almost certainly rejects the rewrite before you finish the RFC.
Which of these services is Python actually the wrong tool for?
Where Python stays right: the salary-versus-servers arithmetic
Glue, orchestration, internal tools, data science, ETL control planes — domains where iteration speed dominates compute. Spell the arithmetic out, because it decides real RFCs. A loaded senior engineer-year is $150-300k. A cloud vCPU-year is roughly $200-600. A Python service wasting 10x on 4 cores burns maybe $10-20k a year — less than one engineer-month. A rewrite must displace more core-dollars than it costs in engineer-dollars, forever, maintenance included. Most services never clear that bar. The Hook’s scorer cleared it more than tenfold — 240 wasted cores — which is exactly why it was the one service worth moving.
A Python service runs on 4 vCPUs (~$1.6k/yr at $400/vCPU-yr). An RFC proposes a 2 engineer-month Go rewrite promising 10x per-core efficiency. Verdict?
Migration pattern: strangler, not big bang
When a path does clear the bar, extract it — do not rewrite the world. Put the measured hot path behind a contract (protobuf or OpenAPI), implement that one service in Go or Rust, shadow traffic against it, shift percentages, keep the Python orchestration around it. Big-bang rewrites stall because the 90% of code that was never the bill must still be re-implemented bug-for-bug, while the strangler ships the 10% that pays. Each subsequent extraction is justified by its own core math — and in the Hook’s system, after the scorer moved, no other service ever qualified.
- 01Walk the decision tree for a slow Python service: the three buckets, the evidence for each, and the prescribed move.
- 02State the salary-versus-servers arithmetic and apply it to both a 4-core and a 300-core interpreter-bound service.
This lesson is the unit’s verdict step. Lessons 1-3 produce the evidence — where the samples land, what the objects cost, what an extension can and cannot buy — and the decision tree turns evidence into a move. I/O-bound and C-library-bound services stay Python, not out of loyalty but because their cores are idle or already executing compiled kernels, so a rewrite buys syntax. Interpreter-bound paths under sustained load are where the structural 10-50x per-core gap converts into fleet size, and even then the escalation runs algorithm, vectorize, extend before rewrite. Free-threaded 3.13 relaxes the parallelism rule without touching per-core speed. The economics gate every move: engineer-years cost six figures, vCPU-years cost hundreds, so a rewrite must displace more core-dollars than it consumes in engineer-dollars forever — arithmetic that rejects the 4-core vanity rewrite and approves the 300-core scorer extraction in the same breath. And the execution pattern is the strangler: one measured hot path behind a contract, traffic shadowed then shifted, Python keeping the orchestration it is best at. Now when you see an RFC proposing to migrate a service to Go or Rust, you know the first question: run py-spy on the real workload, classify the bottleneck, and put the core-dollar arithmetic in the room before the language debate starts.
Practice
Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.
Something unclear?
Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.