Hardware counters and Intel TMA: sub-category diagnosis

Hardware counters: the second pass

A flame graph names the function. Hardware performance counters tell you what the function is doing inside the CPU. Linux’s perf stat -e cycles,instructions,cache-misses,branch-misses attached to the same hot leaf gives IPC, miss-rates, and stall types.

• Wide frame with IPC of 3.0: compute-bound. The CPU is executing the algorithm. Fix family: algorithm, SIMD, specialisation.
• Wide frame with IPC of 0.4 and 15% cache-miss rate: memory-bound. The CPU is stalled on RAM. Fix family: data layout change.

Same width on the flame graph, opposite fixes. Hardware counters are the second-pass diagnostic that prevents wrong-toolbox optimisation on subtle hot paths.

Intel TMA: a rigorous taxonomy

When you hit a case where “is it compute or memory?” isn’t obvious from IPC alone — or when the SLO demands absolute certainty before a sprint of restructuring — you need a finer instrument than the five-shape model.

The five-shape model is a working approximation. The rigorous version is Intel’s Top-Down Microarchitecture Analysis (TMA), formalised in the Intel Optimization Manual and exposed by VTune, Linux perf (via toplev.py), and AMD’s uProf equivalent.

TMA classifies each CPU cycle into four top-level buckets:

• Retiring (~25–50% on optimised code): real work — the CPU executed useful instructions.
• Bad Speculation (~5–15%): branch misprediction — pipeline was flushed, instructions were discarded.
• Front-End Bound (~5–15%): instruction fetch or decode stalls — the CPU cannot keep the pipeline full with new instructions.
• Back-End Bound (~30–60% on typical workloads): memory or compute resources stalled.

Back-End Bound breaks down further:

• Memory Bound → L1 Bound, L2 Bound, L3 Bound, DRAM Bound, Store Bound
• Core Bound (compute ports, dependency chains, long-latency dividers)

The cascade pinpoints exactly which CPU resource the hot path is starved of:

• DRAM-bound → data-layout fix
• Bad Speculation → branch elimination
• Front-End Bound → code size reduction
• Core Bound → true algorithmic redesign or SIMD

For senior performance work on critical-path services, TMA is the highest-resolution diagnosis available. Teams shipping latency-sensitive infrastructure (HFT, database engines, kernel hot paths) treat it as standard.

# perf stat -e cycles,instructions,cache-misses,LLC-load-misses ./service --bench feed-rank

 8,400,000,000  cycles
 3,360,000,000  instructions          #  0.40 insns per cycle (IPC)
   900,000,000  cache-misses          # 10.7% of all memory refs
   700,000,000  LLC-load-misses       # 78% of cache misses miss L3 too

# Hot function from flame graph: score_embeddings()
# Self-time: 42% CPU
# IPC: 0.40   ← CPU stalled 60% of the time
# L3 miss rate: very high — going to DRAM on most accesses