Skip to content
Work Practice Products Coding interviews Machine learning coding Quant developer HFT course Blog Method Why me About Enquire Book a call

Quant developer HFT course · Module 8: Measuring latency

Hardware performance counters

Lesson · 11 min

The question

Two versions of a function run at different speeds. You may only use perf stat. What do you look at to explain the difference?

What the counters are

Every modern core has a performance monitoring unit: a handful of counters that can be programmed to count events in the hardware, such as cycles, instructions retired, cache misses at each level, branch mispredictions, TLB misses and stalls of particular kinds. They count without slowing the program, and perf stat reads them for a run.

perf stat -e cycles,instructions,branches,branch-misses,cache-references,cache-misses,\
L1-dcache-load-misses,LLC-load-misses,dTLB-load-misses ./bench

The numbers that explain most differences

  • Cycles and instructions. Their ratio, instructions per cycle (IPC), is the first diagnostic. A modern core can retire four or more instructions per cycle. IPC above 2 means the code is running well; below 1, the core is mostly waiting.
  • Instructions alone. If one version executes fewer instructions at the same IPC, it is faster because it does less. If the instruction counts match but the cycles differ, it is about stalls, not work.
  • Branch misses per thousand instructions. Each costs around fifteen to twenty cycles; a few per thousand instructions matter in a tight loop.
  • Cache misses at L1, last-level cache and TLB. Last-level misses go to memory, a hundred nanoseconds or more each.

So the method for the question: compare instructions (did the work change?), then IPC (did the efficiency change?), then the miss counters to see which kind of stall accounts for the cycles.

Top-down analysis

Raw counters do not add up into an explanation easily. Intel's top-down method, available as perf stat --topdown on recent kernels and in VTune, divides every pipeline slot (each cycle, each of the core's issue positions) into four categories:

pipeline slotsretiringuseful workbad speculationmispredictionsfront-end boundfetch, decode, i-cacheback-end boundmemory or core
Top-down analysis splits every pipeline slot into four categories. The largest non-retiring category says where to look next: memory, the front end, or mispredicted branches.
  • Retiring: the slot did useful work.
  • Bad speculation: the slot did work that was thrown away, mostly from branch mispredictions.
  • Front-end bound: the slot was empty because instructions were not fetched or decoded in time: instruction cache misses, large code, decoder limits.
  • Back-end bound: the slot was empty because the back end could not accept work: waiting on memory (memory bound) or on execution units (core bound, for example long dependency chains or divider throughput).

Each category has sub-levels. The largest non-retiring category says where to look: back-end memory bound leads to the next lesson; bad speculation to the branches; front-end bound to code size and layout. AMD offers an equivalent pipeline-utilisation analysis.

Using counters well

  • Count over a stable, realistic run. Counters on a cold or unrepresentative run measure the wrong thing.
  • Normalise. Misses per thousand instructions, or per message, compare across versions better than raw totals.
  • Watch multiplexing. If you ask for more events than there are counters, perf time-shares them and scales the results, which adds error; it reports the percentage of time each was counted.
  • Pin the process so counts are not spread across cores and mixed with other work.
  • Counters on virtual machines may be unavailable or partial.

In production

Counters can also be read by the program itself, around a code region, through perf_event_open or a library over it, or by reading the time-stamp and performance counters directly with rdpmc once enabled. Some teams record instructions and cache misses per message alongside timestamps, which turns "this message was slow" into "this message was slow and took 40 LLC misses", a large step towards the cause.

Check yourself

Question 1 of 3

Two versions have the same instruction count but one takes 40% more cycles. What does that tell you?

Question 2 of 3

A loop has an IPC of 0.3. What is the most likely broad cause?

Question 3 of 3

Top-down analysis reports a function as 45% bad speculation. Where do you look?

In the interview

Say your own answer out loud first, then open the one we would give.

What do you look at first with perf stat?

Instructions and cycles, and their ratio, IPC. Fewer instructions means less work; lower IPC means more stalling. Then branch misses and cache misses at each level, normalised per thousand instructions, to see which stall explains the cycles.

What is top-down analysis?

A method that classifies every pipeline slot as retiring, bad speculation, front-end bound or back-end bound, and then subdivides those. It points directly at the class of bottleneck, memory, branches or instruction supply, instead of leaving you to interpret raw counters.

Can you use hardware counters in production?

Yes, around specific regions through perf_event_open or by reading counters directly once enabled, at a small cost. Recording instructions and cache misses per message along with timestamps helps explain individual slow messages.