Low-Latency C++ Patterns for HFT: A One-Page Map

The question

If you had five minutes to explain how you would make a C++ trading system fast, what would you say?

The short answer

Do nothing on the hot path that can wait, allocate, block or miss in cache: preallocate everything, keep state owned by one thread on its own core, pass data through lock-free rings, lay out data flat and compact, resolve dispatch at compile time, keep the kernel out of the data path, and measure everything with hardware timestamps and percentiles.

The table below expands each part into the pattern, the cost it avoids, and where it was covered.

The map

Pattern	Avoids	Module
Preallocate pools, rings and tables at start-up; reuse containers	allocation spikes, page faults	1, 3, 6
Pre-fault and lock memory; huge pages for large regions	page faults, TLB misses	6
Flat, contiguous data: vectors, arrays, open-addressing maps	pointer chasing, cache misses	1, 3
Hot fields together, cold fields elsewhere; struct of arrays for scans	wasted cache lines	1, 5
Integer ids mapped to dense indices at start-up	hashing strings on the hot path	3
Stable handles (index plus generation) instead of pointers	dangling after growth or reuse	2
Templates, CRTP, variants instead of virtual calls on the hot path	indirect calls, lost inlining	1, 3
No `std::function`, `shared_ptr` copies, exceptions or strings per message	hidden allocation, atomics	1, 2
One owning thread per piece of state	locks, contention, races	4
SPSC rings between threads; seqlocks for latest state	locks, kernel wake-ups	4
Padding written data to separate cache lines	false sharing	4, 5
Busy-polling threads pinned to isolated cores	context switches, wake-up latency	4, 6
Kernel bypass for the network	interrupts, system calls, copies	6, 7
Logging as fixed binary records to another thread	formatting and I/O on the hot path	1, 8
Branches laid out for the common case; error paths cold	mispredictions, instruction cache	1, 5
Integer prices in ticks; fixed-point arithmetic	rounding errors, slow conversions	10
Hardware timestamps; percentiles from histograms	wrong conclusions	8, 10
Deterministic replay of recorded data	untestable, irreproducible bugs	9

What the patterns have in common

Most of them move work from the hot path to somewhere else: to start-up (allocation, faults, index maps), to compile time (dispatch, constants), to another thread (logging, slow consumers), or to hardware (bypass, timestamps). The rest make the work that must stay as cheap as possible: fewer cache lines, fewer branches the CPU cannot predict, no cache lines written by two cores.

What not to say

That one technique solves it ("we use lock-free queues"). Speed comes from removing every source of delay, and the tail is set by the worst one.
Numbers you have not measured.
That everything must be custom. Standard containers off the hot path, and as first versions on it, are fine; replacing them is a decision that measurements justify.

How to use this page

Before an interview, go down the table and, for each row, make sure you can say what the cost is, why it happens in the hardware or the OS, and what the code looks like. The drills at the end of each module are built to make you do exactly that.

Check yourself

Question 1 of 3

Which theme connects most low-latency patterns?

Question 2 of 3

Why are stable handles preferred to raw pointers for long-lived references to orders?

Question 3 of 3

Which statement would weaken an answer about making a trading system fast?

In the interview

Say your own answer out loud first, then open the one we would give.

How do you make a C++ trading system fast?

Keep everything that can wait, allocate, block or miss off the hot path. Preallocate and pre-fault memory; use flat, compact data and stable handles; resolve dispatch at compile time; give each piece of state one owning thread on an isolated core and connect threads with lock-free rings; bypass the kernel for networking; move logging to another thread; and measure with hardware timestamps and percentiles, using deterministic replay to test.

Which of those matters most?

It depends on where the time is going, which is why measurement comes first. In practice the largest tail improvements usually come from removing allocation, page faults, lock contention and kernel involvement; the median improves most from data layout and cache behaviour.

The patterns, in one place