Quant developer HFT course · Module 5: The CPU
Drill: the machine
Hardware questions reward mechanisms, not numbers. Say what the cache, the coherence protocol or the pipeline does, then how the code triggers it, then how you would confirm it.
Prompt 1: The big table
The interviewer says
A lookup table used on every message is 8 MB. What would you consider?
Before you answer
Question · choose every correct option
Which points are relevant? Choose all that apply.
The cost of each lookup depends on which cache level serves it, which depends on the working set.
Answers that lose the point
It's O(1), so it's fine.
Constant in complexity, not in memory latency.
A strong answer
At 8 MB the table is larger than any per-core cache and a large share of L3, so lookups will miss to L3 and some to memory, especially after other code has run. I would look at what the hot path actually reads: smaller types, packing, dropping fields it never uses, or splitting the frequently used entries into a small table that fits in L2.
Then I would confirm with cache-miss counters on a recorded day, before and after.
Prompt 2: The popular counter
The interviewer says
Eight threads increment one atomic counter. Why does it not scale?
Before you answer
Question · choose every correct option
Which statements are correct? Choose all that apply.
Memory ordering does not change the need for exclusive ownership of the line.
Answers that lose the point
Use memory_order_relaxed.
Fine for correctness, irrelevant to the coherence traffic.
A strong answer
An atomic increment needs exclusive ownership of the counter's cache line, so with eight writers the line is passed from core to core, each transfer costing tens of nanoseconds or more, and only one increment can happen at a time. Relaxed ordering does not change that.
I would give each thread its own counter on its own cache line and sum them when the total is read, or have one owning thread aggregate messages from the others.
Prompt 3: No shared data, still slow
The interviewer says
Each thread writes its own element of an array of small stats structs. More threads, slower threads. Why?
Before you answer
Question · choose every correct option
Which explain it? Choose all that apply.
It is false sharing, not a race: each thread writes different bytes.
Answers that lose the point
There's a race; add a mutex.
Adds real contention to fix imaginary sharing.
A strong answer
False sharing. The structs are small, so several threads' elements sit in one 64-byte line, and the coherence protocol treats the line as the unit: every write invalidates the others' copies and the line moves between cores although no data is shared.
I would align each element to its own cache line, or 128 bytes if adjacent-line prefetching is a factor, and confirm with perf c2c, which shows different threads writing different offsets of the same line.
Prompt 4: The slow sum
The interviewer says
This loop sums a million doubles with one accumulator. How could it run several times faster on the same core?
Before you answer
Question · choose every correct option
Which statements are true? Choose all that apply.
The core has several floating-point units; the dependency chain keeps them idle.
Answers that lose the point
Use multiple threads.
Possible, but the single core is far from its own limit.
A strong answer
The loop is a single dependency chain: each addition waits for the previous one, so it runs at the latency of an add, around four cycles, while the core could start several additions per cycle. Splitting the sum into four or eight independent accumulators and combining them at the end lets the out-of-order core overlap them, and also lets the compiler use SIMD.
The compiler will not reorder floating-point additions on its own, because it changes rounding, so I would write it explicitly or enable reassociation knowingly, and check the result stays within tolerance.
Prompt 5: Reading through handles
The interviewer says
A loop walks an array of handles and reads each referenced order from a pool. It is slow. Ideas?
Before you answer
Question · choose every correct option
Which ideas fit? Choose all that apply.
Prefetching hides latency; changing the layout removes it. Sorting helps only if it improves locality and the order does not matter.
Answers that lose the point
Add __builtin_prefetch everywhere.
Without a measured distance it can do nothing or harm.
A strong answer
The handles stream sequentially and the hardware prefetches them, but each order's address comes from data, so every order read is a potential miss that the hardware cannot anticipate.
Since the loop knows future handles, I would prefetch the order a measured number of iterations ahead. Better, if the loop only needs a few fields, I would keep those in a structure of arrays indexed like the handles, so the access becomes sequential and needs no prefetching at all. I would check the change with cache-miss counters and a benchmark on real data.
After the drill
The machine rewards three things: a small working set, data written by one core at a time, and independent work it can overlap. Module 6 moves down to the operating system, which can take all of that away by moving your thread or faulting in a page at the wrong moment.