Threads, shared state and data races

++fills_ looks like one operation. On the machine it is three: load the value into a register, add one, store it back. Two threads can interleave those steps:

Both threads load 41, both add one, both store 42. One fill has disappeared. The more often the threads touch the counter at the same time, the more increments are lost, which is why the count is only sometimes short.

In C++, two threads accessing the same memory location, at least one of them writing, without synchronisation, is a data race, and a data race is undefined behaviour. Not "the result may be wrong": the whole program has no defined meaning, and the compiler is entitled to assume it never happens.

That assumption has visible effects. The compiler may keep a variable in a register across a loop because, in a race-free program, no other thread can change it:

bool stop_ = false;                       // plain bool, shared between threads

void worker() {
    while (!stop_) { poll_feed(); }       // the compiler may read stop_ once and loop for ever
}
void shutdown() { stop_ = true; }         // on another thread

If poll_feed is inlined and does not touch stop_, the optimiser can hoist the load out of the loop. The worker never sees the store. Code like this often works at -O0 and hangs at -O2.

The CPU reorders memory operations as well. Stores sit in a store buffer before they reach the cache, and on weakly ordered processors such as ARM, loads and stores to different addresses can become visible to other cores in a different order from the program's. Even on x86, a later load can be satisfied before an earlier store to a different address becomes visible.

So two layers, the compiler and the hardware, can both rearrange what one thread does as seen by another. The C++ memory model is the contract that tells you which arrangements are allowed, and synchronisation is how you rule the others out.

Shared mutable data is safe only if every access is ordered by one of:

• a mutex: everything done while holding it is visible to the next thread that takes it (lesson 2);
• an atomic variable, with the right memory ordering (lesson 4);
• thread start and join: everything before std::thread's constructor is visible in the new thread, and everything the thread did is visible after join() returns.

Data that is never written after it is shared needs no synchronisation: read-only configuration loaded before the threads start is fine. So is data owned by one thread and never touched by another, which is how most trading systems are built: each component owns its state, and threads exchange messages through queues instead of sharing objects.

std::atomic<std::uint64_t> fills_{0};
fills_.fetch_add(1, std::memory_order_relaxed);   // one indivisible read-modify-write

fetch_add is a single atomic operation (lock xadd on x86). relaxed ordering is enough for a pure counter, because nothing else is published through it (lesson 4). Better still for a hot path, give each thread its own counter and sum them when someone reads the total, so the two threads never write the same cache line (Module 5).

Races are timing-dependent and hide from ordinary tests. The tool that finds them is ThreadSanitizer: build with -fsanitize=thread, run the tests, and it reports each pair of conflicting accesses with both stack traces. It slows the program by several times and uses much more memory, so it runs in CI rather than production. Running the test suite under TSan as a matter of course is the answer interviewers want to hear.