Condition variables and producer–consumer queues

template <class T>
class BlockingQueue {
    std::mutex m_;
    std::condition_variable cv_;
    std::deque<T> q_;
public:
    void push(T v) {
        {
            std::lock_guard g(m_);
            q_.push_back(std::move(v));
        }
        cv_.notify_one();                                // after unlocking: the woken thread can take the lock
    }
    T pop() {
        std::unique_lock lk(m_);
        cv_.wait(lk, [&] { return !q_.empty(); });       // re-checks the condition on every wake-up
        T v = std::move(q_.front());
        q_.pop_front();
        return v;
    }
};

Every line has a reason, and interviewers probe each one.

The mutex. The queue is shared state; every access is under the lock.

wait with a predicate. wait(lk, pred) is a loop: while the predicate is false, atomically unlock the mutex and sleep, then on waking relock and test again. The predicate is essential for two reasons:

• Spurious wake-ups. A waiting thread may return from wait without any notification. The standard allows it and some implementations do it.
• Stolen items. With several consumers, another thread may take the item between the notification and this thread reacquiring the lock.

Atomic unlock-and-sleep. wait releases the mutex and goes to sleep as one step from the point of view of other threads. That rules out the lost wake-up, where a notification arrives between checking the queue and starting to wait, and is never seen.

Notify after unlocking. Notifying while still holding the lock is correct, but the woken thread may then immediately block on the mutex. Notifying after the unlock avoids that extra hand-off. (Some implementations optimise the in-lock case; either is correct.)

The bug the predicate and the mutex together prevent:

// WRONG: checks the condition without the lock, then waits
if (q_.empty()) {               // consumer sees empty
                                // producer pushes and notifies here: nobody is waiting yet
    std::unique_lock lk(m_);
    cv_.wait(lk);               // consumer sleeps, notification already gone
}

The rule: the condition must be checked and the wait entered under the same mutex that the producer holds while changing the condition.

An unbounded queue hides a slow consumer until memory runs out. A bounded queue makes the producer wait on a second condition variable when the queue is full, or return a failure so the producer can decide what to drop. For market data, where the producer is a network feed that cannot be paused, the choice is what to drop or conflate, which Module 7 covers.

A consumer that sleeps on a condition variable must be woken by the kernel. The producer's notify becomes a futex system call; the consumer becomes runnable and waits for the scheduler to put it on a core; it then reacquires the mutex. Several microseconds is typical on an idle machine, with a long tail. The queue also allocates (the deque grows) and takes a lock on every push and pop.

A low-latency consumer instead runs on its own core and polls a lock-free queue in a loop. It never sleeps, so there is nothing to wake: a new item is seen within tens of nanoseconds, the time it takes the cache line to travel between cores. That costs a whole core per consumer, which trading firms happily pay. The next two lessons build that queue.

Blocking queues remain right for everything that is not latency-critical: logging, persistence, reporting, administration. Threads that would otherwise spin all day on idle work should sleep.