Packet drops and backpressure

A multicast packet passes several queues between the exchange and your code. Each has finite space, and each drops silently when full:

1. Network switches. Bursts from many sources converging on one port overflow its buffer.
2. The network card's receive ring. The card writes arriving packets by DMA into a ring of buffers in memory. If software does not take them off the ring fast enough, the ring fills and the card drops.
3. The kernel's socket receive buffer. With the normal stack, the kernel copies each datagram into the socket's buffer; if the application does not read fast enough, the buffer fills and the kernel drops.
4. Your own queues. A ring between the feed handler and a strategy fills if the strategy falls behind.

Average rates mislead. Market data arrives in bursts: at the open, at economic announcements, when a large order sweeps the book. A feed whose average is a small fraction of the link's capacity can, for a millisecond, arrive at line rate. Every queue on the path must absorb the burst for its duration. Measuring rates per second shows nothing; measuring per millisecond, or in microseconds, shows the problem.

• Switch: interface drop counters on the switch.
• NIC ring: ethtool -S <iface> counters such as rx_missed_errors or rx_no_buffer, depending on the driver.
• Socket buffer: netstat -su (receive buffer errors), /proc/net/udp drops per socket, or the SO_RXQ_OVFL option that reports the drop count with each recvmsg.
• Your rings: your own counters, which every queue should have.

Gaps in the feed's sequence numbers tell you that something was lost; the counters tell you where.

• Socket buffer: increase SO_RCVBUF and net.core.rmem_max; read with recvmmsg to drain many packets per call; or bypass the kernel entirely.
• NIC ring: increase the ring size with ethtool -G; make sure the receive interrupt or polling thread runs on an isolated core local to the card; use kernel bypass so a dedicated thread polls the ring continuously.
• The handler itself: it must keep up with the burst rate, not the average. Doing the minimum on the receive thread (arbitrate, sequence-check, decode, update, publish) and moving everything else elsewhere is how it does.
• Network: work with the network team on switch buffers and on separating feeds onto different ports.

When a consumer cannot keep up, the producer has three choices: wait, drop, or merge.

• Waiting (blocking the producer) is backpressure in the usual sense. It is right where the producer can be slowed without loss: a TCP sender, an internal batch job. A feed handler cannot slow the exchange, so blocking it only moves the loss upstream to the NIC ring and affects every other consumer too.
• Dropping with detection: the consumer sees a gap and resynchronises, for example from a snapshot of the current book. This is how a slow strategy should handle a broadcast ring it has fallen a lap behind on.
• Conflation: keep only the latest state per key. A strategy that only needs the current top of book for each instrument does not need every intermediate update; a seqlock-protected top of book per instrument, or a ring that overwrites an instrument's pending update with the newer one, gives a slow reader the latest state without a backlog.

The design question is which consumers need every event (an order-by-order book builder, a recorder) and which need only the latest state (most pricing and risk), and to give each the right mechanism. A consumer that needs every event and cannot keep up is a capacity problem to be solved, not hidden.