Drill: the network

Market data is one-to-many and loses value in milliseconds. Multicast lets the network deliver each packet to every subscriber at once, and UDP never holds back new data while waiting for a lost packet to be retransmitted. Loss is handled by sequence numbers, redundant A and B feeds and recovery services.

Orders are one-to-one and every message must arrive, once and in order: a cancel must not overtake its order. TCP gives that, and on dedicated low-loss links its cost in normal operation is close to UDP's. The exchange's session protocol adds its own sequencing on top.

First, arbitration: 5001 may arrive on the B feed, or slightly reordered, so I would wait briefly and buffer 5002 onwards. If it does not arrive, the books on that channel may be wrong, so I would mark them stale and tell strategies not to trade on them.

Then recover: request 5001 from the exchange's retransmission service and apply the buffered packets after it, or, if that is unavailable or the gap is large, rebuild from a snapshot and replay buffered packets newer than the snapshot's sequence number. Gaps and recoveries go into monitoring, because frequent gaps point at drops on our side.

Bursts at the open can reach line rate for short periods, and every queue on our side must absorb them. I would check the switch port counters, ethtool statistics for NIC ring drops, and netstat or /proc/net/udp for socket buffer drops, alongside our own ring counters.

Then fix the layer that drops: larger NIC rings and socket buffers, a handler that keeps up with the burst rate with its work on the receive thread kept minimal, kernel bypass with a dedicated polling core on the card's socket, and measuring rates at millisecond resolution rather than per second.

That pattern is Nagle's algorithm: with data still unacknowledged, TCP holds a new small write back to combine it with later data. The exchange side may delay its acknowledgement, so the hold lasts tens of milliseconds.

I would set TCP_NODELAY on the order connection and make sure each order goes out in a single send call, then confirm with packet captures and hardware transmit timestamps.

Kernel bypass next: Onload gives most of the benefit behind the existing socket code, and ef_vi or DPDK give raw ring access if we handle the protocol headers ourselves. A dedicated thread on an isolated core local to the card polls the receive ring, so there are no interrupts, system calls or copies.

Beyond that, the fastest reactions move into hardware: an FPGA on the card that parses the feed and fires pre-armed orders under limits set by software. Each step costs more development effort and flexibility, so I would choose by how much latency the strategy is actually worth.