Kernel bypass, RDMA and FPGAs

Kernel bypass removes the operating system from the data path. The network card's receive and transmit rings, and the registers that control them, are mapped into the application's address space. A thread polls the receive ring directly: when a descriptor shows a new packet, the payload is already in a buffer the application can read. Sending writes a descriptor and rings the card's doorbell from user space. No interrupts, no system calls, no copies, no kernel code.

Three common forms:

• Onload (from Solarflare, now AMD) intercepts the standard socket calls, with LD_PRELOAD, and implements TCP and UDP in a user-space library over the card's queues. Existing socket code gets most of the benefit without changes.
• ef_vi is the lower-level interface on the same cards: raw access to the queues, with the application handling Ethernet, IP and UDP headers itself. It is the fastest software receive path on that hardware, at the cost of writing protocol handling yourself.
• DPDK takes over the card completely with user-space poll-mode drivers, packet buffer pools and libraries for protocol processing. It is vendor-neutral and common in network appliances; for trading it is used where its throughput and control matter.

NVIDIA's VMA plays the Onload role on Mellanox cards, and several card vendors offer their own low-level APIs.

What bypass gives up: the kernel's protections and conveniences. The application must poll (a dedicated core per receive path), handle protocol details or trust the bypass library's implementation of them, and work without the standard tools that look at kernel sockets. Card vendors and models become part of the design.

Remote direct memory access lets one machine read or write another machine's memory through the network cards, without either host's CPU or kernel touching the data path. The application registers memory regions with its card, the peers exchange keys, and then a write posted on one side appears in the other side's memory. It runs over InfiniBand, or over Ethernet as RoCE.

Its strengths are moving data between servers in a data centre with very low latency and almost no CPU cost: replicating state between a primary and a backup, distributing data inside a firm's own cluster, storage. It is not how exchanges talk to participants; exchange connectivity is Ethernet with standard IP protocols. In a trading-firm interview, RDMA fits the internal architecture discussion: how to move data between your own machines fast.

The fastest trading paths no longer run on the CPU at all. An FPGA (field-programmable gate array) on the network card, or in a separate appliance, implements logic as hardware circuits: it can parse market-data packets as they arrive, bit by bit, update a small book, evaluate a simple trading rule, and send an order, all before the full packet has even finished arriving. Software tick-to-trade is measured in microseconds; FPGA paths are measured in hundreds of nanoseconds or less.

The trade-offs are significant: FPGA logic is written in hardware description languages or high-level synthesis, takes much longer to develop and test, and can hold only simple strategies and small state. A common division of labour is that software decides the parameters (prices, sizes, thresholds) and the FPGA executes the fast reaction to specific triggers within limits set by software. Risk checks move into the FPGA too, because the order never passes through the CPU.

None of this makes C++ performance irrelevant. Most strategies, all of the risk and control logic, and everything that needs flexibility run in software, and the software must still meet microsecond budgets and, increasingly, feed parameters to hardware quickly and correctly. In an interview, knowing the hierarchy is the point: tuned kernel path, then kernel bypass, then hardware offload, each faster, each costing more in development effort and flexibility.