SIMD and vectorisation

SIMD (single instruction, multiple data) registers hold several values side by side: 128 bits in SSE and NEON, 256 in AVX2, 512 in AVX-512. One instruction adds, compares or multiplies every lane at once. With 32-bit integers, an AVX2 compare tests eight values in one instruction.

Finding a price in an array of a few dozen levels, a common operation in a book, becomes: broadcast the target into all lanes, compare eight prices at a time, turn the comparison result into a bit mask, and find the first set bit.

#include <immintrin.h>
int find_level(const std::int32_t* px, int n, std::int32_t target) {   // n a multiple of 8 here
    const __m256i t = _mm256_set1_epi32(target);
    for (int i = 0; i < n; i += 8) {
        __m256i v = _mm256_loadu_si256(reinterpret_cast<const __m256i*>(px + i));
        int mask = _mm256_movemask_ps(_mm256_castsi256_ps(_mm256_cmpeq_epi32(v, t)));
        if (mask) return i + __builtin_ctz(mask);
    }
    return -1;
}

Compilers vectorise simple loops on their own at -O2 or -O3 when they can prove it is safe and profitable. Things that stop them:

• Possible aliasing. In void add(float* out, const float* a, const float* b, int n), the compiler must allow for out overlapping a. It either adds a run-time overlap check or gives up. __restrict (a common extension) promises no overlap.
• Early exits. A loop with break or return inside, like the search above, has an unknown trip count; GCC and Clang have limited support for vectorising these, so search loops often stay scalar.
• Function calls that cannot be inlined.
• Floating-point reductions, which need reassociation, as in the previous lesson.
• Non-contiguous access such as walking a linked structure or gathering through indices; gathers exist but are slow.
• Complicated control flow in the body.

-fopt-info-vec-missed (GCC) or -Rpass-missed=loop-vectorize (Clang) tell you why a loop was not vectorised. Reading those reports is more productive than guessing.

SIMD wants the same field of many objects next to each other. An array of structs (Quote quotes[n]) puts each quote's price 24 bytes apart; a struct of arrays (int32_t px[n]; int32_t qty[n];) puts the prices contiguously, ready to load eight at a time. Hot paths that scan one field across many records often keep that field in its own array for this reason.

• Compile for the production machine. -march=native on the build host, or the exact target such as -march=icelake-server, enables the instructions that machine has. A binary built for a newer CPU than it runs on crashes with an illegal instruction.
• AVX-512 doubles the width again and adds masking that makes awkward loops easier to vectorise. On some older Intel server generations, heavy 512-bit instructions lowered the core's clock speed for a while afterwards, which could slow the scalar code around them; newer generations largely removed that penalty. It is a classic interview detail, worth stating with that qualification.

The per-message path in a trading system is often short, branchy and dominated by memory access, which SIMD does not help. It pays off in specific places: parsing (finding delimiters, validating fields, converting digits), searching small arrays of levels or ids, checksums and decoding, and batch computations in pricing or risk. Use it where profiling shows such a loop, and keep a scalar version for testing and for machines without the instructions.