Small buffer optimisation

Many objects are usually small. Most strings in a trading system are symbols, venue codes and short tags; most callbacks capture a pointer or two; most vectors of legs hold one or two legs. Allocating heap memory for each is wasteful when the contents would fit in the object itself.

Small buffer optimisation reserves space inside the object for small contents and uses the heap only when they do not fit. The object is a little larger; most instances never allocate.

All three major standard libraries implement it for strings, with different layouts:

So std::string("ESZ6") stores four characters and a terminator inside the object; a 40-character string allocates. The boundary is a property of the library, not of the language, which is a good thing to say when asked.

Some consequences:

• Moving a short string copies its characters. There is no heap buffer to steal, so a move of an inline string is a small copy. It is still cheap.
• Pointers into a short string move with it. s.data() points inside the object, so after the string is moved or copied elsewhere, a pointer taken earlier refers to the old object's buffer.
• capacity() of an empty string is not zero. It reports the inline capacity, 15 or 22.

• std::function stores small callables inline, as covered in Module 1. The buffer is small, typically room for two pointers in libstdc++.
• std::any does the same for small values.
• Small vectors. Libraries such as Boost, Folly and LLVM provide small_vector<T, N>: a vector with room for N elements inside it that moves to the heap only beyond N. The standard does not have one; C++26 adds std::inplace_vector<T, N>, which has fixed capacity and never allocates at all.

boost::container::small_vector<Leg, 4> legs;   // up to 4 legs without touching the heap
std::inplace_vector<Leg, 4> fixed;              // C++26: at most 4, never allocates

• Size. Every instance pays for the inline buffer, used or not. A vector of a million std::strings with libstdc++ is 32 MB before any characters are stored.
• A branch. Operations check whether the contents are inline or on the heap. This is cheap and predictable for uniform data.
• Larger moves. Moving an object with inline contents copies the buffer rather than a pointer.

For hot-path data with a known bound, the simplest version is a fixed buffer with no heap fallback at all: a symbol stored as char sym[8] and compared as one 64-bit integer, or a fixed-capacity array of legs. The fallback branch disappears, the size is fixed, and running past the bound becomes a validation error at the edge of the system.

struct Symbol {
    char c[8] = {};
    bool operator==(const Symbol& o) const { return std::memcmp(c, o.c, 8) == 0; }  // one 8-byte compare
};
static_assert(sizeof(Symbol) == 8);