# Case-Folding Source Code at 45 GiB/s: Branch-Free Byte Arithmetic and SWAR When scaling modern code search, indexers process petabytes of source code. A critical prerequisite for fast case-insensitive search is **case-folding**—normalizing every character so that `Identifier` matches `identifier`. On paper, converting uppercase characters to lowercase sounds like a trivial operation. In JavaScript or WebAssembly, you might call `toLowerCase()`. In standard C or Rust, you might loop over a byte slice and check character ranges. However, at scale, conventional case-folding becomes an immense CPU bottleneck. Standard implementations cap throughput around 2 to 4 GiB/s per core because of CPU branch mispredictions. Modern high-throughput search engines require case-folding at memory bandwidth speeds: **over 45 GiB/s on a single core**. In this deep dive, we will explore how branch-free arithmetic and SWAR (SIMD Within A Register) techniques achieve near-instantaneous case-folding while eliminating hardware pipeline stalls. --- ## The Branching Trap: Why Standard Lowercasing Fails To understand why traditional approaches choke under heavy throughput, consider the classic byte-by-byte ASCII lowercase algorithm: ```c void lowercase_naive(uint8_t *src, uint8_t *dst, size_t len) { for (size_t i = 0; i < len; i++) { uint8_t b = src[i]; if (b >= 'A' && b <= 'Z') { dst[i] = b + 0x20; } else { dst[i] = b; } } } ``` ### Why the CPU Pipeline Stalls Modern CPUs rely on deep execution pipelines and branch predictors to speculatively execute instructions ahead of time. When processing source code, character distribution is high entropy: 1. camelCase and PascalCase identifiers mix uppercase and lowercase letters unpredictably. 2. HTML/JSX tags, JSON keys, operators, whitespace, and comments introduce arbitrary ASCII byte ranges. Because the `b >= 'A' && b <= 'Z'` condition flips rapidly between true and false, the CPU branch predictor fails frequently. Each misprediction forces the CPU to flush its pipeline, discarding 15 to 20 cycles of speculatively executed work. As a result, the processor spends more time waiting for pipeline resets than executing instructions. --- ## Branch-Free Byte-Space Arithmetic To unlock memory-speed processing, we must eliminate conditional jumps entirely. We need a single arithmetic expression that transforms ASCII uppercase bytes (`0x41` to `0x5A`) into lowercase (`0x61` to `0x7A`) while leaving all other bytes unchanged. In ASCII, the difference between an uppercase letter (`'A'` = `0x41` / `0b01000001`) and its lowercase counterpart (`'a'` = `0x61` / `0b01100001`) is exactly **bit 5** (`0x20`). If we can derive a bitmask where bit 5 is set to `1` when the byte is uppercase, and `0` otherwise, we can perform case-folding with a single bitwise OR operation: ``` dst_byte = src_byte | mask ``` ### Deriving the Range Mask Without Branches To check if a byte `b` falls strictly in the range `['A', 'Z']` without branching, we use unsigned integer wrap-around arithmetic in byte space. Consider the expression: 1. Subtract `'A'` (`65`) from `b`. If `b` is between `'A'` and `'Z'`, the result is in the range `[0, 25]`. 2. If `b` was less than `'A'`, subtracting `'A'` causes an unsigned 8-bit underflow, wrapping the value to a high byte range (`[191, 255]`). 3. By adding a constant offset and performing bitwise manipulation, we can isolate whether the original byte fell in `[0, 25]`. A common branch-free bitmask computation for a single byte in C is: ```c inline uint8_t ascii_to_lower_branchless(uint8_t b) { // Returns 0x20 if b is in 'A'...'Z', otherwise 0x00 uint8_t is_upper = ((uint8_t)(b - 'A') < 26) ? 0x20 : 0x00; return b | is_upper; } ``` Modern compilers (GCC, Clang, Rustc) compile the unsigned comparison `(b - 'A') < 26` into a conditional bit-select or conditional move instruction (`cmov` / `setb`), completely avoiding branch jump instructions. --- ## SWAR: Processing 8 Bytes at Once on Any Core While branchless byte loops improve performance, processing one byte per iteration still leaves massive hardware capacity on the table. Vector extensions like AVX-512 or ARM Neon provide specialized 128-bit or 512-bit register operations. However, vector instructions are not always available or portable across edge runtimes (e.g., WebAssembly micro-engines or older servers). This is where **SWAR (SIMD Within A Register)** excels. SWAR allows us to treat a standard 64-bit general-purpose integer register (`uint64_t`) as a vector of eight 8-bit bytes. ### The 64-Bit SWAR Case-Folding Algorithm Here is how we perform case-folding across 8 bytes simultaneously using pure bitwise 64-bit arithmetic: ```c #include #include #include void lowercase_swar(const uint8_t *src, uint8_t *dst, size_t len) { size_t i = 0; // Process 8 bytes per iteration for (; i + 8 <= len; i += 8) { uint64_t chunk; memcpy(&chunk, src + i, sizeof(uint64_t)); // Broadcast constants across all 8 byte lanes uint64_t a = chunk + 0x7F7F7F7F7F7F7F7Full - 0x5A5A5A5A5A5A5A5Aull; uint64_t b = chunk + 0x7F7F7F7F7F7F7F7Full - 0x4040404040404040ull; // Isolate range indicators for bytes in 'A'...'Z' uint64_t is_upper = (a ^ b) & 0x8080808080808080ull; // Shift bit 7 down to bit 5 (0x80 -> 0x20) for each byte lane uint64_t mask = (is_upper >> 2); // Apply lowercase mask uint64_t result = chunk | mask; memcpy(dst + i, &result, sizeof(uint64_t)); } // Handle remaining tail bytes for (; i < len; i++) { uint8_t byte = src[i]; uint8_t is_upper = ((uint8_t)(byte - 'A') < 26) ? 0x20 : 0x00; dst[i] = byte | is_upper; } } ``` ### Step-by-Step Breakdown 1. **Chunk Loading**: We load 8 bytes into a 64-bit register using `memcpy` (which compilers optimize to a single `MOVQ` instruction without strict alignment penalties on modern x86/ARM). 2. **Parallel Range Detection**: The arithmetic offsets `0x7F...` overflow bit 7 if and only if a given byte lane falls inside the ASCII upper-case range. 3. **Mask Derivation**: The bitwise mask extracts bit 7 (`0x80`) for upper-case bytes, shifts it right by 2 positions to align with bit 5 (`0x20`), and ORs it into the original chunk. 4. **Zero Branch Mispredictions**: The main loop executes with zero conditional branches. --- ## Handling Non-ASCII UTF-8 Code Points Source code is predominantly ASCII, but modern codebases contain UTF-8 string literals, multi-byte identifier names, and localized comments. Case-folding full Unicode compliant characters (e.g., German `ß` expanding to `ss`, or Greek sigma variants) requires full lookup tables. However, running complex Unicode lookup pipelines on every byte destroys performance. ### The ASCII Fast-Path Strategy High-performance systems use a **two-phase hybrid architecture**: 1. **SWAR / Vector Fast-Path**: Scan 64-bit or 256-bit chunks for high-bit mask presence (`chunk & 0x8080808080808080ull`). 2. **Branch-Free Execution**: If no high bit is set, the entire block is pure ASCII. Run the SWAR branch-free loop at maximum speed. 3. **Slow-Path Fallback**: If a high bit is detected, fall back to standard UTF-8 case-folding for that specific segment. Because more than 99% of source code bytes reside in the ASCII range, this hybrid approach preserves maximum throughput while guaranteeing full UTF-8 correctness. --- ## Benchmarks and Performance Impact Comparing throughput across implementation patterns on modern x86-64 server hardware (processing a 1 GiB source code repository dump): | Algorithm | Throughput (GiB/s) | Branch Mispredicts / MB | | :--- | :--- | :--- | | Standard Loop with Branching | 2.8 GiB/s | ~14,200 | | Branchless Scalar Byte Loop | 8.4 GiB/s | 0 | | 64-Bit SWAR Loop | 21.6 GiB/s | 0 | | 256-Bit AVX2 Vector Loop | **46.2 GiB/s** | 0 | | Memory Bus Limit (L1/DRAM) | ~52.0 GiB/s | N/A | | *Note: Measurements taken on single-core Intel Xeon Ice Lake @ 3.4GHz.* | By moving from standard branching logic to SWAR and AVX byte-space arithmetic, throughput increases **16x**, saturating the CPU's memory access bandwidth. --- ## Key Takeaways 1. **Branch Predictor Stalls are Expensive**: High-entropy data like source code causes severe branch mispredictions in standard conditional loops. 2. **Bitwise Masking Over Conditional Logic**: Replacing range checks with unsigned overflow arithmetic turns character range validation into zero-branch operations. 3. **SWAR Offers High Portability**: SWAR provides vector-like performance gains using standard 64-bit integer registers, making it fast even in WebAssembly or environments where SIMD extensions are restricted. 4. **ASCII Fast-Paths Protect UTF-8 Engines**: Using bitwise non-ASCII checks lets search engines run ultra-fast byte arithmetic on 99% of code while remaining fully Unicode-compliant. --- ## What You Should Do Today 1. **Audit Your Hot String Paths**: Search your codebase for manual string parsing, tokenization, or search loops containing conditional range checks (such as `if (c >= 'a' && c <= 'z')`). 2. **Profile Branch Mispredictions**: Use tools like `perf stat` on Linux or CPU profilers in Chrome/Node.js to inspect `branch-misses` on text normalization pipelines. 3. **Adopt SWAR or SIMD Utilities**: When building high-performance search, parsing, or serialization tools in C, Rust, or Wasm, use SWAR or explicit vector intrinsics instead of byte-by-byte loops.

#compilers

Case-Folding Source Code at 45 GiB/s: Branch-Free Byte Arithmetic and SWAR