Uncompromising Speed.
Absolute Determinism.
A high-frequency order matching engine engineered in Rust. An LMAX-inspired single-threaded core measured at sub-microsecond per-order latency and 4.4M orders/sec on a single machine. Every number comes from a command you can rerun: the ring buffer is model-checked under the C11 memory model, the persistence layer is fault-injected at every byte offset, and the core is ported to C++ for a like-for-like comparison.
End-to-End Latency Profile
Ryzen 7 7735HS · Windows 11 · rustc 1.93.1 · TSC clock, 0.3 ns step · 3 runs · 2026-10-10
Ring pop → match → encode execution report, per order over 990,000 orders (63% fill, 37% rest). The earlier published 100/200/500/800 ns percentiles were quantized by the 100 ns step of std::time::Instant on Windows; the load generator now times with the x86-64 timestamp counter and prints the clock step it observed.
LMAX-Inspired Data Flow
Path to Sub-Microsecond
Established the baseline logic with HashMap and VecDeque. Fixed-point i64 prices were implemented on day one to prevent IEEE 754 floating-point rounding errors.
Replaced collections with a 1M-slot Arena object pool and custom intrusive doubly linked lists. Eliminated all OS malloc contention.
Swapped HashMaps for BTreeMaps. Best-price recomputation went from an O(n) linear scan to an O(log n) tree lookup upon level exhaustion.
Implemented a Disruptor-style SPSC ring buffer for inter-thread communication. Engineered with CachePadded atomics and Acquire/Release memory ordering.
Dropped JSON overhead for custom fixed-size little-endian structs. Integrated TCP ingestion and UDP multicast execution report broadcasting.
Added a memory-mapped Write-Ahead Log (WAL) and bincode snapshots. Ensures bit-exact state recovery in under 1.4 milliseconds after a crash.
Eliminated the final Vec<Fill> allocation via borrowed slices. Verified architecture limits using HdrHistogram and a load generator that times each order with the TSC and prints its own clock resolution.
Model-checked the ring under the C11 memory model with loom, added differential property tests against a naive oracle, ported the ring to C++20 for a like-for-like benchmark, and measured what SeqCst, false sharing, cursor caching and bounds checks each cost.
Fault-injected the WAL and snapshots at every byte offset and killed the writer mid-write, finding two silent-corruption defects. Root-caused the single-thread Rust-vs-C++ gap in LLVM IR, settled the cross-thread comparison with pinned, interleaved statistics, and ran Miri, TSan, ASan, UBSan and libFuzzer in CI.
Systems Engineering
Lock-Free SPSC Pipeline
A Disruptor-style ring buffer bridges ingestion to matching. Acquire/Release cursors compile to plain MOV instructions, verified from the generated assembly; SeqCst costs 5.4x on the same transfer. 3.2x faster than std::sync::mpsc on 8-byte payloads, and 10–17% faster than its own C++20 port on 48-byte orders (p ≤ 0.003, n = 40, pinned cores).
Arena Memory & Intrusive Lists
The order book is backed by a pre-allocated 1M-slot arena. Custom intrusive doubly linked lists bypass OS malloc contention entirely. Cancel complexity reduced from O(n) to O(1) via direct index unlinking.
BTreeMap Best-Price Tracking
Replaced naive HashMaps with BTreeMaps for sorted price levels. Best-price recomputation upon level exhaustion improved from an O(n) linear scan to an O(log n) tree lookup, cutting multi-level sweep latency by 67%.
Cache-Line Optimization
Structs are padded to 64 bytes (#[repr(C, align(64))]) to occupy exactly one x86 cache line. The cost of getting this wrong was measured, not assumed: removing the padding doubles cross-thread transfer time through false sharing alone, without changing a line of logic.
Deterministic Recovery
Load the latest snapshot, replay the WAL, resume. Recovery completes in under 1.4 milliseconds with bit-exact state reconstruction. After every injected fault and process kill, the recovered book equals a fresh replay of the same input.
Data Integrity
Every WAL record carries a CRC32 checksum, and the snapshot checksum covers every field. Corrupt or torn records are truncated to the longest valid prefix, and the bytes beyond it are zeroed so discarded history can never resurface after new appends.
Graceful Back-Pressure
When the matching engine falls behind ingestion rate, the 65,536-slot ring buffer absorbs burst traffic. Inbound orders are delayed via TCP flow control, never dropped.
Verified, Not Asserted
Benchmarks say the engine is fast. Phases 8 and 9 ask whether it is right, what each design decision is worth, and which of the earlier claims survive re-measurement.
Model Checking
The ring compiles against loom, which explores every interleaving the C11 memory model allows. FIFO across wraparound and backpressure and exactly-once drop pass. Downgrading the cursors to Relaxed is caught immediately: Causality violation.
Differential Testing
Random add/cancel sequences run through the engine and a flat-Vec reference book. Status, fills, cancels, best prices and full book contents must agree after every step, and replaying the same input must produce the same transcript.
Crash-Consistency Fault Injection
The WAL and snapshot files are truncated and corrupted at every byte offset with three patterns, then written to again and recovered again. A writer process is killed mid-write for 12 cycles. This found two silent-corruption defects, both fixed.
Sanitizers & Miri
ThreadSanitizer, ASan and UBSan on the C++ ring; Miri with strict provenance and a sanitizer-instrumented std on the Rust ring, in CI. Both Relaxed negative controls are caught on the exact access the acquire/release pair orders.
Coverage-Guided Fuzzing
libFuzzer targets for the wire decoder, the WAL loader and the snapshot loader. First CI run found a CRC-valid record that does not decode; the engine now refuses to start rather than guess, and the input is committed as a seed and a unit test.
C++ Port & Statistics
A line-for-line C++20 port of the ring with the same compile-time knobs. Pinned cores, interleaved ABBA runs, bootstrap confidence intervals and Mann-Whitney tests. Rust is 10–17% faster on 48-byte transfer; C++ is 1.35x faster single-threaded, mostly the bounds check safe Rust cannot elide.
What re-measurement changed
P99 latency
The 100 ns step of std::time::Instant on Windows quantized every percentile. Re-timed with the TSC.
Ring vs std::sync::mpsc
The original gap came from an unpreserved session and never reproduced. The number shown is the one that does.
Unsafe inventory
A recount found blocks in the tracking allocator and ring internals that the README had left out.
Rust vs C++ cross-thread
Two unpinned runs could not resolve the effect. Forty pinned, interleaved runs do, at p ≤ 0.003.
Proven Under Pressure
Order Cancellation
Middle of 1,000 resting orders
Multi-Level Price Sweep
100 ask levels exhausted sequentially
Worst-Case Cancel
1,000 orders across 1,000 distinct prices
Cross-Thread Throughput
SPSC ring vs best std::sync::mpsc, 1M u64, 2026-10-10
Single-Thread Push+Pop
ManuallyDrop<Arc> lets LLVM keep cursors in registers · n = 40, p = 1.4e-14
Acquire/Release vs SeqCst
Cost of over-synchronizing · 1M u64 cross-thread, median of 30
Cache-Line Padding
Cost of false sharing · same transfer, cursors on one line vs padded
Production Horizon
Deliberately out of scope for this project — but engineered with these production realities in mind.
DPDK / AF_XDP
Kernel bypass networking for true wire-to-wire latency elimination
Multi-Instrument Routing
Gateway distributing orders to per-instrument engine instances
Aeron Transport
Reliable UDP multicast with built-in flow control and backpressure
FIX Protocol Gateway
Industry-standard order entry interface for external client connectivity
Hot-Hot Failover
Secondary engine replaying the same WAL for zero-downtime recovery
Power-Loss Durability
fsync/msync ordering for the WAL and snapshots; the fault-injection harness covers process crashes only