Vector database · C++20
Epzero
Faster than FAISS. Written from scratch. A C++20 vector database with hand-written SIMD, lock-free reads and crash-safe replication. It beats hnswlib and FAISS at equal recall, and it runs in your browser tab.
- 1.08×
- hnswlib’s queries per second at recall 0.95, on SIFT1M
- 1.12×
- FAISS’s throughput at the same recall
- 175 µs
- p50 latency at recall 0.974, one thread
- 10×
- scaling on the 12 cores of one socket
The problem
Vector search sits under semantic search, recommendations and RAG, and most engineers only ever call it through a library. Epzero builds every layer itself: the graph, the distance kernels, the concurrency, the storage engine, replication and a browser build.
It also targets a production gap. Approximate indexes trade accuracy for speed, and in production almost nobody measures that accuracy. It drifts with deletes, filters and new queries, and a RAG system then answers from the wrong documents without anyone noticing.
Epzero’s recall guard re-answers a small sample of real queries exactly, reports live recall with a confidence interval, and adjusts search effort to hold a target. It can also explain why a given document was not retrieved.
One engine, three targets
- 01
C++20 engine
HNSW with lock-free search, SQ8 quantization, and a database layer with a write-ahead log and snapshots.
- 02
Distance kernels
AVX2, AVX-512, NEON and WASM SIMD128, each compiled separately and selected at runtime.
- 03
Native
Python bindings, benchmarks and a gRPC data node.
- 04
Cluster
A Go router over C++ nodes handles sharding, replication and failover.
- 05
Browser
The same engine in WebAssembly, with CLIP text and image encoders running in the tab.
Results
SIFT1M, one search thread
| ef | recall@10 | QPS | p50 | p99 | vs brute force |
|---|---|---|---|---|---|
| 10 | 0.7052 | 28,541 | 34 µs | 59 µs | 1,175× |
| 40 | 0.9266 | 10,520 | 96 µs | 132 µs | 433× |
| 80 | 0.9743 | 5,912 | 175 µs | 220 µs | 243× |
| 200 | 0.9954 | 2,651 | 392 µs | 485 µs | 109× |
| 400 | 0.9985 | 1,432 | 727 µs | 911 µs | 59× |
1M × 128-D, M=16, efConstruction=200, median of 3 runs on a dedicated Xeon Gold 6226 node. Runs agree within 0.5%.
Against the field
| Comparison | Result |
|---|---|
| QPS at recall 0.95, vs hnswlib | 1.08× |
| QPS at recall 0.95, vs FAISS | 1.12× |
| 1M build time, Epzero / hnswlib / FAISS | 24.0 / 27.8 / 37.8 s (fastest build) |
| GIST1M, DEEP10M in 8-bit mode | Fastest of all tested, 1.08× and 1.15× Glass |
| GloVe-100, DBpedia OpenAI embeddings | Second to Glass, by 1.18× and 1.31× |
Same algorithm, same graph quality, every library on huge pages. Build time is for 1M vectors on 24 threads.
Under load and over time
| Scenario | Result |
|---|---|
| 4,049 writes/s against concurrent readers | Reader throughput −9%, p99 236 → 262 µs |
| All 24 threads, one index copy per socket | 109,186 QPS |
| Recall guard under drift, SIFT1M | Held 0.95, where a fixed setting fell to 0.948 |
| Recall guard vs over-provisioning | 6–15% lower cost, estimates within 0.011 |
| Filtered search at recall 0.95 | 39k QPS at 0.1% allowed, 5k QPS at 50% |
Key decisions
Search is a memory problem
Profiling showed that search is bound by memory latency, so the work targeted the memory system: 2 MiB huge pages, each vector stored beside its neighbour list, and prefetching so that up to 32 cache misses overlap instead of running in sequence.
An epoch-tagged visited set
An 8-bit tagged array takes 1 MiB for a million nodes, which fits in L2. It resets without clearing and allocates nothing per query.
Lock-free reads, per-node writes
Neighbour slots are atomics with release stores and acquire loads, so a reader that sees a node id also sees its vector. A thread never holds two node locks, which rules out lock-order deadlocks.
Filtering picks its own strategy
Each query chooses between an exact scan, a graph walk that skips excluded nodes, and a two-hop walk across them, using a cost model calibrated on a dedicated server.
Rejected after measurement
Graph reordering, early abandoning, LTO/PGO and in-memory RaBitQ were each measured, didn’t help, and were removed. Every optimisation had to win an interleaved A/B test on SIFT1M.
What broke
Four crash bugs in the index loaders
libFuzzer runs on every parser of untrusted bytes in CI. All four bugs were fixed and kept as regression cases.
Two replication bugs
Found by a deterministic simulation that checks strong reads for linearizability, with Porcupine, under crashes, partitions and lossy links.
The write-ahead log under fault injection
It survives a torn write at every byte offset, mid-log corruption and random kill -9s. ThreadSanitizer checks the engine with 8 writers and a concurrent reader.
What these numbers don’t prove
- Where huge pages are opt-in, Epzero requests them and other libraries don’t. The lead then grows to 1.46×, but the headline figure gives every library huge pages.
- Glass leads on GloVe-100 and DBpedia.
- The cluster is built and tested in CI with a real three-node failover. Its cloud deployment is on hold.