Vector search benchmarks

Same data, and the same index settings wherever there is an index. SQLite 3.53 and PostgreSQL 17 with pgvector 0.8.6, on an Apple M-series, 100 000 rows × 128 dimensions with clustered embeddings. Every figure is the median of three runs.

make pgvector-up      # start PostgreSQL + pgvector in Docker
make compare          # without it, that arm is skipped

Method.

• All three follow the same workflow: bulk load first, index after. That is already the recommended order for SQLite and PostgreSQL; in fenecdb create index exists for exactly this.
• SQLite: WAL with synchronous=NORMAL, the bulk write in one transaction, the index built afterwards. fenecdb does not fsync per write.
• PostgreSQL: in Docker, bulk load with COPY, then hnsw (m=16, ef_construction=200) — the same parameters as fenecdb — with max_parallel_maintenance_workers set so the build gets one process per core, as fenecdb's does: seven workers and the leader here. At query time hnsw.ef_search = 100, fenecdb's default.
• fenecdb and SQLite compute vector distance with the same code: SQLite's rows are read back and scored by fenecdb's kernel, so what separates the two is the cost of fetching the data, not the arithmetic. PostgreSQL computes it inside pgvector.
• SQLite's core has no ANN index, so a vector search there is a full scan. The comparison is therefore exact against exact first, with ANN on a separate row.
• The write is split in two stages, rows and index. A single "total" would weigh an engine that builds an ANN index and one that does not on the same scale.
• The PostgreSQL figures include a 0.41 ms TCP round trip, about 40% of its ANN time. That is not a measurement flaw, it is the client-server architecture.

How to read the numbers.

fenecdb leads on raw write speed: 936 k rows/s, more than twice SQLite's. The append-only segment store reduces a write to appending to an arena — no B-tree rebalancing, no page splits.

fenecdb builds the index faster: 3.4 s against pgvector's 8.6 s, at the same m and ef_construction on the same cores. It was the other way round, 10.2 s against 8.7, while fenecdb linked each batch of new nodes on one core: pruning the neighbour lists they joined was 70% of its build, and the lists are now pruned in parallel, grouped by list, into the same graph. On aarch64 the distances are written out in NEON as well, a walk measures its neighbours four at a time, to the same bits, and it asks for the next neighbour list and the neighbours' vectors before it reads them, so that one cache miss is not waited out before the next. Before that, this page said fenecdb was faster, 10.2 s against 14.2 s — pgvector was then building with its default two workers while fenecdb used every core. SQLite's 0.03 s is not an ANN index but a single-column B-tree; that cost does not vanish, it is deferred to query time, where a vector search takes 30.2 ms — 330× slower than fenecdb's ANN.

The clearest loss is the open. SQLite opens in 0.07 ms and reads a row by id 0.46 ms after it, because it loads nothing until it is asked and then reads the pages it needs. fenecdb restores its graph as it opens: it copies the 51 MB of vectors out of the file into memory as unit vectors, decodes the graph's links and validates the graph against the documents, 6.0 ms, more than half of it the copy. That is the other side of the same coin — it is why queries are 10–330× faster. Asked the same first question, fenecdb is ahead: a vector top-10 right after the open takes 6.1 ms in all, where SQLite's scan takes 30.5. Each open figure is the median of 31 opens, the file's pages in the page cache as a close leaves them. This page once gave 9 ms against 0.9, from one open each: fenecdb's open and a vector query against SQLite's open and a count through its index, two different first questions.

fenecdb leads on disk as well, 2.4× smaller than pgvector — though not for the reason this page used to give. Of pgvector's 145.0 MB, 83.2 MB is the HNSW index, which keeps its own copy of every vector so a search reads no heap tuple. The heap is 58.7 MB, as large as fenecdb's whole file with its graph, and its MVCC headers are about 2.8 MB of that. fenecdb's graph stores links and nothing else; the vectors are read from the one arena. The WAL is in neither figure.

The table above measures fenecdb in process. Here one client program asks both servers the same questions: fenec-server through a kept-alive HTTP/1.1 connection a client thread, POST /query with the vector as a parameter, and PostgreSQL through the postgres crate with pgvector-rust's types, a statement prepared once. At the million, fenec-server loads and indexes in 49.6 s against 108.5, takes 612 MB of disk against 1 432, and answers a beam of 100 at 99.1% recall in 0.141 ms against 98.6% in 2.08 ms; eight clients get 18 852 queries a second through, against 2 289. Its memory is what its engine counts, 738 MB; macOS puts the process's physical footprint at 992 MB.

make pgvector-up
make scale-bench                                   # 1 000 000 x 128
make scale-bench SCALE_ROWS=250000 SCALE_DIM=768

The vectors are quant's: 64 centres spread along 32 directions, as embeddings vary along far fewer directions than they have, 1 000 held-out queries, and recall against the exact ten found by brute force. Each row also carries a tag from a hash of its own, which a filter keeps 1% or 25% of the rows by, with an index on it in both. Both HNSW indexes take m 16 and ef_construction 64, pgvector's defaults.

Each side gets its best way. fenec-server keeps its index as the rows land, as JSON arrays of rows, a POST /items of about 13 MB at a time (10 000 rows of 128 dimensions, 1 666 of 768), the next one written while the last is sent. pgvector builds after a binary COPY, as it advises, in memory and on every core of Docker's virtual machine (4 GB, 8 CPUs; its parallel_workers is set, since 768-dim vectors live in TOAST and PostgreSQL planned one process for them), reads its index into its buffers before the queries (pg_prewarm), and searches a filter with hnsw.iterative_scan = relaxed_order. PostgreSQL answers from inside that virtual machine, so an empty query's round trip is 0.13 to 0.22 ms against fenec-server's 0.06 to 0.11 ms on the host: it accounts for part of each answer, not for the rest.

Each server was run on its own after four idle minutes, on an Apple M1 MacBook Air: fanless, it slows under minutes of load on every core, and a run straight after another took up to three times as long. pgvector's recall moves from build to build, since its workers insert in parallel: 95.3 to 98.3% at a beam of 100 over four builds of the million. The greedy descent through its upper layers leaves some queries in another cluster than their own, and 3 to 23 of the 1 000 found none of their ten. A million 768-dim vectors does not fit this machine: pgvector's build alone needs about 4 GB.

The memory a server holds is what the kernel cannot take back without swapping it out, on Linux for both: PostgreSQL's is its container's anonymous memory and the shared memory of its buffers, from the container's cgroup — mostly the buffers, 1 GB as configured here and full once the index is read into them — measured in a container started afresh for each run, since the buffers keep what an earlier run left. fenec-server's is its vector arena, its graph and its indexes, and at most 16 MB of documents written since it last handed them over to its file (The file is mapped): what its engine counts (fenec_memory_bytes). On a Mac its physical footprint is larger, since macOS's allocator keeps freed blocks in a cache of its own. The pages of their files each keeps besides are clean, and the kernel takes them back as it needs the memory: the pages of fenec-server's mapped file that the queries read, and PostgreSQL's page cache of its table and index.

What a load holds at its peak is another matter, and a block of every row at once shows it. A block holds its writes until it lands, and it used to hold them twice, in its record and in the stores: 250 000 768-dim rows in one block peaked at 2 389 MB of anonymous memory against the 820 they left, measured in process in a Linux container. A block now spills into the file every 16 MB before it lands (File format), and the same load peaks at 904 MB; in blocks of 50 000 rows, 888 against 1 144.

Apple M-series, single thread, --release. 100 000 documents × 128 dimensions, clustered embedding distribution.

make bench
cargo run --release -p fenec-core --example bench

Without a persisted graph, reopening the same data takes 3.4 seconds — the index is rebuilt from scratch, which would make a page refresh in the browser unusable. The graph record makes that roughly 340× faster and costs 12% more space in the image, 52.7 MB to 58.9 MB; the index of the records, which a mapped open reads rather than walking them, is 0.4 MB more.

A filter that no row near the query matches leaves the walk with nothing to keep, and the search then measures every row the filter does match: the answer is exact, and the set is read on every core, the nearest ten kept as they are measured.

The same measurement at 1 000 000 × 128, one run, and the full vectors of make quant-bench QUANT_ROWS=1000000 at 1 000 000 × 768, stored as f16 and spread along 32 directions, as embeddings are:

cargo run --release -p fenec-core --example bench -- 1000000 128
cargo run --release -p fenec-core --example quant -- 1000000 768 none --rank 32

A build grows faster than the data. Each vector's walk reads more of a graph that no longer fits in the caches: a node cost 0.27 ms of the cores at 100 000 and 0.63 at a million, with about six of the eight kept busy. bench spreads each vector in every dimension, which no embedding does, and at a million its recall at the default beam falls to 93%; the 768-dim vectors spread along 32 directions keep 97.4%, and a beam of 200 buys another point.

cargo run --release -p fenec-core --example bench -- 100000 128 --ef 200
make sweep      # walks the trade-off on both distributions

The default is 100: recall is complete and it is still roughly 13× faster than the engines compared above. It is tunable per query — get docs near embed $1 ef 200 limit 10 — and exact performs a full scan when you want ground truth.

On uniformly random vectors — the pathological case for any ANN index, where in 128 dimensions all distances converge and no neighbourhood structure survives — recall at the same ef is around 82%. Real embedding models output clustered data. --uniform measures that worst case deliberately.

Build-side tuning. 100 000 × 128, single-threaded measurement:

If write speed matters more than the last point of recall, @hnsw(cosine, ef_construction=100) gains 19%.

Same data, 100 000 × 128, clustered. The trade is explained in How it works.

cargo run --release -p fenec-core --example bench -- 100000 128 --f16