Concurrency and request benchmarks

fenecdb and SQLite both take one writer at a time. What that costs when many want to write, and what readers wait meanwhile, measured in one process: fenecdb's threads share one database behind a lock, as fenec-server does; SQLite's hold a connection each, WAL, with a 30 s busy timeout. A row a write, a row by id a read, one run on an M1 after idle minutes. Durable is an fsync per commit, F_FULLFSYNC for both (Rust's sync_data on macOS; PRAGMA fullfsync for SQLite, synchronous=FULL); without one, fenecdb's writes stay in its buffer and SQLite's reach the operating system (synchronous=NORMAL).

make concurrency-bench

Durable writes scale with the writers. fenecdb's fsync runs after its lock is released, and a write an earlier fsync already covered waits for none, so sixteen writers share their fsyncs; SQLite syncs a commit while it holds the write lock. Without an fsync, one fenecdb writer is twelve times SQLite's, and handing the lock between sixteen costs more than half of that. A write, or a /batch of them, holds the lock only while it runs — microseconds for a row — and there is no transaction held open across requests to keep the readers or the other writers waiting. A block of 1 000 writes — a /batch's — held it while it ran, about half a millisecond, and readers waited for it, where SQLite's WAL reads the last commit meanwhile: 213 k reads a second, the longest wait 4.8 ms. Most of that half millisecond had nothing to do with the database as the lock holds it — each row built from the statement and its parameters, its values made the fields' types, encoded, and freed once written — and a block's rows are now built and encoded under the shared lock, beside the readers, the write lock held only to land them, about 0.15 ms; whatever the database decides, the ids, a value already held, the indexes, is decided under the lock as before, and the file is the same byte for byte. A REST insert of 1 000 rows read its body under the write lock as well: a read beside them waited 0.4 ms at the median for rows of a text and an int, 3.3 ms with a 128-dim vector each, where the body is now read with no lock held and its rows built beside the readers, 0.08 and 0.2 ms. The native library builds an app's write of many rows the same way, and a read beside writes of 1 000 128-dim rows waits 0.3 ms rather than 0.9. The rest of the longest waits were the disk: on macOS a write that takes a file's dirty pages past each 8 MB waits for it, 1.5 to 11 ms, and the buffer was written under the lock as it filled. It is written once the lock is let go now, and the handover — the documents written since the open pointed at their places in the file — runs in pieces of 16 384 writes, 0.3 ms each, as the next block starts, with nothing left to write. Until it also ran by count the longest wait was 17.4 ms: it ran once the documents reached 16 MB, which for rows this small was 930 000 of them.

A long read is pinned. A read took the shared lock for as long as it ran, and a write waited for every read, so an aggregate over a million events held every write — and every read by id behind a waiting write — for its whole 0.8 s. Now a long read takes the shared lock only to pin what it reads: each collection's id index copied, 12 bytes a row, its stored documents shared, 0.26 to 0.87 ms for the million rows. It then reads with no lock held, on the database as it stood at the pin, and answers what the read under the lock would have; alone it takes what it took, 151.9 ms against 151.4 for the batch. The writes beside it waited at most what they wait with no long read at all, a few milliseconds of handovers and the copy of the segment the pin shares. A read an index answers is short, and still reads under the lock: indexes change in place, and copying one would cost the read it spares. The same over HTTP, a ledger's reconciliation every second beside four clients sending transfers (make recon-bench), four runs each in turns: the transfers' longest wait went from 187–512 ms, 36 of them past 50 ms in every run, to 5.4–29 ms and none in three runs; the fourth met one stall of 385 ms, four transfers past 50 ms, of the kind both servers met in the round with no reconciliation at all (118–223 ms). Those had two causes, both gone: in three runs since, the transfers' longest wait was 1.9–10.4 ms, none past 50.

A write waits for no index's growth and no fsync. A @unique index holds a key a row, and its table grew whole: the write that found it full moved every key into one twice its size, under the write lock. The ledger's journal passing 1.8 million entries held one transfer, and every request behind it, 120 to 137 ms in every run; four million puts one at a time waited 7, 19, 49, 109 and 235–267 ms at each doubling from 229 376 keys (make growth-bench). Past 114 688 keys the table is now 256 that grow on their own, and the longest put is 2.5 to 3.6 ms. Each key keeps its hash beside it, so a lookup hashes it once — 83 to 93 ns over four million keys against 121 to 144 in the shards before, 24 to 34 under that size against 33 to 42 — and a table grows without hashing a key again, for 8 bytes a key. A text index's terms grew the same way, 276 to 324 ms at 3.7 million terms, and are 256 tables now too, the longest put 5 ms. And a vector index's table of the vectors it holds, which finds one written again, was made again from every vector once half full: 242 to 281 ms at a million vectors of 128 dimensions, 1.45 s at 524 288 of 768. It grows from what it holds now, a 256th at a time, and no put waits for it. The first write after an open made it by reading every vector once — 275 to 441 ms over 1.1 million of 128 dimensions, 33 to 169 on every core; it makes it now from four bytes a vector the file keeps beside the graph, and that first write took 16 to 31 ms there, 4.4 to 27 over 300 000 of 768 against 68 to 312, the open as fast as it was. And the two writes still made under the lock — the 1 MB buffer's once it is full, and a handover's — waited for the disk while the server's periodic fsync held it, 52 ms of macOS's F_FULLFSYNC; they are put off until it is done now. The rest of those first-round stalls was the bench's own: --warm built an index its warm-up had not read just as the round began, and it reads that one too now.

What one request costs a client of fenec-server, on loopback: a row by id, a filter with a limit, a near for the top ten and a put, over 10 000 rows of 128 dimensions, asked by one client at a time — the round trip's p50 — and by eight at once. Each is POST /query with the parameters beside the text, on a kept-alive connection. One run on an M1, after idle minutes.

make pgvector-up      # optional: PostgreSQL's arm, skipped without it
make requests-bench

Most of a round trip is the loopback. For a row by id the server's own work is a few microseconds: one read and one write on the socket, and the statement. POST /query keeps the statement a text parsed to, so it does not read its text again for each request, and an answer goes out in one write.

PostgreSQL with pgvector, asked the same in SQL, answers a row by id in 0.142 ms and a near in 0.653 ms, from inside Docker's virtual machine on the same Mac, where its empty query alone takes 0.127 ms. The network, not the engine, is most of that difference, and the bench prints the empty query beside the rest so that it reads that way.

What loading 100 000 rows of a text, an int and a 128-dim vector costs, each way a client can send them, into an empty collection: with the vector's HNSW graph kept as the rows land, and with no vector index. fenec-server on loopback, 1 000 rows a request — POST /docs with a JSON array; a put of 1 000 sent to POST /query; a POST /batch of single puts. PostgreSQL with pgvector is sent the same rows by COPY and by INSERTs of 1 000, with synchronous_commit = off, as fenec-server's writes reach the disk within --sync 250. One run on an M1, after idle minutes.

make pgvector-up      # optional: PostgreSQL's arm, skipped without it
make load-bench

Without a vector index, the vectors' text is the cost. Each row's vector is 128 numbers written out, read straight into its f32s wherever it comes from — a JSON array, a parameter, a FenecQL literal — in one pass that finds each number's end. In process the rows arrive parsed, which is what the rest can at most reach. PostgreSQL answers from inside Docker's virtual machine on the same Mac, whose network carries every byte.

With the graph kept, the graph is the cost. Every way lands close to linking the vectors in process. A /batch's vectors are linked together, 512 at a time on every core, whether they came a row a statement or 1 000 a put: linked a row at a time, as they were before, the same rows went in at 5 400 rows/s. PostgreSQL's HNSW index takes each row on its own, on one core.