YCSB benchmarks

YCSB's core workloads, written in the bench crate (crates/fenec-bench/src/bin/ycsb.rs) to YCSB's own definitions rather than run through its Java client: a record is a key and ten fields of 100 random characters, a read reads the whole record, an update writes one field, keys are drawn by YCSB's scrambled zipfian (0.99, hashed over ten billion items) or, for D, its skewed-latest, and a scan reads 1 to 100 records from a key. The key is an integer from 1 (insertorder=ordered without the user prefix), so each system keys by what it keys best by: fenecdb's id, SQLite's INTEGER PRIMARY KEY, a bigint primary key, MongoDB's _id; E's scan is get usertable where id >= $1 limit n, which walks the id index from the key. A read never asks for a key whose insert has not been answered. The sequence is YCSB's: a load, then A, B, C, F and D over that data, then a load afresh for E, at 1, 4 and 16 threads, each cell 30 s after a 5 s warm-up. Latencies go into a log-linear histogram of 64 steps an octave.

The systems. fenecdb 0.1.9 in process, reads under the shared lock and writes under the exclusive one as the server takes them, against SQLite 3.53.2 through rusqlite, a connection a thread, WAL, the file mapped. fenec-server natively, each client thread a kept-alive HTTP connection sending POST /query with parameters, against PostgreSQL 17 (shared_buffers 1 GB, prepared statements) and MongoDB 8 (the official Rust driver), each in a container started for its turn and removed with its volume after, so the virtual machine (8 cores, 3.8 GB) holds one at a time. fenec-server in Docker is the same server in a container of its own image. Durable is an fsync a write in each engine's own terms: a flush and its fsync (F_FULLFSYNC on macOS) and --sync always, synchronous=FULL with fullfsync, synchronous_commit=on, j: true; buffered is each one's default: a flush every 250 ms (--sync 250), synchronous=NORMAL, synchronous_commit=off, w: 1, j: false. C writes nothing and runs once, buffered. Docker's fsync is the Linux VM's, which does not flush the Mac's disk as F_FULLFSYNC does: a 1 KB append and its F_FULLFSYNC took 3.9 ms on the Mac and its plain fsync 0.023 ms, while in the VM, on a volume as the databases' data, pg_test_fsync gave 0.076 ms for fdatasync and 0.103 for fsync (crates/fenec-bench/ycsb/fsync.txt). So the server is measured in Docker as well, at 1, 4 and 16 threads, durable and buffered, beside it on the Mac.

Why fenecdb's reads slowed down after the updates, and no longer do. An update writes the whole record again at the file's end, and the version it replaces stays dead until a compact. The first three runs (r1-r3) were measured before anything compacted on its own: after A's and B's updates the buffered file had grown to 4.7 to 6.5 GB on this 8 GB machine, against SQLite's 2.1 to 2.5 GB updated in place, and a profile of C at one thread (crates/fenec-bench/ycsb/profile-c1.txt) put 86% of the reading thread's time on the first byte of the record in the mapped file, waiting for its page: about one read in five went to the disk. C at one thread read 30.1 k a second, where the same cell over a file just loaded read 443 k. fenecdb now compacts a file beside its queries once half of it is dead (Compaction), in an app's process as in the server, and the harness runs it as an app would get it: the in-process database with the engine's Compactor, the server with its default --auto-compact. Runs c1-c3 measured fenecdb and fenec-server again with it; the buffered file stood at 3.7 to 4.0 GB after D, mid-way between compacts of a file A's updates grow by 100 MB a second, and C at one thread read 438 k. The earlier runs of fenecdb and fenec-server stay in the results, marked superseded (crates/fenec-bench/ycsb/superseded.tsv); SQLite was not measured again, and the servers were measured again since, all of them on one day (below).

And once the file nears what the page cache can hold. Since these runs a file is also compacted as it nears the memory the system has for its pages, or once the process waits for the disk (Compaction); the cells above predate it. Measured apart on 2026-10-11 and 12, in process, buffered, A, B and F at 1 and 16 threads, the build before it and the one with it in turns, its cells in none of the medians: with the machine's memory as it stood, two runs each, A at 16 threads went from 82 and 92 k to 133 k twice and the rest stayed within the runs' spread; beside 1 GB of memory held by another program, one run each, B went from 23.5 k to 63.5 k at one thread and from 62.3 k to 112.8 k at 16, F from 23.6 k to 48.5 k at 16, the file at its largest 1.2 to 1.5 GB against 1.5 to 2.0, and A at one thread from 29.7 k to 24.1 k.

The one-client server cells, measured again the same day. On 2026-10-04 fenec-server natively and in Docker (A to F at one thread, durable and buffered, A and C at 16) and PostgreSQL beside them, in turns with three idle minutes between, three times, after the sync log and the rendering changes of the round trip: each cell measured again counts as its run's last line, under the runs' ids, c1-c3 for the servers and r1-r3 for PostgreSQL, so every cell compared at one client is of one day. Comparing them against PostgreSQL's cells of days before had put the servers behind where Docker's network had been quicker then: its C at one client was 3.34 k, 3.53 k this time, and 3.00 k in a check in between. The --verify pass over all 84 cells found no mismatch, and nor did a shorter one after the sync log's fixes (crates/fenec-bench/ycsb/verify.tsv, runs c and fix).

Again after the read path's changes. On 2026-10-05, with fenec-server writing a plain get's rows as JSON straight from the stored documents (the round trip), fenec-server natively and in Docker and PostgreSQL beside them measured B, C, D and E at 1 and 16 clients, durable and buffered, in turns, three times, under the same ids: each cell measured again counts as its run's last line, so these replace those cells of the day before, PostgreSQL's too, and A and F stay as they were. The first of the three ran while other work on the machine still held a core, its idle probe taken at a load of five (65.0 ms against 62.5 idle), and its cells came in up to a quarter under the other two, PostgreSQL's as much as the servers'; it was measured again on the quiet machine, and those lines are its cells. A --verify pass over fenecdb, fenec-server and fenec-server in Docker, A to F at 1 and 16 threads, durable and buffered, 8 s a cell, checked 42.1 million reads, 1.8 million scans of 89.2 million rows and 6.1 million writes and found no mismatch (verify.tsv, run gap).

Every server on one day. On the night of 2026-10-13, fenec-server natively and in Docker, PostgreSQL, MongoDB and Redis measured every cell, A to F at 1, 4 and 16 clients, durable and buffered, three times, in turns with three idle minutes between (runs rx1-rx3, 8.5 hours). They replace the cells of the two paragraphs above, the servers' of c1-c3 and PostgreSQL's and MongoDB's of r1-r3, so every cell compared over the wire is of one day; fenecdb's and SQLite's in process stay as they were. A first attempt (rd1-rd3) is in the results and in no median: it read the containers' file sizes beside the cells, a container started for each look ten times a second, and the VM's work took fenec-server in Docker's C at one client over 100 000 records from 3.14-3.17 k to 2.33-2.38 k, in turns. A size read through a container is read once the clients stop now.

The machine. A fanless M1 MacBook Air, 8 GB, macOS, which slows to a third under minutes of load on every core: each cell starts once a one-core probe runs within 4% of its idle time, the probe's ratio written beside the cell, and the systems take turns with three idle minutes between them.

make ycsb YCSB_ARGS="--runs 3 --out ycsb.tsv"
./target/release/ycsb report ycsb.tsv

The answers are checked. --verify logs every operation and what it was answered, and after each cell holds each one to what was written: a read must answer the record it asked for, each field holding the value of a write that could have been the last one before it (one that ended before the read began and no other ended after it, or one that overlapped the read); a scan must answer consecutive keys from its own, as many as it asked for wherever they existed, each record as written; a write must report one record written. Every write is logged, not a sample, because judging a read under concurrent writes takes every write's span; the log is a push into each thread's own list and the check runs once the cell is over, so nothing is compared in an operation's time. The same check holds every system. A pass over all six — 100 000 records, 10 s a cell, workloads A, E and F at 1 and 16 threads, durable and buffered — checked 13.8 million reads, 2.7 million scans of 134.7 million rows and 9.8 million writes and found no mismatch (crates/fenec-bench/ycsb/verify.tsv); its timings are not published.

Held to YCSB itself. Yahoo's YCSB 0.17.0 — its Java client, the jdbc and mongodb bindings, PostgreSQL's JDBC driver 42.7.4 — and this harness ran the same cells against the same containers: workloads A and C, 1 and 16 threads, 100 000 records, durable, each 30 s (YCSB's maxexecutiontime, which counts its start; the harness warms up 5 s first). Operations a second and the p99 of the slowest operation:

The harness lands between 14% under and 2% over YCSB's own client, six of the eight cells within 9%; where it differs by more than 2% it measures PostgreSQL and MongoDB a little slower than YCSB does, not faster. The p99s agree to a few tenths of a millisecond but for MongoDB's C at 16 threads, 2.54 ms here against YCSB's 4.18. YCSB keys by strings (user…) where the harness keys by integers, and each side drives its own client library. The outputs and the script are crates/fenec-bench/ycsb/calibration/.

The full grid, every cell at 1, 4 and 16 threads with each operation's p50 and p99, is below; the runs themselves are crates/fenec-bench/ycsb/results.tsv.