Size and memory benchmarks

What the browser downloads, how large each build is, what each feature adds to the module, and how the memory fenecdb counts compares with what the process uses.

Memory

How the memory fenecdb counts for itself compares with what the process really uses at its peak. Use it to choose --max-memory.

 measured footprintpeak RSSratio
100 000 × 128128.2 MB172.9 MB74%
200 000 × 457.6 MB97.2 MB59%
  • Set --max-memory to a third of the container's memory.
Sizing --max-memory from the ratio
make memory       # for calibrating --max-memory

The footprint is an early warning rather than a guarantee. A third of the container memory is a sensible --max-memory: it covers both this 60–75% ratio and compact's 3× peak. Details in Limits.

Binary size

How large each build is. For the module and the client together, what the browser pays is 324 KB brotli; for an app whose queries run on a server, the client alone in its bundle.

BuildOn diskBrotligzip
cargo build --release -p fenec-cli, default3013 KB——
--profile cli, default, as a release builds it2400 KB——
--no-default-features, no import, backup or MCP2236 KB——
--profile cli --no-default-features (make small)1880 KB——
wasm32, browser803 KB273 KB330 KB
wasm32, no index or schema check, make wasm FEATURES=language SCHEMA=0584 KB202 KB242 KB
wasm32, no feature at all, make wasm FEATURES=none SCHEMA=0524 KB179 KB214 KB
web/fenec.js and its modules, the client194 KB51 KB60 KB
@fenecdb/web/client in an app's bundle, connect and a query's rows, esbuild, minified29 KB9 KB10 KB
container image, scratch + musl, amd642.99 MB——
Method, and where the module's bytes go

The compressed columns are filled in only where they mean something: the wasm module and the client are fetched over HTTP, so what the browser pays is 324 KB brotli for both together. A binary you execute and an image whose layers are already compressed are not served that way, hence the dashes.

The brotli column is -q 11, and the level matters. That is the right number for a file compressed once at build time and served as a static asset, which is how this site ships the module. A server compressing on the fly does not spend that: the same two files come out at 306 KB at -q 5, nginx's dynamic default, and 325 KB at -q 4. Quoting the best case without naming it would flatter the number by 12–19%.

The on-disk column is not the transfer size and is not trying to be. It is what gets decompressed, compiled and held: WebAssembly.compile and the module's memory footprint track that figure, not the compressed one, and so does reading the thing end to end.

fenec-server stays on the release profile on purpose: a connection thread that panics unwinds and takes down only its own session, and the server stays up. The cli profile turns panic unwinding into abort, which is right for a single process with nothing to recover.

Where the module's bytes go. make size-report builds the module with its function names kept and puts each function's code where it is written: a method with its impl's module, a closure with its function, and an iterator adapter of the standard library — a fold, an extend — with the closure it runs, which it holds inlined. What is left of the standard library is split by what it is for. make size-report BASE=main shows what a change did against main, down to the ten functions that moved most, and every pull request's CI run shows the same against the branch it goes into.

Code, 637.2 KB of the 713.7KBShare
fenecdb's own510.580.1%
the engine, fenec_core::engine202.031.7%
FenecQL's parser and lexer49.77.8%
the vector index41.96.6%
@sorted20.63.2%
the rest196.330.8%
the standard library's126.619.9%
Vec, String, Box46.37.3%
the sort23.03.6%
HashMap12.11.9%
core::fmt4.60.7%
the rest40.66.4%

The data section is another 67.6 KB. Most of the standard library's share is generic code compiled once for each type it is used with, so the module sorts few types and keeps few maps: ids as u64s, scored ids through one comparator that the text index's terms share, rows through one ordering that lookup shares, and a collection's indexes by field in a list rather than a map. That took the sort from eight element types to five and the maps from eleven entry types to seven, 28.5 KB of code; and later the stable sort of ids an in list's buckets went through — merged where their runs end now — and the maps of a database's collections by name and a load's by number, to five, 6.9 KB more. A function inlined into two callers is two copies of itself, and a generic one a copy for each type and closure it is called with: the handles' table each export reached through a closure of its own, build_index and facet inlined twice, the standard library's reading of an integer inlined at each of its three readers, were 8.6 KB, which each took one copy of. Writing a float was 20.8 KB more, Grisu and Dragon behind {}, until the module wrote its own: Ryu's shortest digits, the text {} writes to the byte, in 2.1 KB (num.rs). And 16.5 KB went with a panic's formatting of a char, kept by every slice of a string that could panic, the standard library's to_lowercase, to_uppercase and contains among them: the module slices without, and puts a string in a case through case.rs, character by character, the same text to the byte. The link writes its relocations short (--compress-relocations), 36 KB of padded LEBs, and V8 runs the smaller module faster: near 12%, a page of vectors as JSON 15% (make wasm-speed). make size-report WHY=<pattern> names what pulls a part in.

The browser module, feature by feature

A page that does not search text, needs no graph behind its vector search, or asks no facets, can load a smaller module built without what it does not use. With every feature the module is 244.1 KB brotli; with none, 149.0 KB, 39% less, and a near still answers, every vector measured. The release ships the module with every feature; any other set is a build option.

FeatureAdded to none, brotli KBLeft out of every one, brotli KBOn disk, KB saved
vector, the HNSW graph20.419.061.4
text, match with fuzzy and prefix13.115.243.5
sparse, @inverted2.41.43.9
sorted, @sorted, and composite with it8.723.576.4
composite, an index over several fields, beside sorted11.610.932.8
geo, points, and @geo where sorted is2.56.016.8
nocase2.82.67.1
collate, ICU's order8.98.419.2
facets3.64.512.5
highlight, beside text3.83.610.0
analytics5.55.114.3
the schema check, SCHEMA=0 without9.59.528.2
Method, what stands in for a missing index, and near without the graph

A page loads a module built with what it uses (A smaller module): make wasm FEATURES="text sorted", FEATURES=indexes or language for a group. make wasm-sizes builds the module with every feature (713.8 KB on disk), with none (432.5 KB), with each added to none and with every one but each, and measures each under gzip -9 and brotli -q 11. The two columns differ where features share code: the sort and the map the sparse index ranks through are the other indexes' too, and the ordered index's chunks are the index over several fields' and @geo's, so sorted left out takes both with it.

What stands in for a missing index is a type no value can have, so the compiler drops the index whole, and the engine's paths through it with it; a fold of no value does the same for nocase, and a function or an aggregate a feature brings is refused by name. A module without a feature opens every file: a collection declaring an index it lacks is answered by the scan, a point is read and written, and a statement over a collection whose nocase or collated field it cannot compare is refused, the feature named. Making these optional cost the module with every feature nothing: its code came out 14 bytes smaller, functions moved and closures renumbered, and its compressed sizes within the noise of a build's layout. Left in every module, measured: json fields and their paths, which would save 0.6 KB brotli, and @ttl, 0.7 KB.

Without the graph a near measures every vector, as near ... exact does in the full module, with the same rows and scores: the scan cost the modules without it 2.2 KB brotli. Measured with make wasm-exact-speed in Node, the median of 50 queries, limit 10, over clustered vectors under cosine:

Rows × dimensionsThrough the graphExact, full moduleBuilt without vector
1 000 × 1280.20 ms0.17 ms0.38 ms
10 000 × 1280.22 ms0.49 ms2.4 ms
50 000 × 1280.34 ms1.9 ms11.7 ms
1 000 × 3840.47 ms0.43 ms0.90 ms
10 000 × 3840.51 ms1.1 ms6.2 ms
50 000 × 3840.75 ms3.9 ms29.4 ms

The full module's exact scan reads its arena, the vectors already made unit ones; without the graph each is read out of its document and made one for the query, four lengths summed side by side, which is the rest of the difference: a near 2 to 40 times slower.