Collections and types

An object's keys are kept sorted and unique: a key given twice — in FenecQL, in a JSON body, in a parameter — refuses the write rather than keep one of them, since which one a reader keeps differs from reader to reader. Its numbers are kept as written, an integer an integer and 19.99 the decimal it is, everywhere one comes in: FenecQL, a parameter over HTTP and in the browser, a JSON body, a row fenec import reads. A list of numbers alone is read the quick way, into the f32s of the vector it nearly always is; where it is given a json field, or compared with a path into one, the statement's text or the parameter's JSON is read again as written, so [0.1, 12345678901] is kept as those two numbers. A vector that was never text — a Float32Array handed to the browser module, a vector built through the Rust API — is refused there rather than kept as numbers it was not given. A timestamp becomes its ISO text, as JSON writes one; bytes, a sparse vector and a number that is not finite have no JSON and are refused. A value nests at most 64 levels, and a path names at most 64 keys past its field: past either a write or a query is refused, never cut.

A path is a field's name, a dot, and keys, each a name as a field's is (meta.source.rank); a key with a space or a dash in it cannot be named, and is reached by reading the whole field. It works in every comparison, in, has, ~ and is null, in select — answering under the path's text as its column's name — in order, and in set and put, which set the one key inside the field. A filter binds a path once a query, as it binds a field, and walks its keys over each row's bytes, passing the members before the key without decoding them. An aggregate and group read a field, not a path; a lookup's keys are fields too. Comparisons are the ones every value already has: 3 equals 3.0, every number orders below every text, text below a list and a list below an object, and null is equal only to null.

An index on a path is a field of its own beside the fields, named by the path, and its kind is the field's: @hash keys a value as a typed field's does, a whole float filed as the int it equals so that 3 and 3.0 share a bucket; an equality or an in reaches it with null, a boolean, text and a number an f64 holds exactly, and anything else — an int past 2⁵³, which equals floats near it that no bucket holds — is answered by the scan. @unique refuses a value held twice as a field's does, 3 and 3.0 the same value. @sorted orders the numbers as f64s and the text by its bytes, every number before every text; a value it has no place for — true or false, a list, an object, an int past 2⁵³ — is held apart, and while the index holds one it answers nothing and the scan does: a wrong order is never what an index costs. tests/json.rs holds both to a twin collection without them, every filter, order and page, through writes and a reopen. Text, vector and sparse indexes stay on the fields that declare their type. On a server an index on a path is built under the write lock, as it reads one value a row: 11 ms for @hash and 21 for @sorted over 100 000 documents.

Renaming a json field renames the paths into it, indexes and all, and dropping it drops them. In the file a json field is type tag 14 and an object value tag 13 — not 11 or 12, which a schema writes for a collation and a dropped place — and each index on a path is written inside its field's entry, its keys and its kind: a binary from before them refuses the file (unknown type tag 14) rather than read it. The indexes are derived, as every hash and ordered index is: rebuilt from the documents by the first read after an open.

Measured over 100 000 documents of a title, a year, a price and a meta object of three keys: where meta.lang = "tr" scans in 5.7 ms where the same comparison of a text field takes 4.2; meta.source.rank >= 990 in 7.2; both together in 10.4; the first 20 by meta.source.rank in 7.3. With @hash and @sorted on the two paths: 0.025, 0.009, 0.12 and 0.010 ms. A scan of a collection with no json field takes what it took before them (3.47 against 3.38 ms by two numbers, 4.17 against 4.21 by a text, 3.06 against 3.08 by ~, 2.72 against 2.60 for an ordered page), and the browser module's filter 1.67 against 1.66.

timestamp holds UTC epoch milliseconds as an i64. Writes accept both text and numbers; a text literal in a comparison is parsed.

The representation is always ISO-8601 (2026-09-19T12:34:56.789Z), in JSON and in FenecQL alike. The calendar conversion is integer arithmetic including leap-year and century rules — no table, no dependency.

It exists as a separate type rather than an alias over int because ResultSet does not carry the schema, so an alias would show every client a raw number.

Text is ordered by its bytes unless a key says otherwise, and the bytes are wrong in every language: every capitalised word comes before every lowercase one, an accented letter lands after z, and every script past ASCII falls where its code points do. collate und orders the key as Unicode's root collation does — CLDR's, which PostgreSQL calls und-x-icu — the order every language's starts from: each script in a run of its own, its letters in their order, accents and case deciding only between words whose letters are the same. A language whose letters sort otherwise is a tailoring over it, and collate tr is Turkish, PostgreSQL's tr-x-icu: ç after c, ğ after g, ı before i with I as its capital and İ as the capital of i, ö, ş and ü after o, s and u.

It compares a level at a time over the whole string, as ICU does: the letters, then their accents (kar < kâr < kara), then their case (ince < İnce), so an accent or a capital only decides between strings whose letters are the same. What all three leave equal goes by its bytes, as under a deterministic PostgreSQL collation — the order is total, and a page never changes with the plan. It applies wherever order does: the query's rows, a lookup's children, an aggregate's groups by their key or a min or max of text. A key that holds no text — a number, id, count(*) — refuses it rather than ignore it.

A field can be declared in a collation, name text collate und, and its text is in it wherever it compares: order without naming one, <, <=, > and >= in where, min and max, and a @sorted index over it, which holds that order and is walked for it. So a listing pages by the last name it showed — where name > $last order name limit 20 — and the index answers as the scan does, row for row. Equality stays the bytes', since the collation ties no two different strings. The collation is the field's: it is written with the schema into the file, and describe shows it. A @sorted field without one keeps its bytes' order, so a key collated in the query is sorted rather than walked, and a comparison in where compares bytes.

Over a million Turkish names (160 797 of them distinct) a comparison costs 33.5 ns in tr and 35.2 in und where the bytes cost 6.7, because it starts where two strings part — Ahmet Yılmaz against Ahmet Kaya is Y against K — which took it from 68 ns, and a Latin letter's weights are made once per collation and read, where looking each up through the tailoring and the root's tables took it to 41:

fenecdb in process, PostgreSQL 17 by its own execution time with the sort in memory; both on one core (make collate-bench).

The browser module carries the Latin chunk and fetches the others as a page first needs them, from collate/ beside the module (JavaScript). A statement that would compare text of a script whose data it has not got is refused, the chunk named, rather than answered in another order; a write is refused before it changes anything, and an image whose collated text needs a chunk loads once it is there — a @sorted index built without it would be out of order.

The root and its chunks made the browser module 17.5 KB larger, 8.2 KB under gzip and 6.8 KB under brotli: the Latin chunk, the Turkish tailoring over it, and the code that reads them and has the others fetched. Every other chunk together is 145 KB, 67 KB gzip, 61 KB brotli. Their words are written as differences, which gzip and brotli take a third off: plain they were 102 and 90 KB. The Latin chunk alone comes out 0.3 KB larger under gzip that way, 0.9 KB under brotli, and 2.9 KB smaller in the module.

The fold is Unicode's full case folding, the C and F mappings of CaseFolding.txt, which Python's str.casefold() is too, and it is held to that at every code point (crates/fenec-core/tests/nocase.rs reads Python's own fold of each). Full rather than simple folding, which maps one code point to one: simple folding keeps straße apart from STRASSE, which ICU's case-insensitive comparison and every German reader find equal. A field in collate tr folds by Turkish's T mappings over it, as a Turkish reader expects:

A nocase field in a collation orders by the collation over the folds — collate tr nocase puts ılık before İnce and finds ince equal to the second — so its order and its equality agree: a collation tells every two different texts apart, and so it ties exactly the texts that fold to one, which come out in id order. Without a collation it orders by the folds' bytes. A query's collate orders it in another collation, still by its folds. ~, which ignores case on any text by lowercasing both sides, asks of a nocase field whether its fold holds the needle's: straße ~ "SS".

Its indexes hold the folds: a @hash or @unique bucket every spelling of one text, a @sorted index and an index over several fields their order, so an equality, an in, a range and an order … limit walk are answered as the scan answers them, row for row — a test holds every filter, order, group, facet and unique index over mixed-case data in ASCII, Turkish, German, Greek, Cherokee and full width to a twin collection holding Python's folds in plain fields. A lookup finds a child by its key as the child's field compares: a nocase one by the parent's value's fold. The schema writes the word into the file (a binary from before refuses it), describe, GET /_schema?as=fenecql and the declared schema carry it, a replica — a server's or a synced page's — compares as the primary does, and fenec import takes PostgreSQL's citext and SQLite's COLLATE NOCASE as it.

What folding costs, over 100 000 rows (make nocase-bench: a plain field holding one spelling and asked it, a nocase one holding a mix and asked another; names a third past ASCII):

An index lookup folds the one value it is asked. A scan folds a row only as far as it differs from the value — mostly its first character — with nothing allocated; ~ folds it whole, and still comes out quicker than lowercasing both sides. An order folds each row's key once, then sorts the folds; a @sorted index holds them folded and is walked instead. The fold is the standard library's lowercase mapping, and for the 297 code points whose fold is not their lowercase — 41 runs — its lowercase, uppercase and lowercase again: no table of its own beyond those runs. The browser module grew 11.5 KB, 5.0 KB gzip, 3.7 KB brotli — the engine's indexes, filters, groups, facets and orders 6.4 KB of it, the fold and its comparisons 2.1 — and a collection without a nocase field reads and writes as it did.

A document is its values one after another in field order, so what a change costs is what it does to that order. A field added goes last: a document written before it ends before the field's place, and reads null there. A rename is the schema alone. A field dropped leaves its place in the documents written before the drop — skipped on read, written as null after it, refused by name everywhere — and compact writes every document without it, copying each other value as its bytes stand. A field added under a dropped one's name is a new field: the old values stay in the dropped place, and the new field reads null until it is written. A change of type is the one thing that would rewrite every document under the write lock, and is not done.

Each is a write as a create index is: one of a block, put back by its ROLLBACK with the writes after it, seen by a subscriber as a schema change, applied by a replica, carried by the change stream (op: "alter", the documents after it read by the schema it made). In the file it is record kind 12, and a dropped place a nameless field of type tag 12, both of which a binary from before them refuses rather than read the fields after a dropped one a place early. The index of a field added is built over the documents there, every one of them null; a dropped field's goes, and a renamed one's goes with the name, a vector field's graph included.

Measured over a million rows in memory: an add, a rename and a drop take 0.025, 0.021 and 0.004 ms. Reading a field past where a document ends costs nothing a scan can measure: five scans of the million took 21.2 to 39.1 ms before the feature and 21.6 to 39.3 after, from 1.7% slower to 1.7% faster by the scan — as the same build with the reads written as before was, which makes it where the code lands, not what it does — and the browser module's scan of 20 000 rows 1.70 ms against 1.71. A compact that takes a dropped field out took 95 ms against 56 for one with none to take: it writes each document again rather than copy it. On a server it holds the write lock, where a compact otherwise runs beside the database.

compact drops the dead records and rewrites the file. The indexes stand — the documents are the same ones, and every index is keyed by id — except a vector graph holding tombstones, which is rebuilt: nothing else takes a deleted or rewritten document's node out of it. See Limits.