Indexes

One index makes one kind of question fast: hash, ordered, unique, over several fields, and rows that expire.

An index makes one kind of question fast and gives the rows that reading every row would. Declare it on a field when the collection is made, or add it later with create index.

Run in the shell, answers and all; the timings are from one run.

fenec> create collection events (user int, kind text, at timestamp);
collection `events` created
fenec> put events [{user: 1, kind: "view", at: "2026-05-03T09:00:10Z"}, {user: 1, kind: "buy", at: "2026-05-03T09:05:30Z"}, {user: 2, kind: "view", at: "2026-05-03T09:01:00Z"}, {user: 3, kind: "view", at: "2026-05-03T10:20:00Z"}, {user: 3, kind: "buy", at: "2026-05-03T10:40:00Z"}];
5 records (196.17µs)
fenec> explain get events where kind = "buy" order at desc limit 2
plan
-----------------------------------------------------------
filter: a full scan, 5 of 5 rows tested, 2 matched
order: at desc, every key read, 2 matches, all put in order
rows: 2
(3 rows, 188.29µs)
fenec> create index on events (kind) @hash;
index built on `events.kind`
fenec> create index on events (at) @sorted;
index built on `events.at`
fenec> explain get events where kind = "buy" order at desc limit 2
plan
-----------------------------------------------------------------------
order: the ordered index on at not walked, an equality names fewer rows
filter: the hash index on kind, 2 rows, which is the answer
order: at desc, every key read, 2 matches, all put in order
rows: 2
(4 rows, 327.04µs)
fenec> explain get events where at >= "2026-05-03T10:00:00Z" order at desc limit 2
plan
--------------------------------------------------
order: walked the ordered index on at desc, 2 rows
rows: 2
(2 rows, 253.75µs)
fenec> describe events
events
  user           int           
  kind           text            @hash
  at             timestamp       @sorted

With no index the filter tested all five rows. With a @hash on kind it names the two rows holding "buy", and with a @sorted on at a range and an order walk the index and stop at the page. Text and vector indexes are on Text search and Vector search.

Indexes

An index makes one kind of question fast. Declare it on a field when you create the collection, or add it later with create index.

create collection articles (
  title      text,
  body       text @text(k1=0.9, b=0.4, prefix=0),
  year       int @hash,
  slug       text @unique,
  published  timestamp @sorted,
  embed      vector<768> @hnsw(cosine, m=16, ef_construction=200)
)
  • One index per field: @hash for equality, @sorted for ranges and order, @text for ranked text search, @hnsw for nearest neighbours, @geo for points near a place. A path into a json field takes @hash, @unique or @sorted of its own (Objects and paths), and an index over several fields is the collection's, beside them (below).
  • @unique is a @hash that refuses a second document holding a value one holds already. null is no value: any number of documents may hold it.
  • Only filters joined by and use an index. Anything under or, and !=, ~ or has, reads every row.
  • For Turkish, Chinese or Japanese text, see prefix and chars below.
Index options and what they cost, measured
create index on articles (embed) @hnsw(cosine)
create index on articles (body) @text
create index on articles (body) @text(prefix=6)   -- for an inflected language
create index on products (title) @text(chars)     -- short Chinese or Japanese text
create index on articles (title) @sorted
create index on users (email) @unique             -- refused if a value is held twice

A field takes one index. @hash answers equality, @sorted ranges and order, @text ranked text retrieval, @hnsw nearest neighbours. @sorted is for int, float, timestamp and text — the types with an order.

@unique is a @hash index that refuses a second document holding a value one holds already: a put, an insert or a set that would write one is refused, and nothing of the statement is written — 409 over HTTP, as a taken id is. It answers equality and a lookup as @hash does. A block — a /batch, a run of several — is checked against its own earlier writes, so it may free a value and take it again, and a document keeping its own value is no second one. null is no value, as in SQL: any number of documents hold it. Values are one where = finds them one, so -0.0 and 0.0 collide. create index ... @unique over documents that hold a value twice is refused and names the value and two of its documents. A replica takes what its primary took and does not ask again; describe shows the index as @unique. The refusal names the document holding the value and the value, but to a scoped token it names the field alone (`users.email` is unique, and the value is taken): the row is another user's, which the token may not read.

The check asks the bucket the value would go in, so the index has to hold every document: the first write after an open builds it, where a plain @hash waits for the first read that wants it. An open costs what it did — 19 ms for an image of a million rows either way — and that first write takes 15 ms over 100 000 rows and 259 ms over a million; after it a put takes 940 ns against 850 for the same field under @hash, the price of the lookup. A collection with no unique field writes as fast as before (848 and 852 ns).

An @hnsw graph holds a vector once. Documents written with the same vector — the embedding of an empty chunk, a line of boilerplate, a zero vector — share its node, and near hands back each of them at the one distance. A node each, a vector written hundreds of times filled its neighbours' lists with copies of itself: over 20 000 rows of 128 dimensions, a third of them copies of 20 vectors, 1 812 copies were found by no search and other queries' recall fell from 0.82 to 0.67; one node each, none is missed and it is 0.87.

prefix=N indexes each word's prefixes alongside it (3..N by default; prefix_min moves the lower bound). It is a stemmer that needs no dictionary and no language setting, and it is off unless asked for because it is a trade the corpus decides. Word-boundary matching suits English; it does not suit a language that inflects by gluing suffixes on, where kitap, kitabı and kitapların are three terms sharing a stem the index never sees.

Measured on Turkish WebFAQ (144 846 documents, 10 000 queries): nDCG@10 0.4841 → 0.5547, +14.6%, closing about two fifths of the gap to a 278M-parameter multilingual transformer (0.650). The cost is the 3.4× postings it indexes — 58 MB → 182 MB, and 82 → 485 µs per query. On English SciFact the same option is worth +0.0137.

A script written without spaces is indexed in runs of characters. Chinese, Japanese and Thai run their words together, and Korean hangs its particles on them, so a word is not a term there: a run of Han, kana or Hangul is indexed as its overlapping pairs of characters — 東京都に is 東京, 京都, 都に — and one of Thai, Lao, Khmer or Myanmar, whose characters are letters, as its triples. A query is split the same way, so match body "東京" finds the run it is written in. There is no dictionary and no language setting; a script that spaces its words keeps its words.

chars indexes each character of Han, kana and Hangul as well, so a query of one character finds every run holding it, which pairs alone do only for a run of one. It pays on short text and costs on long: product titles gain a fifth, news and questions lose a little, and the index holds 1.6 to 1.9× the postings for a query 3 to 8× slower. Off unless asked for.

nDCG@10, BM25documentswords wholeruns of characterswith chars
C-MTEB EcomRetrieval, Chinese product titles100 9020.0060.4390.528
C-MTEB CovidRetrieval, Chinese news100 0010.1620.8680.872
WebFAQ, Chinese132 4910.1460.6860.666
JaGovFaqs-22k, Japanese22 1350.1190.5820.576
WebFAQ, Korean102 0000.5370.6820.651
WebFAQ, Thai (triples; pairs 0.495)47 4290.4480.547—

The same pipeline as BEIR (make beir over each set in its layout); English SciFact scores what it did, to the fourth decimal. A query over those sets takes 0.09 to 0.66 ms at the median.

A filter reaches an index only inside an and chain. Equality — = and in [..] — reaches a @hash field; in is a set of equalities written short, so it is answered as the union of one bucket per element. Either reaches id, which the store's own id index answers without a @hash being declared or possible. Comparisons — <, <=, >, >= and = — reach a @sorted field, which narrows them to a range of keys.

Everything else — !=, ~, has, comparisons on a field without @sorted, and anything under an or — is a full scan: each matching row is decoded and evaluated. ~ in particular is substring matching without ranking and reaches no index; ranked retrieval is match over a @text field.

A full scan reads rows in id order, and starts at the lowest id an id > x or id >= x in the and chain lets through, so a page read after the last id of the one before — where id > $1 limit 1000 — reads only its own rows: a million rows read that way a page at a time took 8.6 s and take 110 ms. The browser module leaves this out.

Ordered indexes

@sorted makes ranges, counts over a range and order … limit fast. It always gives the same rows as reading every row would.

get articles order published desc limit 20
get articles where published >= "2026-09-01" order published desc limit 20
import { connect } from '@fenecdb/web/client'
await db.from('articles').order('published', 'desc').limit(20).rows();

await db.from('articles')
  .where('published', '>=', '2026-09-01')
  .order('published', 'desc')
  .limit(20)
  .rows();
from fenecdb import Client
db.collection("articles").order("published", "desc").limit(20).rows()

(db.collection("articles")
    .where("published", ">=", "2026-09-01")
    .order("published", "desc")
    .limit(20)
    .rows())
import fenecdb "github.com/fenecdb/fenecdb/integrations/go"
db.From("articles").Order("published", "desc").Limit(20).Rows(ctx)

db.From("articles").
	Where("published", ">=", "2026-09-01").
	Order("published", "desc").
	Limit(20).
	Rows(ctx)
using FenecDb;
await db.From("articles")
    .Order("published", "desc")
    .Limit(20)
    .RowsAsync();

await db.From("articles")
    .Where("published", ">=", "2026-09-01")
    .Order("published", "desc")
    .Limit(20)
    .RowsAsync();
import FenecDB
try await db.from("articles")
    .order("published", "desc")
    .limit(20)
    .rows()

try await db.from("articles")
    .where("published", ">=", "2026-09-01")
    .order("published", "desc")
    .limit(20)
    .rows()
import com.fenecdb.*
db.from("articles").order("published", "desc").limit(20).rows()

db.from("articles")
    .where("published", ">=", "2026-09-01")
    .order("published", "desc")
    .limit(20)
    .rows()
import 'package:fenecdb/fenecdb.dart';
await db.from('articles').order('published', 'desc').limit(20).rows();

await db.from('articles')
    .where('published', '>=', '2026-09-01')
    .order('published', 'desc')
    .limit(20)
    .rows();
sends
get articles order published desc limit $1
-- [20]
get articles where published >= $1 order published desc limit $2
-- ["2026-09-01", 20]
  • It costs memory and write time: about 20 bytes a row for a number, plus the text for text.
  • It is rebuilt after an open by the first query that needs it; it is never written to the file.
When an ordered walk wins over the scan
get articles where title >= "k" and title < "l" count
get articles where published >= "2026-01-01" near embed $1 limit 10
import { connect } from '@fenecdb/web/client'
await db.from('articles')
  .where('title', '>=', 'k')
  .where('title', '<', 'l')
  .count();

await db.from('articles')
  .where('published', '>=', '2026-01-01')
  .near('embed', [0.9, 0.1, 0])
  .limit(10)
  .rows();
from fenecdb import Client
(db.collection("articles")
    .where("title", ">=", "k")
    .where("title", "<", "l")
    .count())

(db.collection("articles")
    .where("published", ">=", "2026-01-01")
    .near("embed", [0.9, 0.1, 0])
    .limit(10)
    .rows())
import fenecdb "github.com/fenecdb/fenecdb/integrations/go"
db.From("articles").
	Where("title", ">=", "k").
	Where("title", "<", "l").
	Count(ctx)

db.From("articles").
	Where("published", ">=", "2026-01-01").
	Near("embed", []float64{0.9, 0.1, 0}).
	Limit(10).
	Rows(ctx)
using FenecDb;
await db.From("articles")
    .Where("title", ">=", "k")
    .Where("title", "<", "l")
    .CountAsync();

await db.From("articles")
    .Where("published", ">=", "2026-01-01")
    .Near("embed", new[] { 0.9, 0.1, 0 })
    .Limit(10)
    .RowsAsync();
import FenecDB
try await db.from("articles")
    .where("title", ">=", "k")
    .where("title", "<", "l")
    .count()

try await db.from("articles")
    .where("published", ">=", "2026-01-01")
    .near("embed", [0.9, 0.1, 0])
    .limit(10)
    .rows()
import com.fenecdb.*
db.from("articles")
    .where("title", ">=", "k")
    .where("title", "<", "l")
    .count()

db.from("articles")
    .where("published", ">=", "2026-01-01")
    .near("embed", listOf(0.9, 0.1, 0.0))
    .limit(10)
    .rows()
import 'package:fenecdb/fenecdb.dart';
await db.from('articles')
    .where('title', '>=', 'k')
    .where('title', '<', 'l')
    .count();

await db.from('articles')
    .where('published', '>=', '2026-01-01')
    .near('embed', [0.9, 0.1, 0])
    .limit(10)
    .rows();
sends
get articles where title >= $1 and title < $2 count
-- ["k", "l"]
get articles where published >= $1 near embed $2 limit $3
-- ["2026-01-01", [0.9, 0.1, 0], 10]

order over a @sorted field with a limit walks the index in order and stops at the page, testing the rest of the filter on each row it passes; without one, every match's key is read and only offset + limit of them are put in order. A filter that matches too little to fill the page within an eighth of the collection sends the walk back to the scan: over a million rows one matching nothing cost 152 ms against the scan's 121, while one matching 1% fills the page in 0.29 ms against 125. A range on a @sorted field is read from the index as a set of ids — for a filter, a count, or the allowed set of a filtered near.

1 000 000 rows unless notedscan@sorted
order created desc limit 2026.8 ms0.001 ms
the newest 1%, order created desc limit 2063.1 ms0.001 ms
where price >= 99900 limit 20, 0.1% of the rows1.44 ms0.008 ms
the same range, count65.8 ms0.008 ms
where pct < 1 near embed $1 limit 10, 1% of 200 000 × 12817.9 ms0.16 ms

A comparison with now() — run_at <= now(), at >= now() - 3600000 — is a range of the index as one with a literal is: the time is worked out once, before the query is planned, and is the statement's one time. Left a call, a queue's claim walked past its ready jobs into the leased and delayed ones after them, testing each.

The planner prefers a smaller set when it has one: an equality over a @hash field, or a narrow range on another @sorted field, names fewer rows than the walk would pass, so those are sorted instead. A range wider than half the collection is left to the scan, which is cheaper than gathering and sorting that many ids.

A comparison reaches the index only when its literal says exactly what the scan would compare. price < 12.5 on an int field, or an integer past 253 against a timestamp, is evaluated row by row — the index narrows with the comparisons it can express and the rest are tested — because an answer that differs from the scan's is a wrong answer, not a fast one. Ties come out in ascending id either way, as the scan leaves them.

The cost is paid on write and in memory: about 20 bytes a row for a number, plus the text for text, and an insert into a sorted run. Over a million rows two ordered indexes took a bulk load from 0.52 s to 0.88 s, and the index is built from the documents by the first query that reads it after an open, like the hash and text indexes — it is never written to the file.

Indexes over several fields

@sorted(owner, created desc) keeps the rows in the order of several fields, the first first. The page an app reads most — one user's latest twenty, a tenant's inbox, a feed — is a run of it, read to the page and no further. @unique(org, slug) refuses a second row holding the same values in every part.

create collection posts (
  owner    int,
  created  timestamp,
  title    text,
  @sorted(owner, created desc)
)
create index on pages (org, slug) @unique                   -- or @unique(org, slug) in the collection
get posts where owner = $1 order created desc limit 20     -- a run of the index, 20 rows read
get posts where owner in [$1, $2, $3] order created desc limit 20   -- three runs, merged
get posts where owner = $1 and created >= $2 count          -- counted off the index
import { connect } from '@fenecdb/web/client'
await db.from('posts')
  .where('owner', 7)
  .order('created', 'desc')
  .limit(20)
  .rows();

await db.from('posts')
  .where('owner', 'in', [7, 8, 9])
  .order('created', 'desc')
  .limit(20)
  .rows();

await db.from('posts')
  .where('owner', 7)
  .where('created', '>=', '2026-10-01')
  .count();
from fenecdb import Client
(db.collection("posts")
    .where("owner", 7)
    .order("created", "desc")
    .limit(20)
    .rows())

(db.collection("posts")
    .where("owner", "in", [7, 8, 9])
    .order("created", "desc")
    .limit(20)
    .rows())

(db.collection("posts")
    .where("owner", 7)
    .where("created", ">=", "2026-10-01")
    .count())
import fenecdb "github.com/fenecdb/fenecdb/integrations/go"
db.From("posts").
	Where("owner", "=", 7).
	Order("created", "desc").
	Limit(20).
	Rows(ctx)

db.From("posts").
	Where("owner", "in", []any{7, 8, 9}).
	Order("created", "desc").
	Limit(20).
	Rows(ctx)

db.From("posts").
	Where("owner", "=", 7).
	Where("created", ">=", "2026-10-01").
	Count(ctx)
using FenecDb;
await db.From("posts")
    .Where("owner", 7)
    .Order("created", "desc")
    .Limit(20)
    .RowsAsync();

await db.From("posts")
    .Where("owner", "in", new object?[] { 7, 8, 9 })
    .Order("created", "desc")
    .Limit(20)
    .RowsAsync();

await db.From("posts")
    .Where("owner", 7)
    .Where("created", ">=", "2026-10-01")
    .CountAsync();
import FenecDB
try await db.from("posts")
    .where("owner", 7)
    .order("created", "desc")
    .limit(20)
    .rows()

try await db.from("posts")
    .where("owner", "in", [7, 8, 9])
    .order("created", "desc")
    .limit(20)
    .rows()

try await db.from("posts")
    .where("owner", 7)
    .where("created", ">=", "2026-10-01")
    .count()
import com.fenecdb.*
db.from("posts")
    .where("owner", 7)
    .order("created", "desc")
    .limit(20)
    .rows()

db.from("posts")
    .where("owner", "in", listOf(7, 8, 9))
    .order("created", "desc")
    .limit(20)
    .rows()

db.from("posts")
    .where("owner", 7)
    .where("created", ">=", "2026-10-01")
    .count()
import 'package:fenecdb/fenecdb.dart';
await db.from('posts')
    .where('owner', 7)
    .order('created', 'desc')
    .limit(20)
    .rows();

await db.from('posts')
    .where('owner', 'in', [7, 8, 9])
    .order('created', 'desc')
    .limit(20)
    .rows();

await db.from('posts')
    .where('owner', 7)
    .where('created', '>=', '2026-10-01')
    .count();
sends
get posts where owner = $1 order created desc limit $2
-- [7, 20]
get posts where owner in [$1, $2, $3] order created desc limit $4
-- [7, 8, 9, 20]
get posts where owner = $1 and created >= $2 count
-- [7, "2026-10-01"]
  • Equalities on its first fields, then a range or an order on the next. where owner = $1 order created desc reads its run; where created > $1 alone does not use (owner, created), since owner comes first.
  • An order matches it in its fields' directions, or every one the other way round: (owner, created desc) answers order created desc and order created for one owner.
  • It gives the scan's answer, row for row, ties in ascending id either way, as @sorted does.
  • A lookup keyed by its first fields reads each parent's children as a run of it, walked in the level's order where that is the index's next parts: lookup lines on (org, order) = (org, id) order line asc limit 5.
  • Up to 8 fields of bool, int, float, timestamp, text (in its collation) or json, a path into one among them (meta.lang). A field keeps its own index beside it.
How a key over several fields is laid out
create index on posts (owner, created desc) @sorted        -- built beside the database on a server
drop index on posts (owner, created)                        -- by its fields
explain get posts where owner = 7 order created desc limit 20
-- order: walked the index on (owner, created desc), owner fixed, 20 of 100 rows read, 20 kept

A key is its fields' values laid end to end, each written so that the bytes order as the values do — an int's sign turned over, a float's bits made to order, a text escaped and ended, a descending field's bytes inverted — so an equality on its first fields is a run of the keys and a range on the next one is a run inside it. A text in a collation is kept as written and compared in it. null stands below every value of a field, as order puts it; a row holding a float's NaN, which compares equal to everything, or a json value no order has a place for (a boolean, a list, an object, an int past 253) is held apart: while one is, the index is walked for no order, and a filter's run takes those rows with it, each tested.

The planner takes the and chain's terms in the index's order — an equality, or an in whose every element it can say exactly, on each field as far as one is fixed, then the comparisons on the next — and tests every other term row by row; a term it cannot say exactly (a null, 12.5 against an int) is left to that test, so the answer never changes. An in makes a run for each element, 256 at most together; under an order on the fields after it the runs are walked together, merged by those fields. Of the plans, the narrower set wins by exact counts: the run is two searches, a @hash bucket its length, so a filter the index says whole reads the page and nothing more, and one it does not is walked only while its run is no longer than any set another index names — and gives up past an eighth of the collection, as a @sorted walk does. A count the index says whole is counted off it without a row read. explain names the index, the fields fixed, the runs and the range.

@unique over several fields is asked before every put, insert and set: a second row holding the same values in every part is refused, 409 over HTTP, naming the fields and the other row — to a scoped token the fields alone. A null in any part is no value. put ... if absent passes over the row holding them, and if absent else set sets it. create index ... @unique over rows already holding a key twice is refused naming two of them.

It is derived data, built from the documents by the first read after an open, kept on every write, put back with a block that does not land, and never written to the file. A field renamed keeps it under the new name; a field dropped takes it off. A sync replica builds what the schema declares, a unique one as an ordered one, as its @unique fields are @hash.

1 000 000 posts over 10 000 ownersp50p99built inmemory
where owner = $1 order created desc limit 20, @sorted(owner, created desc)0.006 ms0.011 ms147 ms40 MB
the same over owner @hash, its bucket sorted0.012 ms0.017 ms75 ms11 MB
the same over created @sorted, walked and given up24.4 ms40.5 ms34 ms20 MB
the same by the scan21.3 ms37.4 ms——
a feed, owner in ten owners, @sorted(owner, created desc)0.022 ms0.029 ms
the feed over owner @hash0.100 ms0.136 ms

Writes pay for it: a lone put into those million rows took 715 ns with no index, 851 beside created @sorted and owner @hash, and 1 561 beside @sorted(owner, created desc), whose first field puts each write anywhere in it. A unique one asks its run before each write: 1 745 ns a put against 1 285 for @unique on the two joined into one field, and a read by both fields 2.5 µs against 1.3 by the joined field — a hash index over several fields would answer that read sooner, by a microsecond, and was not worth a third index's code beside this one (make composite-bench).

Rows that expire

@ttl on a timestamp field makes a row expire that long after its value — for sessions, caches and anything else kept for a while.

create collection sessions (user text @hash, seen timestamp @ttl(30m))
  • Reads are exact at once. A row past its time is never returned, even before it is deleted: every read of the collection leaves it out.
  • The server deletes it. A primary sweeps once a minute, as ordinary deletes that replicas, /_changes, archives and subscriptions see as any.
  • expired() reads what the others leave out, for a reaper that gives back what a row held before the sweep deletes it (below).
  • Durations are whole ms, s, m, h or d: @ttl(45s), @ttl(12h), @ttl(7d).
The reaper, and what @ttl costs, measured
put sessions {user: "ada", seen: now()}
set sessions {seen: now()} where user = "ada"        -- seen again: lives on
get sessions where user = "ada"                       -- nothing once 30 minutes pass
import { connect, expr } from '@fenecdb/web/client'
await db.from('sessions').insert({ user: 'ada', seen: expr('now()') });

await db.from('sessions')
  .where('user', 'ada')
  .update({ seen: expr('now()') });

await db.from('sessions').where('user', 'ada').rows();
from fenecdb import Client, expr
db.collection("sessions").insert({"user": "ada", "seen": expr("now()")})

(db.collection("sessions")
    .where("user", "ada")
    .update({"seen": expr("now()")}))

db.collection("sessions").where("user", "ada").rows()
import fenecdb "github.com/fenecdb/fenecdb/integrations/go"
db.From("sessions").
	Insert(ctx, fenecdb.D("user", "ada", "seen", fenecdb.Expr("now()")))

db.From("sessions").
	Where("user", "=", "ada").
	Update(ctx, fenecdb.D("seen", fenecdb.Expr("now()")))

db.From("sessions").Where("user", "=", "ada").Rows(ctx)
using FenecDb;
await db.From("sessions")
    .InsertAsync(new { user = "ada", seen = Computed.Expr("now()") });

await db.From("sessions")
    .Where("user", "ada")
    .UpdateAsync(new { seen = Computed.Expr("now()") });

await db.From("sessions").Where("user", "ada").RowsAsync();
import FenecDB
try await db.from("sessions")
    .insert(["user": "ada", "seen": .expr("now()")] as Value)

try await db.from("sessions")
    .where("user", "ada")
    .update(["seen": .expr("now()")] as Value)

try await db.from("sessions").where("user", "ada").rows()
import com.fenecdb.*
db.from("sessions")
    .insert(mapOf("user" to "ada", "seen" to Computed.expr("now()")))

db.from("sessions")
    .where("user", "ada")
    .update(mapOf("seen" to Computed.expr("now()")))

db.from("sessions").where("user", "ada").rows()
import 'package:fenecdb/fenecdb.dart';
await db.from('sessions')
    .insert({'user': 'ada', 'seen': Computed.expr('now()')});

await db.from('sessions')
    .where('user', 'ada')
    .update({'seen': Computed.expr('now()')});

await db.from('sessions').where('user', 'ada').rows();
sends
put sessions {user: $1, seen: now()}
-- ["ada"]
set sessions {seen: now()} where user = $1
-- ["ada"]
get sessions where user = $1
-- ["ada"]
create index on events (at) @ttl(7d)                  -- an existing timestamp field
alter collection sessions alter field seen @ttl(1h)   -- another expiry
alter collection sessions alter field seen @sorted    -- none: the rows live

A read of the collection — get, count, an aggregate, near, match, a lookup level, an inner get, a set or del by filter, and a subscription's shape — has not (seen <= now - ttl) added to its filter, the way a token's rule is added: a row is gone at the moment its time plus the ttl is reached. A row whose field is null has no time to expire from, and lives. In the browser the module has no clock: now is the page's, Date.now() unless db.now is set, as every time is handed to it; the page deletes nothing, and a row past its time stays in its file, out of every read. A live query in the page is woken by writes, not by time passing.

@ttl is an ordered index whose rows expire, so the field also does everything @sorted does, and a collection's rows expire by one such field. The sweeper deletes a range of it at a time rather than scan for the rows past their time: found under the read lock, deleted under the write lock 1 000 at a time, each batch a block of ordinary deletes, its fsync run once the lock is let go. A replica never sweeps — its writes are its primary's, the sweep's among them — and neither does a tenant that follows another node. The server's Idempotency-Keys are kept this way.

The ttl is a kind of index in the file (kind 9, the milliseconds behind it), so a binary from before it refuses the file rather than hand out the rows past their time. A write takes the place of a row past its time: an insert naming its id writes over it, and a @unique value it holds is let go of with it, the row deleted in the same statement — so a lock taken by insert and left to expire can be taken again at once (Instead of Redis). BM25's statistics count the row until the sweep.

Reading what expired: the reaper

A row past its time is out of every read, so a hold that lapsed cannot be found to give its money back, nor a reservation its stock. expired() in a filter reads those rows — and only those, where the filter says so — wherever a filter goes: get, count, a lookup level, an inner get, set and del. It is until <= now - ttl, a range of the field's ordered index; not expired() is what a read finds without it.

create collection holds (account text @hash, amount int, until timestamp @ttl(1ms))

get holds select id, account, amount where expired() limit 100   -- what lapsed

-- each given back once, in one block (a /batch): the delete is the guard
del holds where expired() and id = $1 require 1
set accounts {held: held - $2} where ext = $3 and held >= $2 require 1

A capture needs no guard of its own against the lapse: get holds where id = $1 limit 1 require 1 finds no hold past its time. Two reapers, or a reaper and a capture, racing for one hold: the del ... require 1 lets one through, and the other's block is put back whole. The sweeper leaves a row a minute past its time before it deletes it, so a reaper that comes by once a minute sees every one; a row it misses for longer is deleted unseen, and what it held is the reconciliation's to find. A scoped token reads them only by the policy's expired grant (Scoped access). Without @ttl on the collection, expired() is an error.

What it costs, measured (make ttl-bench, a million rows, half past their time). A read tests each row's time as part of its filter, as the same test written by hand does: count becomes a scan, 68.8 ms against 1.1 with no expiry, and a filter that scanned already reads one field more, where s = "w7" count 68.5 ms against 51.9 — 70.1 with the test written out. A collection with no @ttl reads as it did. The sweep of 100 000 rows past their time out of a file of 200 000 took 0.56 s in batches of 1 000, each holding the write lock 1.06 ms at the median and 2.3 at most.