Read pipeline
How recall answers a cue — three retrievers fused with RRF, an always-on cross-encoder rerank, a filter chain, and a membership verdict of Single, Many, or None.
There is one read verb and one code path. Every request walks the same pipeline whether or not a user schema has been declared. recall does not hand back a raw ranked list for the caller to sift — it shapes the fused, filtered pool into an answer. For the request/response shapes see API: recall; for the SDK method see SDK: recall.
The three retrievers
A rule-based query router classifies the cue and picks per-retriever weights, but all three lanes are always wired — none is ever absent.
- Semantic — HNSW cosine search over the in-RAM memory (and statement) embeddings, 384-dim. Catches paraphrase and topical similarity.
- Lexical — tantivy BM25 over the memory-text (and statement-text) indexes. Catches exact names, IDs like
ACME-1247, and rare terms that similarity smooths over. - Graph — entity-joined traversal over the typed graph: from an anchor entity, walk relations and statements to surface memories where the entity is subject or object.
Each retriever returns a ranked list; scores are retriever-internal (cosine, BM25, graph proximity) and not comparable across lanes — fusion uses rank, not raw score. Each lane is capped at a top_n (default 100, router-tunable per query class) so fusion cost stays bounded.
A lane surfaces whichever item type its index holds, not always a memory: the semantic and lexical lanes return memories and statements, while the graph lane returns typed-graph items — entities and relations (which fusion resolves back to their source memories). With trace: true, each lane's raw pre-fusion candidates are surfaced with a kind (memory · statement · entity · relation) so you can see exactly what each lane found — e.g. the graph lane's Diego —works_at→ billing team relation alongside the semantic lane's memory hits.
| Query pattern | Lanes the router leans on |
|---|---|
| "What does Priya prefer?" | Graph (Priya) + preference intent |
| "Find memories about budget pushback" | Semantic + Lexical |
| "Show me ticket ACME-1247" | Lexical (exact ID) |
| "Find similar concepts to X" | Semantic |
| "Who is connected to Priya through projects?" | Graph (multi-hop) |
RRF fusion
The ranked lists collapse to one through weighted Reciprocal Rank Fusion:
RRF_score(d) = Σ over retrievers ( wᵢ / (k + rankᵢ(d)) )
where k = 60Only ranks enter the sum, so it doesn't matter that cosine is bounded and BM25 isn't — fusion is score-scale invariant. k = 60 is the canonical Cormack et al. constant: with it, rank 1 contributes 1/61 ≈ 0.0164 and rank 10 contributes 1/70 ≈ 0.0143, a ratio of ~1.15, so no single retriever dominates. Documents a retriever didn't return contribute 0. Default weights bias slightly toward the graph lane (semantic 1.0, lexical 1.0, graph 1.2); the router adjusts them per query class — entity-anchored queries lift graph, exact-term queries lift lexical, paraphrase-likely queries lift semantic.
A soft, post-fusion recency boost (the temporal weight, default half-strength) can nudge comparably-relevant memory hits by event time on a 90-day half-life — but only when the cue carries a temporal signal, so timeless facts are never penalised for being old. It is a tie-breaker capped at one RRF unit, never an override of genuine relevance. There is no separate "temporal retriever."
The filter chain
After fusion, a chain trims the pool before shaping: tombstone state, memory kind, context, temporal bounds, confidence, salience, and supersession. Filters are AND-combined. Type and subject filters can also steer the router's lane choice. Supersession filtering is what keeps a corrected memory's stale predecessor out of the default answer.
Rerank
When the cross-encoder is loaded, the fused-and-filtered top candidates are reordered by bge-reranker-base on every read. This is first-class, not a request flag — the only control is the deploy-time [rerank] enabled load gate. When an operator opts out, no model loads and the pipeline returns RRF-only ordering (no error, no request change). See Policies.
Membership — Single / Many / None
recall does not return a top-K window. It computes an answer set over the full filtered pool using a relevance band, then shapes the verdict by cardinality:
- Let
topbe the best relevance score in the pool. A candidate joins the answer iff it clears both an absolute floor and a relative band aroundtop(score ≥ ABS_FLOORandscore ≥ top × REL_BAND). Lexical- and graph-confirmed hits, and grounded source memories for a resolved subject/predicate, are admitted the same way. answer_kindis then pure cardinality:0 → None,1 → Single,2+ → Many. When several members all assert the same value, the router may collapse them to oneSingle.max_resultsis only a safety ceiling on how many members aManyreturns — it never turns aManyinto aSingleby truncation, and never suppresses the band.
Absence is explicit: None with an empty list, never a fabricated guess. The per-member score and retriever provenance ride along to say why a member surfaced — they are not a ranking the caller is expected to re-sort or threshold.
No client-side re-ranking
Brain deliberately does not expose a "fetch a broad top-K and re-rank on the client" pattern — that is the SaaS-search shape this database rejects. Fusion, rerank, and membership all happen server-side; the answer comes back already shaped.
Read-your-writes
When a recall runs inside a transaction (txn_id set), the transaction's pending encodes are overlaid on the committed result before the verdict is built: pending tombstones drop committed hits, pending encodes are scored against the cue and merged, and membership then runs over the combined list. This is the one read-your-writes path, and it runs whether or not a schema is active. Outside a transaction, pass consistency: ReadAfterWrite to wait for a just-encoded memory to become searchable (HNSW publication lag is ~10 ms).
The graph reads — plan and reason
recall answers "what is relevant." Two sibling read verbs answer graph questions over the same edge graph:
plan— bidirectional BFS from a start state to a goal, returning connecting paths through the edge graph (CAUSED,FOLLOWED_BY,DERIVED_FROM,PART_OFby default).reason— from memories near a claim, followSUPPORTS/DERIVED_FROMfor supporting evidence andCONTRADICTSfor opposing evidence, returning both sets with an aggregate confidence.
Both start from a vector lookup and then traverse; neither fabricates evidence. There are also non-ranked enumeration reads — MEMORY_LIST (what's stored), GRAPH_FETCH (typed-graph export), and MEMORY_INSPECT (one memory's write story) — that skip the retrieval pipeline entirely.
Latency budget
| Phase | p50 |
|---|---|
| embed cue | 2–8 ms (cache hit ~0) |
| retrievers (parallel) | 1–10 ms each |
| fuse + filter | < 5 ms |
| rerank (when loaded) | 10s of ms |
| membership | < 5 ms |
| total (single shard) | ~10 ms p50, ~25 ms p99 |
Latency scales with the candidate-pool size the retrievers fan out over and filter complexity — not with a caller-chosen result count, because there isn't one. Cross-shard recalls fan out in parallel and merge over the global pool, pushing p99 to ~30–50 ms.
Was this page helpful?
Write pipeline
How encode works — a synchronous fast path (validate → embed → reserve → persist) split from asynchronous derivation by the WAL-fsync acknowledgement barrier.
Supersession and provenance
Brain never mutates memories — corrections supersede old records, contradictions are surfaced not hidden, and every statement traces back to the memories it was extracted from.