M3.6.5 — Query: filter-then-recall, R opt-in, relevance floor

Field	Value
Phase	M3.6 — Reference corpora
Size	M — 1–3 days
Status	⬜ Not started
Flags	—
Spec	inlined below
Blocks	M3.6.6
Depends	M3.6.2, M3.3, M3.2, M2.7

Goal

Make R reachable on request, unreachable by default, and stop the retriever answering questions it has no evidence for.

Facts (inlined — no spec read needed)

mem query "why did requests over 10KB fail?"          # L1,L2 — unchanged
mem query --levels R "kubectl describe pod"           # reference only
mem query --levels L1,L2,R "..."                      # both, R marked in output
mem query "…" --min-score 0.4                         # override the floor

Filter before recall, not after. M3.3 recalls 10×k from HNSW and reranks down to k. A corpus is typically an order of magnitude larger than the project's own memory, so R rows compete for those 50 candidate slots even when the caller excluded them — and post-filtering then returns three results instead of five, quietly. The level predicate belongs in the SQL that drives the HNSW scan. The existing (project, level) index already supports it.

Abstention. With a corpus loaded, every question has something moderately close, so unconditional top-k starts returning plausible prose for questions the memory cannot answer — worse than an empty result, because it reads as an answer. If the best post-rerank score is below the floor, return no hits and say why:

no hits above relevance floor (best 0.21 < 0.35 threshold)
try --min-score to lower it, or --levels R to search reference docs

The floor applies to the reranked score, not cosine distance. M3.2's own fixture separates a relevant from an irrelevant passage by four orders of magnitude; cosine distance does not, which is why the floor cannot live at the recall stage.

R is visually distinct in output. A reference hit prints its source URI and heading path where a project hit prints provenance. A caller must never have to infer from wording whether an answer came from this cluster's history or from upstream documentation.

R has no provenance walk. M3.3 walks memory_edge one hop for L1 and two for L2. R has no edges by construction (M3.6.2), so the walk is skipped rather than returning empty — and mem verify gains the assertion that makes that safe.

Steps

Push the level filter into the recall query; assert candidate width is 10×k after filtering.
--levels accepts R; default remains L1,L2.
Apply the relevance floor to reranked scores; --min-score overrides, --min-score 0 disables.
Abstention message names the best score, the threshold, and the two escapes.
Render R hits with source URI and heading path; suppress the provenance walk.
Exit code: abstention is exit 0 with no hits, not an error — it is a valid answer. Unresolvable project stays non-zero (M3.3 assertion 7).
mem verify --edges asserts no memory_edge row names an R sha as parent.

Acceptance

Default query over a database containing a large corpus returns exactly the same hits as before the corpus was added.
--levels R returns reference sections with URI and heading path.
A question with no good match returns nothing and explains itself.
Lowering --min-score surfaces the suppressed hits.
mem verify rejects a hand-inserted L1 -> R edge.

Verify

Harness: seeded database with the poimen log plus a reference corpus large enough to dominate raw recall — at least 10× the project node count. Live reranker for scoring assertions, deterministic embedder elsewhere.

Integration test — tests/it_query_levels.rs:

a1_default_unchanged_by_corpus — snapshot default query results before and after ingesting the corpus; assert byte-identical output. This is the headline assertion of the task.
a2_filter_before_recall — instrument the repository; assert the SQL driving HNSW carries the level predicate and returns 10×k rows post-filter, not 10×k pre-filter then fewer.
a3_levels_r_returns_reference — --levels R returns R nodes with source URI and heading path populated.
a4_floor_abstains — a question with no relevant content returns zero hits, exit 0, message naming best score and threshold.
a5_floor_override_recovers — same question with --min-score 0 returns the suppressed hits, proving abstention is a floor and not a bug upstream.
a6_floor_applies_post_rerank — construct a case where cosine is high and rerank is low; assert it is suppressed. The reverse ordering passes every other assertion here.
a7_r_hits_visually_distinct — assert R hits carry no provenance block and do carry a URI, in both human and --format json output.
a8_no_edge_to_r — hand-insert an L1 -> R edge; assert mem verify --edges exits non-zero and names the offending pair.
a9_mixed_levels_ordering — --levels L1,L2,R returns both kinds ranked together with the level labelled on every row.

Command: cargo test -p mem-cli query_levels

False pass:

Running assertion 1 against a small corpus. If the corpus is smaller than the recall width, post-filtering and pre-filtering give the same answer and assertion 2 is the only thing separating them — the fixture size is part of the test.
Testing abstention with a query that matches nothing at all. Zero recall returns zero hits regardless of the floor; the fixture needs a weak but non-empty match, or assertion 4 passes with the floor unimplemented.
Asserting --levels R works without asserting the default excludes R. Both directions are the contract.

Traps

Applying the floor to the first-stage cosine score. Cosine on nomic puts unrelated text closer than intuition suggests; a floor there either suppresses good hits or does nothing, depending on the corpus.
Treating abstention as an error exit. Callers wrap mem query in scripts; a non-zero exit for "no confident answer" turns a normal outcome into a pipeline failure and the floor gets disabled within a week.
Letting the reranker see 50 R candidates and 3 project candidates in a mixed query. The reranker is not calibrated across levels, and the corpus wins on fluency; recall per level, then merge.

Background: DESIGN.md — reference corpora, pgvector, retrieval

6.4 KiB Raw Permalink Blame History Unescape Escape