M3.2 — Rerank client

Field	Value
Phase	M3 — L2 synthesis and retrieval
Size	S — under 1 day
Status	⬜ Not started
Flags	—
Spec	inlined below
Blocks	M1.1

Goal

Reorder vector-recall candidates by actual relevance, because embedding distance is a coarse filter.

Facts (inlined — no spec read needed)

POST /v1/rerank
{"query":"what is rust","texts":["rust is a language","bananas"]}
-> [{"index":0,"score":0.98176736},{"index":1,"score":0.00008833366}]

BAAI/bge-reranker-base via TEI. Note the response shape is a bare array, not an OpenAI-style {"data": [...]} envelope — this route does not follow the chat convention.

The discrimination is real: 0.98 vs 0.00009 on that probe, four orders of magnitude. Embedding cosine on the same pair would be far closer, which is why recall-then-rerank beats recall alone.

index refers to the position in the submitted texts array; results come back sorted by score, so the index is the only way to map back. Do not assume order.

Batch limits apply as with embeddings — keep candidate lists modest (top-50 from recall is plenty).

Steps

RerankClient::rerank(query, texts) -> Result<Vec<Scored>> in mem-llm.
Parse the bare array; map index back to the caller's items.
Preserve the caller's item type: take &[T], return Vec<(T, f32)> so the caller does not re-associate by position.
Same apikey header, retry and timeout policy as M1.1.
Empty input returns empty without a request.

Acceptance

Results map correctly back to input items via index.
A more relevant text scores above a less relevant one on a live call.
Empty input makes no request.

Verify

Harness: wiremock offline, one #[ignore] live test.

Integration test — tests/it_rerank.rs:

a1_bare_array_parsed — mock returns [{"index":1,...},{"index":0,...}]; assert parsing succeeds.
a2_index_mapping — with the out-of-order mock above, assert the returned items correspond to inputs 1 and 0 respectively, not 0 and 1.
a3_empty_no_request — empty texts; assert zero requests.
a4_apikey_sent — header present.
a5_live_discriminates — #[ignore]; real gateway, query "what is rust" against ["rust is a language","bananas"]; assert the first scores at least 10× the second.

Command: cargo test -p mem-llm rerank (add -- --ignored for a5)

False pass:

Assuming the response preserves input order. A mock that returns results in input order passes a naive test and the live endpoint returns them sorted, which silently mislabels every result. Assertion 2 must use an out-of-order mock.
Expecting an OpenAI envelope. It will fail immediately against the live route, but a mock written to match the wrong shape hides that until integration.

Traps

Re-associating results by position instead of by index. The scores are right and attached to the wrong documents — a bug that looks like poor retrieval quality rather than a mapping error.

Background: DESIGN.md — Verified facts, retrieval

3.1 KiB Raw Blame History Unescape Escape