89 lines
3.1 KiB
Markdown
89 lines
3.1 KiB
Markdown
|
# M3.2 — Rerank client
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
|---|---|
|
|||
|
|
| Phase | M3 — L2 synthesis and retrieval |
|
|||
|
|
| Size | S — under 1 day |
|
|||
|
|
| Status | ⬜ Not started |
|
|||
|
|
| Flags | — |
|
|||
|
|
| Spec | inlined below |
|
|||
|
|
| Blocks | M1.1 |
|
|||
|
|
|
|||
|
|
## Goal
|
|||
|
|
|
|||
|
|
Reorder vector-recall candidates by actual relevance, because embedding distance
|
|||
|
|
is a coarse filter.
|
|||
|
|
|
|||
|
|
## Facts (inlined — no spec read needed)
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
POST /v1/rerank
|
|||
|
|
{"query":"what is rust","texts":["rust is a language","bananas"]}
|
|||
|
|
-> [{"index":0,"score":0.98176736},{"index":1,"score":0.00008833366}]
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`BAAI/bge-reranker-base` via TEI. Note the **response shape is a bare array**, not
|
|||
|
|
an OpenAI-style `{"data": [...]}` envelope — this route does not follow the chat
|
|||
|
|
convention.
|
|||
|
|
|
|||
|
|
The discrimination is real: 0.98 vs 0.00009 on that probe, four orders of
|
|||
|
|
magnitude. Embedding cosine on the same pair would be far closer, which is why
|
|||
|
|
recall-then-rerank beats recall alone.
|
|||
|
|
|
|||
|
|
`index` refers to the position in the submitted `texts` array; results come back
|
|||
|
|
**sorted by score**, so the index is the only way to map back. Do not assume
|
|||
|
|
order.
|
|||
|
|
|
|||
|
|
Batch limits apply as with embeddings — keep candidate lists modest (top-50 from
|
|||
|
|
recall is plenty).
|
|||
|
|
|
|||
|
|
## Steps
|
|||
|
|
|
|||
|
|
1. `RerankClient::rerank(query, texts) -> Result<Vec<Scored>>` in `mem-llm`.
|
|||
|
|
2. Parse the bare array; map `index` back to the caller's items.
|
|||
|
|
3. Preserve the caller's item type: take `&[T]`, return `Vec<(T, f32)>` so the
|
|||
|
|
caller does not re-associate by position.
|
|||
|
|
4. Same `apikey` header, retry and timeout policy as M1.1.
|
|||
|
|
5. Empty input returns empty without a request.
|
|||
|
|
|
|||
|
|
## Acceptance
|
|||
|
|
|
|||
|
|
- Results map correctly back to input items via `index`.
|
|||
|
|
- A more relevant text scores above a less relevant one on a live call.
|
|||
|
|
- Empty input makes no request.
|
|||
|
|
|
|||
|
|
## Verify
|
|||
|
|
|
|||
|
|
**Harness:** `wiremock` offline, one `#[ignore]` live test.
|
|||
|
|
|
|||
|
|
**Integration test** — `tests/it_rerank.rs`:
|
|||
|
|
1. `a1_bare_array_parsed` — mock returns `[{"index":1,...},{"index":0,...}]`;
|
|||
|
|
assert parsing succeeds.
|
|||
|
|
2. `a2_index_mapping` — with the out-of-order mock above, assert the returned
|
|||
|
|
items correspond to inputs 1 and 0 respectively, not 0 and 1.
|
|||
|
|
3. `a3_empty_no_request` — empty texts; assert zero requests.
|
|||
|
|
4. `a4_apikey_sent` — header present.
|
|||
|
|
5. `a5_live_discriminates` — `#[ignore]`; real gateway, query "what is rust"
|
|||
|
|
against `["rust is a language","bananas"]`; assert the first scores at least
|
|||
|
|
10× the second.
|
|||
|
|
|
|||
|
|
**Command:** `cargo test -p mem-llm rerank` (add `-- --ignored` for a5)
|
|||
|
|
|
|||
|
|
**False pass:**
|
|||
|
|
- Assuming the response preserves input order. A mock that returns results in
|
|||
|
|
input order passes a naive test and the live endpoint returns them sorted,
|
|||
|
|
which silently mislabels every result. Assertion 2 must use an out-of-order
|
|||
|
|
mock.
|
|||
|
|
- Expecting an OpenAI envelope. It will fail immediately against the live route,
|
|||
|
|
but a mock written to match the wrong shape hides that until integration.
|
|||
|
|
|
|||
|
|
## Traps
|
|||
|
|
|
|||
|
|
- Re-associating results by position instead of by `index`. The scores are right
|
|||
|
|
and attached to the wrong documents — a bug that looks like poor retrieval
|
|||
|
|
quality rather than a mapping error.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
Background: [DESIGN.md](../DESIGN.md) — Verified facts, retrieval
|