83 lines
3.1 KiB
Markdown
83 lines
3.1 KiB
Markdown
|
# M8.4 — Reciprocal Rank Fusion engine
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
|---|---|
|
|||
|
|
| Phase | M8 — Hybrid Search |
|
|||
|
|
| Size | S — 0.5–1 day |
|
|||
|
|
| Status | ⬜ |
|
|||
|
|
| Flags | — |
|
|||
|
|
| Spec | inlined below |
|
|||
|
|
| Blocks | M8.5 |
|
|||
|
|
| Depends | — (pure logic, no infra dependency) |
|
|||
|
|
|
|||
|
|
## Goal
|
|||
|
|
|
|||
|
|
Implement Reciprocal Rank Fusion (RRF) that merges two ranked lists from different scoring distributions into a single ranked list. No parameter tuning required.
|
|||
|
|
|
|||
|
|
## Why RRF, not weighted linear
|
|||
|
|
|
|||
|
|
pgvector returns cosine similarity in `[0.0, 1.0]`. OpenSearch BM25 returns unbounded scores in `[0, 50+]`. These distributions are incomparable.
|
|||
|
|
|
|||
|
|
**Weighted linear (0.6 * sem + 0.4 * lex)** requires min-max normalisation, which is fragile: one outlier score compresses all other scores to near-zero. It also requires choosing weights, which requires labelled data we don't have yet.
|
|||
|
|
|
|||
|
|
**RRF** ignores score magnitudes entirely. It uses only **rank positions**: the document that appears first in a list gets rank 1, second gets rank 2, etc. The formula is:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
RRF_score(d) = Σ 1 / (k + rank_i(d))
|
|||
|
|
lists
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Where `k = 60` is a constant (academic standard, Cormack et al. 2009). A document in rank 1 of both lists gets `1/61 + 1/61 = 0.0328`. A document in rank 1 of only one list gets `1/61 = 0.0164`. The first always outranks the second, regardless of original score magnitudes.
|
|||
|
|
|
|||
|
|
## Design
|
|||
|
|
|
|||
|
|
```rust
|
|||
|
|
pub struct RRFConfig {
|
|||
|
|
pub k: f32, // 60.0 (constant, don't tune)
|
|||
|
|
pub retrieve_k: usize, // 50 (top-K from each engine)
|
|||
|
|
pub final_k: usize, // 10 (return top-K)
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
pub struct RRFFusion { config: RRFConfig }
|
|||
|
|
|
|||
|
|
impl RRFFusion {
|
|||
|
|
pub fn fuse(
|
|||
|
|
&self,
|
|||
|
|
semantic: Vec<(String, f32)>, // (chunk_id, score) — sorted by score desc
|
|||
|
|
lexical: Vec<(String, f32)>,
|
|||
|
|
) -> Vec<(String, f32)>; // (chunk_id, rrf_score) — sorted desc, truncated
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**Invariants:**
|
|||
|
|
- Input lists must be pre-sorted by score descending (rank = position).
|
|||
|
|
- Output is sorted by RRF score descending.
|
|||
|
|
- Output length ≤ `final_k`.
|
|||
|
|
- A document appearing in both lists always outranks one appearing in only one (given same rank positions).
|
|||
|
|
|
|||
|
|
## Steps
|
|||
|
|
|
|||
|
|
1. Implement `RRFFusion::fuse()`.
|
|||
|
|
2. Implement `RRFFusion::normalize_scores()` as utility (for optional weighted-linear fallback).
|
|||
|
|
3. Write tests: basic fusion, single-engine input, identical lists, disjoint lists, empty inputs.
|
|||
|
|
|
|||
|
|
## Acceptance
|
|||
|
|
|
|||
|
|
1. `fuse([(a,0.9),(b,0.8)], [(a,8.0),(c,7.0)])` → `a` is rank 1 (appears in both lists).
|
|||
|
|
2. `fuse([(a,0.9)], [])` → `a` is rank 1 with score `1/61`.
|
|||
|
|
3. `fuse([], [])` → empty result.
|
|||
|
|
4. `fuse([(a,0.9),(b,0.8)], [(b,8.0),(a,7.0)])` → `a` and `b` have equal RRF scores (both appear in both at same combined rank sum). Either order is acceptable.
|
|||
|
|
5. Output length never exceeds `final_k`.
|
|||
|
|
|
|||
|
|
## Verify
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cargo test -p mem-cli rrf -- --nocapture
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**False pass:** Fusion returns results but sorted by original score, not RRF score. Verify by checking that a document ranked #3 in semantic but #1 in lexical outranks a document ranked #1 in semantic but absent from lexical.
|
|||
|
|
|
|||
|
|
## Artifacts
|
|||
|
|
|
|||
|
|
- `crates/mem-cli/src/query_optimizer.rs` (RRFFusion struct, lives alongside QueryOptimizer)
|