184 lines
8.3 KiB
Markdown
184 lines
8.3 KiB
Markdown
# M3.7.4 — `/memory/context` — three-tier lookup
|
||
|
||
| Field | Value |
|
||
|---|---|
|
||
| Phase | M3.7 — Tool context |
|
||
| Size | M — 1–3 days |
|
||
| Status | ⬜ Not started |
|
||
| Flags | — |
|
||
| Spec | inlined below |
|
||
| Blocks | M3.7.6 |
|
||
| Depends | M3.7.3, M3.7.7, M3.7.8, M3.5.3, M3.6.5 |
|
||
|
||
## Goal
|
||
|
||
One call that answers “what do we already know about this failure, tool or task”,
|
||
cheapest tier first, and says which tier the answer came from.
|
||
|
||
## Existing code to build on
|
||
|
||
**`crates/mem-core/src/lesson.rs`** already implements the tier-1 and tier-2 lookup pattern:
|
||
|
||
| Function | What it does | Reuse plan |
|
||
|---|---|---|
|
||
| `lookup(sig, lessons, floor)` | Exact hash match (tier 1) then trigram similarity (tier 2) with abstention floor | **Extend**: add vector search (M2.1) as tier 2, keep trigrams as offline fallback |
|
||
| `Tier` enum (`Exact`, `Similar(f32)`) | Tier labeling | **Extend**: add `Reference` variant for tier 3 |
|
||
| `Hit` struct | `{ lesson, tier }` | **Extend**: generalize from `Lesson` to `MemoryNode` |
|
||
| `similarity(a, b)` | Jaccard over character trigrams | **Keep** as fallback when embeddings unavailable |
|
||
| `render_injection(hit, max_chars)` | Capped injection text | **Reuse** for budget management |
|
||
|
||
**`crates/mem-cli/src/lessons_cmd.rs`** already implements:
|
||
|
||
| Command | What it does | Reuse plan |
|
||
|---|---|---|
|
||
| `mem lookup --tool T --file F` | CLI tier-based lookup with floor | **Model for** the HTTP endpoint |
|
||
|
||
## Files
|
||
|
||
| Action | Path |
|
||
|---|---|
|
||
| **Exists** | `crates/mem-core/src/lesson.rs` — `lookup()`, `Tier`, `Hit`, `similarity()` |
|
||
| **Exists** | `crates/mem-cli/src/lessons_cmd.rs` — `mem lookup` CLI |
|
||
| Create | HTTP endpoint in `mem-api` crate (M3.5.1 server) |
|
||
| Modify | `crates/mem-core/src/lesson.rs` — extend `Tier` enum with `Reference` variant |
|
||
| Create | `tests/it_context_endpoint.rs` — integration tests (12 assertions) |
|
||
|
||
## Facts (inlined — no spec read needed)
|
||
|
||
```
|
||
POST /memory/context
|
||
{ "tool": "github-actions",
|
||
"signature_source": "<50KB run log>",
|
||
"project": "homelab", "scope": "all-projects", "budget": 6000 }
|
||
|
||
→ 200 {
|
||
"tier": 1,
|
||
"lessons": [
|
||
{"tier":1,"level":"L1","seen_count":3,"last_seen":"2026-07-02",
|
||
"text":"peer dep conflict @types/react 18 vs 19; npm ci --legacy-peer-deps
|
||
unblocks, real fix is pinning in overrides",
|
||
"parents":[{"level":"L0","source":"pi:…"}]},
|
||
{"tier":2,"level":"L2","score":0.71,"matched_kind":"symptom","text":"…"}
|
||
],
|
||
"skills": [{"name":"ci-triage","score":0.77}],
|
||
"budget": {"limit":6000,"used":2140,"dropped":[]}
|
||
}
|
||
```
|
||
|
||
Three accepted inputs, any combination: `tool`, `task`, `signature_source`. At
|
||
least one is required; `signature_source` without `tool` is allowed and the tool
|
||
is inferred by the extractor's rule match.
|
||
|
||
**Tiers, cheapest first:**
|
||
|
||
| Tier | Mechanism | Meaning |
|
||
|---|---|---|
|
||
| 1 | `sig_sha` primary-key hit on `failure_signature` | this exact failure happened here before |
|
||
| 2 | vector over `kind='symptom'` then `kind='text'`, reranked | something similar happened |
|
||
| 3 | R reference corpus | nobody here has hit this; here are the docs |
|
||
|
||
Tier 1 does not short-circuit tiers 2 and 3 — it *leads*. An exact hit plus two
|
||
related memories is a better answer than an exact hit alone, and the tiers cost
|
||
milliseconds relative to the caller's own inference.
|
||
|
||
**`tier` is in the response and this matters.** The caller, and the human reading
|
||
its output, must be able to tell "we hit this exact error in July" from "here is
|
||
what the manual says". Presenting tier 3 in the register of tier 1 is how a
|
||
retrieval system becomes untrustworthy.
|
||
|
||
**Ordering is by tier, then rerank, and never by raw score.** A cheatsheet is
|
||
written to match the phrasing of a question and will routinely out-score the
|
||
terse memory that actually solved it. Precedence is a rule: tier 1 > L1/L2 > R.
|
||
|
||
**Scope defaults differ by tier.** Signature and symptom lookups federate across
|
||
projects — `ERESOLVE` is not homelab-specific — while task-shaped queries stay
|
||
project-scoped unless asked otherwise. Project match becomes a rank boost, not a
|
||
filter.
|
||
|
||
**Superseded memories are excluded, not demoted.** A lesson about Kong config is
|
||
wrong now, not merely old. M2.4 filters them at the repository; this endpoint
|
||
surfaces the successor if one is linked.
|
||
|
||
**Budget, fixed truncation order:** drop R, then trim tier-2 results toward the
|
||
floor, then drop skills. Tier-1 hits are never dropped — they are the smallest and
|
||
most valuable content in the response.
|
||
|
||
**Legs degrade independently.** A skills timeout returns `"skills":[]` with a
|
||
`degraded` note and a 200. There is no leg whose failure justifies a 5xx here; a
|
||
thinner answer beats no answer when someone is mid-incident.
|
||
|
||
## Steps
|
||
|
||
1. Route in the M3.5.1 server; accept `tool`, `task`, `signature_source`.
|
||
2. If `signature_source` present, extract and normalise (M3.7.7), then tier 1.
|
||
3. Tier 2 concurrently: symptom-vector search, then text-vector, merge, rerank.
|
||
4. Tier 3 only if tiers 1–2 leave budget unfilled.
|
||
5. Skills leg (M3.7.3) concurrently with tier 2.
|
||
6. Order by tier, apply precedence, apply budget, record drops.
|
||
7. On tier-1 hit, append an occurrence record to the log so `seen_count` grows.
|
||
8. One structured log line: tiers fired, latencies, scores, drops.
|
||
|
||
## Acceptance
|
||
|
||
- A previously-seen failure returns `tier: 1` with `seen_count` ≥ 2.
|
||
- An unseen but similar failure returns tier 2 with `matched_kind: "symptom"`.
|
||
- A wholly unknown failure returns tier 3 and says so.
|
||
- Tier 1 outranks a higher-scoring R result.
|
||
- Identical requests return byte-identical bodies apart from the occurrence side
|
||
effect.
|
||
|
||
## Verify
|
||
|
||
**Harness:** M3.5.1 test server over a store seeded with the poimen log, a
|
||
reference corpus, promoted skills, and signatures from replayed real failures.
|
||
Fault injection per leg.
|
||
|
||
**Integration test** — `tests/it_context_endpoint.rs`:
|
||
1. `a1_tier1_exact` — replay a failure already in `failure_signature`; assert
|
||
`tier: 1` and the correct memory.
|
||
2. `a2_tier1_counts_occurrence` — call twice; assert `seen_count` incremented and
|
||
an occurrence record is in the log.
|
||
3. `a3_tier2_symptom` — a novel wording of a known incident returns tier 2 with
|
||
`matched_kind: "symptom"`.
|
||
4. `a4_tier2_beats_text_only` — same query with symptom vectors deleted ranks the
|
||
correct memory lower; assert the symptom path strictly improves it.
|
||
5. `a5_tier3_fallback` — an unknown failure returns tier 3 and no lesson claims a
|
||
lower tier.
|
||
6. `a6_precedence_over_score` — seed an R node that reranks above a tier-1 hit;
|
||
assert the tier-1 hit still leads and the response exposes both raw scores.
|
||
7. `a7_superseded_excluded` — mark a memory superseded; assert it is absent and
|
||
its successor is present.
|
||
8. `a8_budget_order` — shrink the budget stepwise; assert drops occur R, then
|
||
tier 2, then skills, and that tier 1 is never dropped.
|
||
9. `a9_skills_degrade` — inject a skills timeout; assert 200, `[]`, `degraded`.
|
||
10. `a10_signature_without_tool` — omit `tool`; assert the extractor infers it.
|
||
11. `a11_scope_federation` — a signature seeded under another project is found
|
||
with `scope: all-projects` and not with `scope: project`.
|
||
12. `a12_used_matches_actual` — tokenize the body independently; assert equality
|
||
with `budget.used`.
|
||
|
||
**Command:** `cargo test -p mem-api context_endpoint`
|
||
|
||
**False pass:**
|
||
- Asserting tier 1 fires without asserting tiers 2 and 3 still populate. A
|
||
short-circuit passes assertion 1 and produces a thin answer in exactly the case
|
||
where the most context is available.
|
||
- Testing precedence on a fixture where the tier-1 hit also scores highest.
|
||
Assertion 6 is only meaningful when score and tier disagree.
|
||
- Testing degradation with an empty leg instead of a failing one. Timeouts take
|
||
the path that 500s in production.
|
||
|
||
## Traps
|
||
|
||
- Running tier 3 unconditionally. Reference chunks are long, they fill the budget,
|
||
and they push real memories out of a response that had better answers available.
|
||
- Incrementing `seen_count` on a retry. The orchestrator retries activities; an
|
||
occurrence should key on the caller's request id or the count inflates and
|
||
`last_seen` stops meaning anything.
|
||
- Returning tier as a label the caller has to interpret from ordering. It is a
|
||
field; if it is implicit, every consumer reimplements the inference differently.
|
||
|
||
---
|
||
|
||
Background: [DESIGN.md](../DESIGN.md) — tool context, retrieval tiers
|