poimen-memory/tasks/M3.7.4-context-endpoint.md

# M3.7.4 — `/memory/context` — three-tier lookup

| Field | Value |
|---|---|
| Phase | M3.7 — Tool context |
| Size | M — 1–3 days |
| Status | ⬜ Not started |
| Flags | — |
| Spec | inlined below |
| Blocks | M3.7.6 |
| Depends | M3.7.3, M3.7.7, M3.7.8, M3.5.3, M3.6.5 |

## Goal

One call that answers “what do we already know about this failure, tool or task”,
cheapest tier first, and says which tier the answer came from.

## Existing code to build on

**`crates/mem-core/src/lesson.rs`** already implements the tier-1 and tier-2 lookup pattern:

| Function | What it does | Reuse plan |
|---|---|---|
| `lookup(sig, lessons, floor)` | Exact hash match (tier 1) then trigram similarity (tier 2) with abstention floor | **Extend**: add vector search (M2.1) as tier 2, keep trigrams as offline fallback |
| `Tier` enum (`Exact`, `Similar(f32)`) | Tier labeling | **Extend**: add `Reference` variant for tier 3 |
| `Hit` struct | `{ lesson, tier }` | **Extend**: generalize from `Lesson` to `MemoryNode` |
| `similarity(a, b)` | Jaccard over character trigrams | **Keep** as fallback when embeddings unavailable |
| `render_injection(hit, max_chars)` | Capped injection text | **Reuse** for budget management |

**`crates/mem-cli/src/lessons_cmd.rs`** already implements:

| Command | What it does | Reuse plan |
|---|---|---|
| `mem lookup --tool T --file F` | CLI tier-based lookup with floor | **Model for** the HTTP endpoint |

## Files

| Action | Path |
|---|---|
| **Exists** | `crates/mem-core/src/lesson.rs` — `lookup()`, `Tier`, `Hit`, `similarity()` |
| **Exists** | `crates/mem-cli/src/lessons_cmd.rs` — `mem lookup` CLI |
| Create | HTTP endpoint in `mem-api` crate (M3.5.1 server) |
| Modify | `crates/mem-core/src/lesson.rs` — extend `Tier` enum with `Reference` variant |
| Create | `tests/it_context_endpoint.rs` — integration tests (12 assertions) |

## Facts (inlined — no spec read needed)

```
POST /memory/context
{ "tool": "github-actions",
  "signature_source": "<50KB run log>",
  "project": "homelab", "scope": "all-projects", "budget": 6000 }

→ 200 {
  "tier": 1,
  "lessons": [
    {"tier":1,"level":"L1","seen_count":3,"last_seen":"2026-07-02",
     "text":"peer dep conflict @types/react 18 vs 19; npm ci --legacy-peer-deps
             unblocks, real fix is pinning in overrides",
     "parents":[{"level":"L0","source":"pi:…"}]},
    {"tier":2,"level":"L2","score":0.71,"matched_kind":"symptom","text":"…"}
  ],
  "skills": [{"name":"ci-triage","score":0.77}],
  "budget": {"limit":6000,"used":2140,"dropped":[]}
}
```

Three accepted inputs, any combination: `tool`, `task`, `signature_source`. At
least one is required; `signature_source` without `tool` is allowed and the tool
is inferred by the extractor's rule match.

**Tiers, cheapest first:**

| Tier | Mechanism | Meaning |
|---|---|---|
| 1 | `sig_sha` primary-key hit on `failure_signature` | this exact failure happened here before |
| 2 | vector over `kind='symptom'` then `kind='text'`, reranked | something similar happened |
| 3 | R reference corpus | nobody here has hit this; here are the docs |

Tier 1 does not short-circuit tiers 2 and 3 — it *leads*. An exact hit plus two
related memories is a better answer than an exact hit alone, and the tiers cost
milliseconds relative to the caller's own inference.

**`tier` is in the response and this matters.** The caller, and the human reading
its output, must be able to tell "we hit this exact error in July" from "here is
what the manual says". Presenting tier 3 in the register of tier 1 is how a
retrieval system becomes untrustworthy.

**Ordering is by tier, then rerank, and never by raw score.** A cheatsheet is
written to match the phrasing of a question and will routinely out-score the
terse memory that actually solved it. Precedence is a rule: tier 1 > L1/L2 > R.

**Scope defaults differ by tier.** Signature and symptom lookups federate across
projects — `ERESOLVE` is not homelab-specific — while task-shaped queries stay
project-scoped unless asked otherwise. Project match becomes a rank boost, not a
filter.

**Superseded memories are excluded, not demoted.** A lesson about Kong config is
wrong now, not merely old. M2.4 filters them at the repository; this endpoint
surfaces the successor if one is linked.

**Budget, fixed truncation order:** drop R, then trim tier-2 results toward the
floor, then drop skills. Tier-1 hits are never dropped — they are the smallest and
most valuable content in the response.

**Legs degrade independently.** A skills timeout returns `"skills":[]` with a
`degraded` note and a 200. There is no leg whose failure justifies a 5xx here; a
thinner answer beats no answer when someone is mid-incident.

## Steps

1. Route in the M3.5.1 server; accept `tool`, `task`, `signature_source`.
2. If `signature_source` present, extract and normalise (M3.7.7), then tier 1.
3. Tier 2 concurrently: symptom-vector search, then text-vector, merge, rerank.
4. Tier 3 only if tiers 1–2 leave budget unfilled.
5. Skills leg (M3.7.3) concurrently with tier 2.
6. Order by tier, apply precedence, apply budget, record drops.
7. On tier-1 hit, append an occurrence record to the log so `seen_count` grows.
8. One structured log line: tiers fired, latencies, scores, drops.

## Acceptance

- A previously-seen failure returns `tier: 1` with `seen_count` ≥ 2.
- An unseen but similar failure returns tier 2 with `matched_kind: "symptom"`.
- A wholly unknown failure returns tier 3 and says so.
- Tier 1 outranks a higher-scoring R result.
- Identical requests return byte-identical bodies apart from the occurrence side
  effect.

## Verify

**Harness:** M3.5.1 test server over a store seeded with the poimen log, a
reference corpus, promoted skills, and signatures from replayed real failures.
Fault injection per leg.

**Integration test** — `tests/it_context_endpoint.rs`:
1. `a1_tier1_exact` — replay a failure already in `failure_signature`; assert
   `tier: 1` and the correct memory.
2. `a2_tier1_counts_occurrence` — call twice; assert `seen_count` incremented and
   an occurrence record is in the log.
3. `a3_tier2_symptom` — a novel wording of a known incident returns tier 2 with
   `matched_kind: "symptom"`.
4. `a4_tier2_beats_text_only` — same query with symptom vectors deleted ranks the
   correct memory lower; assert the symptom path strictly improves it.
5. `a5_tier3_fallback` — an unknown failure returns tier 3 and no lesson claims a
   lower tier.
6. `a6_precedence_over_score` — seed an R node that reranks above a tier-1 hit;
   assert the tier-1 hit still leads and the response exposes both raw scores.
7. `a7_superseded_excluded` — mark a memory superseded; assert it is absent and
   its successor is present.
8. `a8_budget_order` — shrink the budget stepwise; assert drops occur R, then
   tier 2, then skills, and that tier 1 is never dropped.
9. `a9_skills_degrade` — inject a skills timeout; assert 200, `[]`, `degraded`.
10. `a10_signature_without_tool` — omit `tool`; assert the extractor infers it.
11. `a11_scope_federation` — a signature seeded under another project is found
    with `scope: all-projects` and not with `scope: project`.
12. `a12_used_matches_actual` — tokenize the body independently; assert equality
    with `budget.used`.

**Command:** `cargo test -p mem-api context_endpoint`

**False pass:**
- Asserting tier 1 fires without asserting tiers 2 and 3 still populate. A
  short-circuit passes assertion 1 and produces a thin answer in exactly the case
  where the most context is available.
- Testing precedence on a fixture where the tier-1 hit also scores highest.
  Assertion 6 is only meaningful when score and tier disagree.
- Testing degradation with an empty leg instead of a failing one. Timeouts take
  the path that 500s in production.

## Traps

- Running tier 3 unconditionally. Reference chunks are long, they fill the budget,
  and they push real memories out of a response that had better answers available.
- Incrementing `seen_count` on a retry. The orchestrator retries activities; an
  occurrence should key on the caller's request id or the count inflates and
  `last_seen` stops meaning anything.
- Returning tier as a label the caller has to interpret from ordering. It is a
  field; if it is implicit, every consumer reimplements the inference differently.

---

Background: [DESIGN.md](../DESIGN.md) — tool context, retrieval tiers