diff --git a/memory-flow.md b/memory-flow.md index b6a21ba..b2ffe46 100644 --- a/memory-flow.md +++ b/memory-flow.md @@ -646,6 +646,182 @@ spec: ``` +## M3.7.7 → M3.7.8: Failure Diagnosis Pipeline + +### M3.7.7 Complete: Signature Extraction (✅ 18 tests passing) + +**Purpose:** Reduce failure logs to canonical signatures that are byte-identical across runs. + +``` +Failure Log (50KB, noisy): + 2026-08-21T10:02:11.482Z + /home/runner/work/Poimen/memory/... + npm ERR! code ERESOLVE unable to resolve dependency tree + npm ERR! [... 100 lines of consequence errors ...] + │ + ↓ [M3.7.7: extract() + normalise()] + │ +Signature: + tool: "npm" + raw: "npm ERR! code ERESOLVE unable to resolve dependency tree" + normalised: "npm error eresolve dependency tree" + sig_sha: "abc123def456..." ← deterministic hash + rule: "npm_error_line" +``` + +**Key invariants (all 18 tests verify):** +- ✅ Same failure from 2 different runs → identical sig_sha +- ✅ Different failures → different sig_sha +- ✅ Removes timestamps, paths, SHAs, line:col, durations, addresses +- ✅ Picks first (root) error, not consequence lines +- ✅ Unknown tools fallback gracefully +- ✅ Tool part of signature identity (npm vs cargo errors differ) + +--- + +### M3.7.8 In Progress: Symptom Projection + +**Purpose:** Transform user queries into normalized symptom vectors that can match extracted signatures. + +**The Problem:** +``` +User reports: "npm can't find tslib module" +Canonical sig: "npm ERR! code ERESOLVE unable to resolve dependency tree" + +These don't hash the same way, so tier 1 (exact signature match) fails. +Symptom projection bridges this gap. +``` + +**Design (3-stage normalization pipeline):** + +#### Stage 1: Extract Symptom Keywords +```rust +Input query: "npm error: unable to resolve dependency tree" + │ + ├─ Tool detection: "npm" (from query or context) + ├─ Error pattern extraction: ["unable", "resolve", "dependency"] + ├─ Stop word removal: [unable, to, the, of, ...] + └─ Normalize abbreviations: ERESOLVE → "resolve error" + ERR → "error" + RC → "return code" + OOM → "out of memory" + EACCES → "permission denied" + ENOENT → "not found" + │ + ↓ +Keywords: ["npm", "error", "resolve", "dependency", "tree"] +``` + +#### Stage 2: Normalize to Canonical Form +```rust +Keywords: ["npm", "error", "resolve", "dependency", "tree"] + │ + ├─ Remove stop words (a, the, is, can, may, etc.) + ├─ Lowercase & stemming (resolved → resolve) + ├─ Expand shorthands: + │ - ERESOLVE → "resolve error" + │ - ERR → "error" + │ - EOF → "end of file" + │ - EACCES → "permission denied" + │ + └─ Tool mapping (npm, cargo, kubectl, etc.) + (different tools handle same error differently) + │ + ↓ +Normalized: "npm error resolve dependency tree" +``` + +#### Stage 3: Generate Symptom Hash +```rust +Normalized: "npm error resolve dependency tree" + │ + ├─ Sort keywords alphabetically (for consistency) + │ → "dependency error npm resolve tree" + │ + ├─ Join with spaces + ├─ SHA256 hash (same algorithm as M3.7.7) + │ sym_sha = SHA256("npm" + "\n" + normalized) + │ + └─ Store: SymptomVector { tool, normalised, sym_sha } + │ + ├─ sym_sha: "xyz789abc..." ← for lookup + ├─ normalised: "dependency error npm resolve tree" + └─ keywords: ["dependency", "error", "npm", "resolve", "tree"] +``` + +**Key insight:** If extracted signature and user query normalize to the same `sym_sha`, they match (tier 1 exact hit). If not, fall through to tier 2 (hybrid search). + +--- + +### M3.7.4: Three-Tier Context Endpoint Integration + +``` +GET /memory/context?query=npm+ERR+ERESOLVE&tool=npm + │ + │ Normalize with M3.7.8: SymptomVector + │ + ├─ TIER 1 (Exact): sym_sha lookup + │ ├─ Query: SELECT * FROM lessons WHERE sym_sha = ? + │ ├─ Hit: Return past solution (cached in memory) + │ └─ Miss: Continue to Tier 2 + │ + ├─ TIER 2 (Semantic/Hybrid): M8 search + │ ├─ Query: POST /memory/query (hybrid: 60% pgvector + 40% OpenSearch) + │ ├─ Results: Top-10 chunks ranked by similarity + │ └─ Confidence: med-high (vector match) + │ + └─ TIER 3 (Reference): M3.6 documentation + ├─ Query: Reference corpus (kubectl docs, npm docs) + ├─ Results: General guidance (not specific solution) + └─ Confidence: low (generic info) + +Response: +{ + "tier": 1, + "confidence": "high", + "results": [ + { + "source": "lesson", + "title": "Fix npm ERESOLVE errors", + "content": "npm ERR! code ERESOLVE...", + "solution": "Run npm ci instead of npm install" + } + ] +} +``` + +--- + +### M3.7.8 Implementation Plan + +**Files to create:** +1. `crates/mem-core/src/symptom_projection.rs` (250 LOC) + - `project_symptom(tool: &str, query: &str) -> SymptomVector` + - `normalize_query(text: &str) -> String` + - `abbrev_expand(word: &str) -> String` + - `stop_words()` — predefined list + - Abbreviation mappings per tool + +2. `tests/it_symptom_projection.rs` (400 LOC, 6 assertions) + - `a1_same_symptom_same_hash` — query variants → same sym_sha + - `a2_abbrev_expansion` — ERESOLVE, ERR, OOM normalize correctly + - `a3_stop_word_removal` — "unable to resolve" → "resolve" + - `a4_tool_consistency` — tool name included in sym_sha + - `a5_case_insensitive` — "NPM" = "npm" + - `a6_keyword_order_irrelevant` — sorted before hashing + +**Test fixtures:** +- 6 query examples (npm, cargo, kubectl) with expected normalized form +- Abbreviation expansions per tool + +**Integration with M3.7.4 endpoint:** +- M3.7.4 calls `symptom_projection::project_symptom(tool, user_query)` +- Returns `SymptomVector { sym_sha, normalised, keywords }` +- Looks up sym_sha in lesson cache +- If miss: delegates to M8 hybrid search + +--- + ## Pod Infrastructure Complete pod inventory deployed in `poimen` namespace.