docs: add M3.7.7 → M3.7.8 failure diagnosis pipeline design to memory-flow.md

memory-flow.md

@@ -646,6 +646,182 @@ spec:
 ```
 
 
+## M3.7.7 → M3.7.8: Failure Diagnosis Pipeline
+
+### M3.7.7 Complete: Signature Extraction (✅ 18 tests passing)
+
+**Purpose:** Reduce failure logs to canonical signatures that are byte-identical across runs.
+
+```
+Failure Log (50KB, noisy):
+  2026-08-21T10:02:11.482Z
+  /home/runner/work/Poimen/memory/...
+  npm ERR! code ERESOLVE unable to resolve dependency tree
+  npm ERR! [... 100 lines of consequence errors ...]
+              │
+              ↓ [M3.7.7: extract() + normalise()]
+              │
+Signature:
+  tool: "npm"
+  raw: "npm ERR! code ERESOLVE unable to resolve dependency tree"
+  normalised: "npm error eresolve dependency tree"
+  sig_sha: "abc123def456..." ← deterministic hash
+  rule: "npm_error_line"
+```
+
+**Key invariants (all 18 tests verify):**
+- ✅ Same failure from 2 different runs → identical sig_sha
+- ✅ Different failures → different sig_sha
+- ✅ Removes timestamps, paths, SHAs, line:col, durations, addresses
+- ✅ Picks first (root) error, not consequence lines
+- ✅ Unknown tools fallback gracefully
+- ✅ Tool part of signature identity (npm vs cargo errors differ)
+
+---
+
+### M3.7.8 In Progress: Symptom Projection
+
+**Purpose:** Transform user queries into normalized symptom vectors that can match extracted signatures.
+
+**The Problem:**
+```
+User reports:  "npm can't find tslib module"
+Canonical sig: "npm ERR! code ERESOLVE unable to resolve dependency tree"
+
+These don't hash the same way, so tier 1 (exact signature match) fails.
+Symptom projection bridges this gap.
+```
+
+**Design (3-stage normalization pipeline):**
+
+#### Stage 1: Extract Symptom Keywords
+```rust
+Input query: "npm error: unable to resolve dependency tree"
+              │
+              ├─ Tool detection: "npm" (from query or context)
+              ├─ Error pattern extraction: ["unable", "resolve", "dependency"]
+              ├─ Stop word removal: [unable, to, the, of, ...]
+              └─ Normalize abbreviations: ERESOLVE → "resolve error"
+                                         ERR → "error"
+                                         RC → "return code"
+                                         OOM → "out of memory"
+                                         EACCES → "permission denied"
+                                         ENOENT → "not found"
+              │
+              ↓
+Keywords: ["npm", "error", "resolve", "dependency", "tree"]
+```
+
+#### Stage 2: Normalize to Canonical Form
+```rust
+Keywords: ["npm", "error", "resolve", "dependency", "tree"]
+          │
+          ├─ Remove stop words (a, the, is, can, may, etc.)
+          ├─ Lowercase & stemming (resolved → resolve)
+          ├─ Expand shorthands:
+          │   - ERESOLVE → "resolve error"
+          │   - ERR → "error"
+          │   - EOF → "end of file"
+          │   - EACCES → "permission denied"
+          │
+          └─ Tool mapping (npm, cargo, kubectl, etc.)
+             (different tools handle same error differently)
+              │
+              ↓
+Normalized: "npm error resolve dependency tree"
+```
+
+#### Stage 3: Generate Symptom Hash
+```rust
+Normalized: "npm error resolve dependency tree"
+            │
+            ├─ Sort keywords alphabetically (for consistency)
+            │  → "dependency error npm resolve tree"
+            │
+            ├─ Join with spaces
+            ├─ SHA256 hash (same algorithm as M3.7.7)
+            │  sym_sha = SHA256("npm" + "\n" + normalized)
+            │
+            └─ Store: SymptomVector { tool, normalised, sym_sha }
+                      │
+                      ├─ sym_sha: "xyz789abc..." ← for lookup
+                      ├─ normalised: "dependency error npm resolve tree"
+                      └─ keywords: ["dependency", "error", "npm", "resolve", "tree"]
+```
+
+**Key insight:** If extracted signature and user query normalize to the same `sym_sha`, they match (tier 1 exact hit). If not, fall through to tier 2 (hybrid search).
+
+---
+
+### M3.7.4: Three-Tier Context Endpoint Integration
+
+```
+GET /memory/context?query=npm+ERR+ERESOLVE&tool=npm
+   │
+   │ Normalize with M3.7.8: SymptomVector
+   │
+   ├─ TIER 1 (Exact): sym_sha lookup
+   │  ├─ Query: SELECT * FROM lessons WHERE sym_sha = ?
+   │  ├─ Hit: Return past solution (cached in memory)
+   │  └─ Miss: Continue to Tier 2
+   │
+   ├─ TIER 2 (Semantic/Hybrid): M8 search
+   │  ├─ Query: POST /memory/query (hybrid: 60% pgvector + 40% OpenSearch)
+   │  ├─ Results: Top-10 chunks ranked by similarity
+   │  └─ Confidence: med-high (vector match)
+   │
+   └─ TIER 3 (Reference): M3.6 documentation
+      ├─ Query: Reference corpus (kubectl docs, npm docs)
+      ├─ Results: General guidance (not specific solution)
+      └─ Confidence: low (generic info)
+
+Response:
+{
+  "tier": 1,
+  "confidence": "high",
+  "results": [
+    {
+      "source": "lesson",
+      "title": "Fix npm ERESOLVE errors",
+      "content": "npm ERR! code ERESOLVE...",
+      "solution": "Run npm ci instead of npm install"
+    }
+  ]
+}
+```
+
+---
+
+### M3.7.8 Implementation Plan
+
+**Files to create:**
+1. `crates/mem-core/src/symptom_projection.rs` (250 LOC)
+   - `project_symptom(tool: &str, query: &str) -> SymptomVector`
+   - `normalize_query(text: &str) -> String`
+   - `abbrev_expand(word: &str) -> String`
+   - `stop_words()` — predefined list
+   - Abbreviation mappings per tool
+
+2. `tests/it_symptom_projection.rs` (400 LOC, 6 assertions)
+   - `a1_same_symptom_same_hash` — query variants → same sym_sha
+   - `a2_abbrev_expansion` — ERESOLVE, ERR, OOM normalize correctly
+   - `a3_stop_word_removal` — "unable to resolve" → "resolve"
+   - `a4_tool_consistency` — tool name included in sym_sha
+   - `a5_case_insensitive` — "NPM" = "npm"
+   - `a6_keyword_order_irrelevant` — sorted before hashing
+
+**Test fixtures:**
+- 6 query examples (npm, cargo, kubectl) with expected normalized form
+- Abbreviation expansions per tool
+
+**Integration with M3.7.4 endpoint:**
+- M3.7.4 calls `symptom_projection::project_symptom(tool, user_query)`
+- Returns `SymptomVector { sym_sha, normalised, keywords }`
+- Looks up sym_sha in lesson cache
+- If miss: delegates to M8 hybrid search
+
+---
+
 ## Pod Infrastructure
 
 Complete pod inventory deployed in `poimen` namespace.