From d8173f6bcd8b09a173ae428a7689875f6b2a92d7 Mon Sep 17 00:00:00 2001
From: Story Crater Bot <19826264+Riotpiaole@users.noreply.github.com>
Date: Fri, 28 Aug 2026 07:50:12 -0700
Subject: [PATCH] docs: add M3.7 failure diagnosis pipeline complete design
 guide

---
 docs/M3.7-FAILURE_DIAGNOSIS_SUMMARY.md | 364 +++++++++++++++++++++++++
 1 file changed, 364 insertions(+)
 create mode 100644 docs/M3.7-FAILURE_DIAGNOSIS_SUMMARY.md

diff --git a/docs/M3.7-FAILURE_DIAGNOSIS_SUMMARY.md b/docs/M3.7-FAILURE_DIAGNOSIS_SUMMARY.md
new file mode 100644
index 0000000..28d5a40
--- /dev/null
+++ b/docs/M3.7-FAILURE_DIAGNOSIS_SUMMARY.md
@@ -0,0 +1,364 @@
+# M3.7 Failure Diagnosis Pipeline — Complete Design
+
+**Phase:** M3.7 — Tool context  
+**Status:** M3.7.7 ✅ Complete | M3.7.8 🔄 In design | M3.7.4 ⬜ Pending M8.2  
+**Last updated:** 2026-08-28
+
+---
+
+## Overview
+
+M3.7 answers: **"What do we already know about this failure or tool?"** over HTTP.
+
+Provides three-tier context lookups with increasing cost and falling confidence:
+
+1. **Tier 1 (Exact)** — Exact signature match (past solution)
+2. **Tier 2 (Semantic)** — Hybrid search (similar cases)
+3. **Tier 3 (Reference)** — Documentation (general guidance)
+
+---
+
+## Pipeline Architecture
+
+```
+User Query
+  │ (e.g., "npm ERESOLVE error resolving typescript")
+  │
+  ├─────────────────────────────────────────────────────────────────┐
+  │                                                                  │
+  │ LAYER 1: SYMPTOM PROJECTION (M3.7.8)                            │
+  │ ├─ Normalize query to symptom vector                            │
+  │ ├─ Extract keywords, expand abbreviations (ERESOLVE → error)    │
+  │ ├─ Generate deterministic hash (sym_sha)                        │
+  │ └─ Output: SymptomVector { tool, normalised, sym_sha }          │
+  │                                                                  │
+  └──────────────────────┬──────────────────────────────────────────┘
+                         │
+                         ↓
+  ┌─────────────────────────────────────────────────────────────────┐
+  │                                                                  │
+  │ LAYER 2: SIGNATURE MATCHING (M3.7.7 + M3.7.8)                   │
+  │ ├─ Query DB: lessons WHERE sym_sha = ?                          │
+  │ ├─ If hit → TIER 1: Return cached solution                      │
+  │ └─ If miss → Continue to LAYER 3                                │
+  │                                                                  │
+  └──────────────────────┬──────────────────────────────────────────┘
+                         │ (Tier 1 miss)
+                         ↓
+  ┌─────────────────────────────────────────────────────────────────┐
+  │                                                                  │
+  │ LAYER 3: HYBRID SEARCH (M8)                                     │
+  │ ├─ Embed query (semantic, 60% weight)                           │
+  │ ├─ Search pgvector (cosine similarity)                          │
+  │ ├─ Search OpenSearch (BM25, 40% weight)                         │
+  │ ├─ Fuse results (weighted linear: 0.6*sem + 0.4*lex)            │
+  │ └─ TIER 2: Return top-10 ranked results                         │
+  │                                                                  │
+  └──────────────────────┬──────────────────────────────────────────┘
+                         │ (Tier 2 miss)
+                         ↓
+  ┌─────────────────────────────────────────────────────────────────┐
+  │                                                                  │
+  │ LAYER 4: REFERENCE CORPUS (M3.6)                                │
+  │ ├─ Query reference docs (kubectl, npm, etc.)                    │
+  │ └─ TIER 3: Return general guidance (lowest confidence)          │
+  │                                                                  │
+  └─────────────────────────────────────────────────────────────────┘
+
+Response (M3.7.4 context endpoint):
+{
+  "tier": 1,
+  "confidence": "high",
+  "query_normalised": "npm error resolve dependency typescript",
+  "results": [
+    {
+      "source": "lesson",
+      "title": "Fix npm ERESOLVE errors",
+      "solution": "npm ci (deterministic); npm install (latest versions)"
+    }
+  ]
+}
+```
+
+---
+
+## Task Breakdown
+
+### M3.7.7 ✅ COMPLETE — Failure Signature Extraction
+
+**Commit:** `463958b`  
+**Status:** 18/18 unit tests passing  
+**Implementation:** `crates/mem-core/src/lesson.rs` (871 LOC)
+
+#### What it does:
+- Extracts root error from 50KB failure logs
+- Normalizes timestamps, paths, SHAs, addresses
+- Generates deterministic SHA256 hash (sig_sha)
+- Handles cascading failures (picks root cause, not consequence)
+
+#### Example:
+```
+Raw log (50KB):
+  2026-08-21T10:02:11.482Z
+  /home/runner/work/Poimen/memory/k8s/app/opensearch.yaml
+  error: error validating data: [ValidationError(...)]
+  The server is rejecting the request. (422)
+  [... 100 lines of structured output ...]
+
+↓ [M3.7.7 extract()]
+
+Signature:
+  tool: "kubectl"
+  raw: "error: error validating data: [ValidationError(...)]"
+  normalised: "error validating data kubernetes"
+  sig_sha: "abc123def456789..." ← deterministic
+  rule: "kubectl_error_line"
+```
+
+#### Tests (18 passing):
+- ✅ Same failure (2 runs) → identical hash
+- ✅ Different failures → different hashes
+- ✅ Removes ANSI, timestamps, paths, SHAs
+- ✅ Picks first error (cascade suppression)
+- ✅ Unknown tools fallback gracefully
+- ✅ Tool name part of identity
+- ✅ Similar wording still matches
+- ✅ + 11 more (see `cargo test -p mem-core lesson`)
+
+---
+
+### M3.7.8 🔄 IN DESIGN — Symptom Projection
+
+**Status:** Full design complete (see `docs/M3.7.8-SYMPTOM_PROJECTION.md`)  
+**Estimated:** 1–2 days  
+**Implementation Plan:** 250 LOC + 6 test assertions
+
+#### What it does:
+Transforms **user queries** into normalized **symptom vectors** that can match extracted signatures.
+
+**Three-stage pipeline:**
+
+**Stage 1: Extract Keywords**
+```
+Query: "npm can't find tslib module error"
+↓
+Keywords: ["npm", "find", "tslib", "module", "error"]
+```
+
+**Stage 2: Normalize**
+```
+Keywords: ["npm", "find", "tslib", "module", "error"]
+↓ [Remove stop words: can, find (weak), t]
+Normalized: "error module npm tslib"
+↓ [Sort alphabetically for idempotence]
+Canonical: "error module npm tslib"
+```
+
+**Stage 3: Generate Hash**
+```
+Canonical: "error module npm tslib"
+↓ [SHA256(tool + "\n" + normalized)]
+sym_sha: "xyz789abc..." ← matches M3.7.7 signature if similar
+```
+
+#### Example Match:
+```
+Extracted signature (M3.7.7):
+  normalised: "error module tslib"
+  sig_sha: "abc123..."
+
+User query (M3.7.8):
+  query: "npm cannot resolve tslib"
+  normalised: "error module npm tslib" (after expand "cannot")
+  sym_sha: "abc123..." ← MATCH!
+  
+→ Tier 1 hit: Return past solution
+```
+
+#### Tests (6 assertions):
+1. Same symptom variant → same hash
+2. Abbreviation expansion (ERESOLVE, ERR, OOM, EACCES)
+3. Stop word removal (can, the, able, to)
+4. Tool consistency (npm != cargo for same error)
+5. Case insensitive (NPM = npm)
+6. Keyword order irrelevant (sorted before hash)
+
+#### Files to create:
+- `crates/mem-core/src/symptom_projection.rs` (250 LOC)
+- `tests/it_symptom_projection.rs` (400 LOC, 6 assertions)
+- `fixtures/symptoms/` (test query examples)
+
+---
+
+### M3.7.4 ⬜ PENDING — Context Endpoint Integration
+
+**Status:** Waiting for M3.7.8 + M8.2 (hybrid search live)  
+**Purpose:** HTTP endpoint gluing tiers 1–3 together  
+**Endpoint:** `GET /memory/context?query=...&tool=...`
+
+#### Three-tier logic:
+```rust
+pub async fn get_context(query: &str, tool: Option<&str>) -> ContextResult {
+    // TIER 1: Exact signature match
+    let symptom = project_symptom(tool.unwrap_or("unknown"), query); // M3.7.8
+    if let Some(lesson) = find_lesson_by_sym_sha(&symptom.sym_sha) {
+        return high_confidence_result(lesson);
+    }
+    
+    // TIER 2: Hybrid search (requires M8.2)
+    let hybrid = hybrid_search(query, tool).await?;
+    if !hybrid.is_empty() {
+        return medium_confidence_result(hybrid);
+    }
+    
+    // TIER 3: Reference corpus (requires M3.6)
+    let reference = reference_corpus_search(query, tool).await?;
+    return low_confidence_result(reference);
+}
+```
+
+#### Response format:
+```json
+{
+  "tier": 1,
+  "confidence": "high",
+  "query_normalised": "error module npm tslib",
+  "results": [
+    {
+      "source": "lesson",
+      "title": "Fix npm module not found",
+      "keywords": ["npm", "error", "module"],
+      "solution": "Check package.json, npm install",
+      "applies_to": ["npm", "yarn"],
+      "severity": "medium"
+    }
+  ]
+}
+```
+
+---
+
+### M3.7.6 ⬜ PENDING — Composition Gate
+
+**Status:** Depends on M3.7.4 + M3.7.8  
+**Purpose:** Verify tiers work end-to-end (accuracy, latency, coverage)
+
+#### Gate assertions:
+1. Tier 1 queries resolve in < 50ms (cached lookup)
+2. Tier 2 queries resolve in < 500ms (hybrid search)
+3. Tier 3 queries resolve in < 1000ms (reference corpus)
+4. Exactly one tier returns results (no duplicates across tiers)
+5. Confidence scores monotonically decrease (tier 1 > 2 > 3)
+6. Coverage: 95% of real failures find a tier 1 or 2 match
+
+---
+
+## Retired Tasks
+
+✅ **M3.7.3** — `GET /memory/skills?task=...` (skill matching)  
+   → Redundant with M8 hybrid search  
+   → Removed: 1 task
+
+✅ **M3.7.5** — `tool-failures` standing query  
+   → Redundant with M8 hybrid search  
+   → Removed: 1 task
+
+**Outcome:** M3.7 reduced from 6 tasks → 4 tasks (gain: 2 simplified, no loss)
+
+---
+
+## Integration Points
+
+| Component | Depends On | Used By |
+|-----------|-----------|---------|
+| M3.7.7 (Signature) | M3.6.1 (DocCorpus) | M3.7.8 (Symptom) |
+| M3.7.8 (Symptom) | M3.7.7 (Signature) | M3.7.4 (Context endpoint) |
+| M3.7.4 (Context) | M3.7.8 + M8.2 + M3.6 | M3.7.6 (Gate) |
+| M3.7.6 (Gate) | M3.7.4 | — |
+
+**Critical path:**
+```
+M3.7.7 ✅ → M3.7.8 🔄 → M3.7.4 ⬜
+                           ↓ [waits for M8.2]
+                        M3.7.6 ⬜
+```
+
+---
+
+## Performance Targets
+
+| Operation | Latency | Notes |
+|-----------|---------|-------|
+| M3.7.7: Extract signature | < 50ms | 50KB log → hash (rule-based) |
+| M3.7.8: Project symptom | < 10ms | Query → normalized vector |
+| Tier 1 lookup (M3.7.4) | < 50ms | DB hash lookup |
+| Tier 2 lookup (M3.7.4) | < 500ms | Hybrid search (parallel engines) |
+| Tier 3 lookup (M3.7.4) | < 1000ms | Reference corpus search |
+
+---
+
+## Success Criteria
+
+### M3.7.7 ✅
+- [x] 18 unit tests passing (signature extraction)
+- [x] Deterministic hashing (same failure → same hash)
+- [x] CLI command: `mem sig explain`
+- [x] Fixtures: real captured logs (npm, cargo, kubectl, etc.)
+
+### M3.7.8 🔄
+- [ ] 6 integration tests passing (symptom projection)
+- [ ] Deterministic hashing (same query variant → same hash)
+- [ ] Abbreviation expansion per tool (ERESOLVE, ERR, EACCES, etc.)
+- [ ] Integrated with M3.7.4 context endpoint
+- [ ] Tier 1 lookups work end-to-end
+
+### M3.7.4 ⬜
+- [ ] HTTP endpoint operational
+- [ ] Three-tier logic working
+- [ ] Response includes `tier`, `confidence` fields
+- [ ] Integrated with M8.2 (hybrid search)
+
+### M3.7.6 ⬜
+- [ ] Gate assertions passing (latency, coverage, accuracy)
+- [ ] End-to-end failure diagnosis working
+
+---
+
+## Documents
+
+- **Core Implementation:** `crates/mem-core/src/lesson.rs` (M3.7.7, 871 LOC)
+- **M3.7.8 Design:** `docs/M3.7.8-SYMPTOM_PROJECTION.md` (this guide, 14KB)
+- **Pipeline Flow:** `memory-flow.md` § "M3.7.7 → M3.7.8: Failure Diagnosis Pipeline" (176 lines)
+- **Tests:** `tests/it_signature.rs` (M3.7.7, 9 assertions)
+
+---
+
+## Timeline
+
+**Completed:** M3.7.7 (18 unit tests passing)  
+**Next (1–2 days):** M3.7.8 symptom projection  
+**Then (1 day):** M3.7.4 context endpoint  
+**Finally (1 day):** M3.7.6 composition gate
+
+**Total M3.7:** ~5 days (1 done, 4 remaining)
+
+---
+
+## Team Notes
+
+- **Determinism is critical:** Both M3.7.7 and M3.7.8 must hash identically across runs
+  - Use sorted keywords (not insertion order)
+  - Include tool name in identity
+  - Strip all volatile data (timestamps, addresses, etc.)
+
+- **Tier 1 only fires if M3.7.8 sym_sha matches M3.7.7 sig_sha**
+  - If query is ambiguous or doesn't match any signature → skip to tier 2
+  - Fall back to hybrid search (tier 2) gracefully
+
+- **M8.2 is a blocker for M3.7.4**
+  - Hybrid search must be live before context endpoint can work
+  - (Tier 2 fallback requires functional search)
+
+---
+
+**End of M3.7 summary.**