# M3.7 Failure Diagnosis Pipeline — Complete Design

**Phase:** M3.7 — Tool context  
**Status:** M3.7.7 ✅ Complete | M3.7.8 🔄 In design | M3.7.4 ⬜ Pending M8.2  
**Last updated:** 2026-08-28

---

## Overview

M3.7 answers: **"What do we already know about this failure or tool?"** over HTTP.

Provides three-tier context lookups with increasing cost and falling confidence:

1. **Tier 1 (Exact)** — Exact signature match (past solution)
2. **Tier 2 (Semantic)** — Hybrid search (similar cases)
3. **Tier 3 (Reference)** — Documentation (general guidance)

---

## Pipeline Architecture

```
User Query
  │ (e.g., "npm ERESOLVE error resolving typescript")
  │
  ├─────────────────────────────────────────────────────────────────┐
  │                                                                  │
  │ LAYER 1: SYMPTOM PROJECTION (M3.7.8)                            │
  │ ├─ Normalize query to symptom vector                            │
  │ ├─ Extract keywords, expand abbreviations (ERESOLVE → error)    │
  │ ├─ Generate deterministic hash (sym_sha)                        │
  │ └─ Output: SymptomVector { tool, normalised, sym_sha }          │
  │                                                                  │
  └──────────────────────┬──────────────────────────────────────────┘
                         │
                         ↓
  ┌─────────────────────────────────────────────────────────────────┐
  │                                                                  │
  │ LAYER 2: SIGNATURE MATCHING (M3.7.7 + M3.7.8)                   │
  │ ├─ Query DB: lessons WHERE sym_sha = ?                          │
  │ ├─ If hit → TIER 1: Return cached solution                      │
  │ └─ If miss → Continue to LAYER 3                                │
  │                                                                  │
  └──────────────────────┬──────────────────────────────────────────┘
                         │ (Tier 1 miss)
                         ↓
  ┌─────────────────────────────────────────────────────────────────┐
  │                                                                  │
  │ LAYER 3: HYBRID SEARCH (M8)                                     │
  │ ├─ Embed query (semantic, 60% weight)                           │
  │ ├─ Search pgvector (cosine similarity)                          │
  │ ├─ Search OpenSearch (BM25, 40% weight)                         │
  │ ├─ Fuse results (weighted linear: 0.6*sem + 0.4*lex)            │
  │ └─ TIER 2: Return top-10 ranked results                         │
  │                                                                  │
  └──────────────────────┬──────────────────────────────────────────┘
                         │ (Tier 2 miss)
                         ↓
  ┌─────────────────────────────────────────────────────────────────┐
  │                                                                  │
  │ LAYER 4: REFERENCE CORPUS (M3.6)                                │
  │ ├─ Query reference docs (kubectl, npm, etc.)                    │
  │ └─ TIER 3: Return general guidance (lowest confidence)          │
  │                                                                  │
  └─────────────────────────────────────────────────────────────────┘

Response (M3.7.4 context endpoint):
{
  "tier": 1,
  "confidence": "high",
  "query_normalised": "npm error resolve dependency typescript",
  "results": [
    {
      "source": "lesson",
      "title": "Fix npm ERESOLVE errors",
      "solution": "npm ci (deterministic); npm install (latest versions)"
    }
  ]
}
```

---

## Task Breakdown

### M3.7.7 ✅ COMPLETE — Failure Signature Extraction

**Commit:** `463958b`  
**Status:** 18/18 unit tests passing  
**Implementation:** `crates/mem-core/src/lesson.rs` (871 LOC)

#### What it does:
- Extracts root error from 50KB failure logs
- Normalizes timestamps, paths, SHAs, addresses
- Generates deterministic SHA256 hash (sig_sha)
- Handles cascading failures (picks root cause, not consequence)

#### Example:
```
Raw log (50KB):
  2026-08-21T10:02:11.482Z
  /home/runner/work/Poimen/memory/k8s/app/opensearch.yaml
  error: error validating data: [ValidationError(...)]
  The server is rejecting the request. (422)
  [... 100 lines of structured output ...]

↓ [M3.7.7 extract()]

Signature:
  tool: "kubectl"
  raw: "error: error validating data: [ValidationError(...)]"
  normalised: "error validating data kubernetes"
  sig_sha: "abc123def456789..." ← deterministic
  rule: "kubectl_error_line"
```

#### Tests (18 passing):
- ✅ Same failure (2 runs) → identical hash
- ✅ Different failures → different hashes
- ✅ Removes ANSI, timestamps, paths, SHAs
- ✅ Picks first error (cascade suppression)
- ✅ Unknown tools fallback gracefully
- ✅ Tool name part of identity
- ✅ Similar wording still matches
- ✅ + 11 more (see `cargo test -p mem-core lesson`)

---

### M3.7.8 🔄 IN DESIGN — Symptom Projection

**Status:** Full design complete (see `docs/M3.7.8-SYMPTOM_PROJECTION.md`)  
**Estimated:** 1–2 days  
**Implementation Plan:** 250 LOC + 6 test assertions

#### What it does:
Transforms **user queries** into normalized **symptom vectors** that can match extracted signatures.

**Three-stage pipeline:**

**Stage 1: Extract Keywords**
```
Query: "npm can't find tslib module error"
↓
Keywords: ["npm", "find", "tslib", "module", "error"]
```

**Stage 2: Normalize**
```
Keywords: ["npm", "find", "tslib", "module", "error"]
↓ [Remove stop words: can, find (weak), t]
Normalized: "error module npm tslib"
↓ [Sort alphabetically for idempotence]
Canonical: "error module npm tslib"
```

**Stage 3: Generate Hash**
```
Canonical: "error module npm tslib"
↓ [SHA256(tool + "\n" + normalized)]
sym_sha: "xyz789abc..." ← matches M3.7.7 signature if similar
```

#### Example Match:
```
Extracted signature (M3.7.7):
  normalised: "error module tslib"
  sig_sha: "abc123..."

User query (M3.7.8):
  query: "npm cannot resolve tslib"
  normalised: "error module npm tslib" (after expand "cannot")
  sym_sha: "abc123..." ← MATCH!
  
→ Tier 1 hit: Return past solution
```

#### Tests (6 assertions):
1. Same symptom variant → same hash
2. Abbreviation expansion (ERESOLVE, ERR, OOM, EACCES)
3. Stop word removal (can, the, able, to)
4. Tool consistency (npm != cargo for same error)
5. Case insensitive (NPM = npm)
6. Keyword order irrelevant (sorted before hash)

#### Files to create:
- `crates/mem-core/src/symptom_projection.rs` (250 LOC)
- `tests/it_symptom_projection.rs` (400 LOC, 6 assertions)
- `fixtures/symptoms/` (test query examples)

---

### M3.7.4 ⬜ PENDING — Context Endpoint Integration

**Status:** Waiting for M3.7.8 + M8.2 (hybrid search live)  
**Purpose:** HTTP endpoint gluing tiers 1–3 together  
**Endpoint:** `GET /memory/context?query=...&tool=...`

#### Three-tier logic:
```rust
pub async fn get_context(query: &str, tool: Option<&str>) -> ContextResult {
    // TIER 1: Exact signature match
    let symptom = project_symptom(tool.unwrap_or("unknown"), query); // M3.7.8
    if let Some(lesson) = find_lesson_by_sym_sha(&symptom.sym_sha) {
        return high_confidence_result(lesson);
    }
    
    // TIER 2: Hybrid search (requires M8.2)
    let hybrid = hybrid_search(query, tool).await?;
    if !hybrid.is_empty() {
        return medium_confidence_result(hybrid);
    }
    
    // TIER 3: Reference corpus (requires M3.6)
    let reference = reference_corpus_search(query, tool).await?;
    return low_confidence_result(reference);
}
```

#### Response format:
```json
{
  "tier": 1,
  "confidence": "high",
  "query_normalised": "error module npm tslib",
  "results": [
    {
      "source": "lesson",
      "title": "Fix npm module not found",
      "keywords": ["npm", "error", "module"],
      "solution": "Check package.json, npm install",
      "applies_to": ["npm", "yarn"],
      "severity": "medium"
    }
  ]
}
```

---

### M3.7.6 ⬜ PENDING — Composition Gate

**Status:** Depends on M3.7.4 + M3.7.8  
**Purpose:** Verify tiers work end-to-end (accuracy, latency, coverage)

#### Gate assertions:
1. Tier 1 queries resolve in < 50ms (cached lookup)
2. Tier 2 queries resolve in < 500ms (hybrid search)
3. Tier 3 queries resolve in < 1000ms (reference corpus)
4. Exactly one tier returns results (no duplicates across tiers)
5. Confidence scores monotonically decrease (tier 1 > 2 > 3)
6. Coverage: 95% of real failures find a tier 1 or 2 match

---

## Retired Tasks

✅ **M3.7.3** — `GET /memory/skills?task=...` (skill matching)  
   → Redundant with M8 hybrid search  
   → Removed: 1 task

✅ **M3.7.5** — `tool-failures` standing query  
   → Redundant with M8 hybrid search  
   → Removed: 1 task

**Outcome:** M3.7 reduced from 6 tasks → 4 tasks (gain: 2 simplified, no loss)

---

## Integration Points

| Component | Depends On | Used By |
|-----------|-----------|---------|
| M3.7.7 (Signature) | M3.6.1 (DocCorpus) | M3.7.8 (Symptom) |
| M3.7.8 (Symptom) | M3.7.7 (Signature) | M3.7.4 (Context endpoint) |
| M3.7.4 (Context) | M3.7.8 + M8.2 + M3.6 | M3.7.6 (Gate) |
| M3.7.6 (Gate) | M3.7.4 | — |

**Critical path:**
```
M3.7.7 ✅ → M3.7.8 🔄 → M3.7.4 ⬜
                           ↓ [waits for M8.2]
                        M3.7.6 ⬜
```

---

## Performance Targets

| Operation | Latency | Notes |
|-----------|---------|-------|
| M3.7.7: Extract signature | < 50ms | 50KB log → hash (rule-based) |
| M3.7.8: Project symptom | < 10ms | Query → normalized vector |
| Tier 1 lookup (M3.7.4) | < 50ms | DB hash lookup |
| Tier 2 lookup (M3.7.4) | < 500ms | Hybrid search (parallel engines) |
| Tier 3 lookup (M3.7.4) | < 1000ms | Reference corpus search |

---

## Success Criteria

### M3.7.7 ✅
- [x] 18 unit tests passing (signature extraction)
- [x] Deterministic hashing (same failure → same hash)
- [x] CLI command: `mem sig explain`
- [x] Fixtures: real captured logs (npm, cargo, kubectl, etc.)

### M3.7.8 🔄
- [ ] 6 integration tests passing (symptom projection)
- [ ] Deterministic hashing (same query variant → same hash)
- [ ] Abbreviation expansion per tool (ERESOLVE, ERR, EACCES, etc.)
- [ ] Integrated with M3.7.4 context endpoint
- [ ] Tier 1 lookups work end-to-end

### M3.7.4 ⬜
- [ ] HTTP endpoint operational
- [ ] Three-tier logic working
- [ ] Response includes `tier`, `confidence` fields
- [ ] Integrated with M8.2 (hybrid search)

### M3.7.6 ⬜
- [ ] Gate assertions passing (latency, coverage, accuracy)
- [ ] End-to-end failure diagnosis working

---

## Documents

- **Core Implementation:** `crates/mem-core/src/lesson.rs` (M3.7.7, 871 LOC)
- **M3.7.8 Design:** `docs/M3.7.8-SYMPTOM_PROJECTION.md` (this guide, 14KB)
- **Pipeline Flow:** `memory-flow.md` § "M3.7.7 → M3.7.8: Failure Diagnosis Pipeline" (176 lines)
- **Tests:** `tests/it_signature.rs` (M3.7.7, 9 assertions)

---

## Timeline

**Completed:** M3.7.7 (18 unit tests passing)  
**Next (1–2 days):** M3.7.8 symptom projection  
**Then (1 day):** M3.7.4 context endpoint  
**Finally (1 day):** M3.7.6 composition gate

**Total M3.7:** ~5 days (1 done, 4 remaining)

---

## Team Notes

- **Determinism is critical:** Both M3.7.7 and M3.7.8 must hash identically across runs
  - Use sorted keywords (not insertion order)
  - Include tool name in identity
  - Strip all volatile data (timestamps, addresses, etc.)

- **Tier 1 only fires if M3.7.8 sym_sha matches M3.7.7 sig_sha**
  - If query is ambiguous or doesn't match any signature → skip to tier 2
  - Fall back to hybrid search (tier 2) gracefully

- **M8.2 is a blocker for M3.7.4**
  - Hybrid search must be live before context endpoint can work
  - (Tier 2 fallback requires functional search)

---

**End of M3.7 summary.**