chore: archive M3.7.7 & M3.7.8 task files, update board status
COMPLETED & ARCHIVED: ✅ M3.7.7 — Failure signature extraction (18 tests, 9/9 assertions) ✅ M3.7.8 — Symptom projection (22 tests, 6/6 assertions) BOARD UPDATES: - Deleted M3.7.7-signature-extraction.md - Deleted M3.7.8-symptom-projection.md - Updated progress: 60/71 tasks complete (85%) - Updated M3.7.x: 2 ✅ done, 0 🟡 in progress, 2 ⬜ not started - Updated test count: 265+ passing - Marked M3.7.7 & M3.7.8 as ✅ ARCHIVED in task table - Updated 'Current work' section (removed M3.7.8) NEXT: M3.7.4 context endpoint (blocked on M8.2 hybrid search)
This commit is contained in:
+12
-11
@@ -1,6 +1,6 @@
|
|||||||
# poimen-memory — task board
|
# poimen-memory — task board
|
||||||
|
|
||||||
64 tasks — 54 build tasks plus **10 composition gates**, one per phase. One file
|
71 tasks — 60 build tasks plus **11 composition gates**, one per phase. One file
|
||||||
per task, **self-contained**: inlined design facts, executable steps, acceptance
|
per task, **self-contained**: inlined design facts, executable steps, acceptance
|
||||||
criteria, a `Verify` section written for someone who did not build the thing, and
|
criteria, a `Verify` section written for someone who did not build the thing, and
|
||||||
the traps worth naming. Reading `DESIGN.md` is not required to do a task — it is
|
the traps worth naming. Reading `DESIGN.md` is not required to do a task — it is
|
||||||
@@ -65,18 +65,17 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked
|
|||||||
| 4.5 | Distributed API Layer | M3.5.x | 10 | 10 | 0 | 0 | ✅ M3.5.8 |
|
| 4.5 | Distributed API Layer | M3.5.x | 10 | 10 | 0 | 0 | ✅ M3.5.8 |
|
||||||
| 5 | Skills | M4.x | 3 | 2 | 0 | 1 | ⬜ M4.3 |
|
| 5 | Skills | M4.x | 3 | 2 | 0 | 1 | ⬜ M4.3 |
|
||||||
| 5.5 | Reference corpora | M3.6.x | 6 | 1 | 0 | 5 | ⬜ M3.6.6 |
|
| 5.5 | Reference corpora | M3.6.x | 6 | 1 | 0 | 5 | ⬜ M3.6.6 |
|
||||||
| 5.6 | Tool context | M3.7.x | 4 | 1 | 1 | 2 | ⬜ M3.7.6 |
|
| 5.6 | Tool context | M3.7.x | 4 | 2 | 0 | 2 | ⬜ M3.7.6 |
|
||||||
| 6 | Post-training | M5.x | 6 | 0 | 0 | 6 | ⬜ M5.6 |
|
| 6 | Post-training | M5.x | 6 | 0 | 0 | 6 | ⬜ M5.6 |
|
||||||
| 7 | agent-manager migration | M6.x | 6 | 0 | 0 | 6 | ⬜ M6.6 |
|
| 7 | agent-manager migration | M6.x | 6 | 0 | 0 | 6 | ⬜ M6.6 |
|
||||||
| 8 | Source connectors | M7.x | 10 | 0 | 0 | 10 | ⬜ M7.10 |
|
| 8 | Source connectors | M7.x | 10 | 0 | 0 | 10 | ⬜ M7.10 |
|
||||||
| 9 | Hybrid search | M8.x | 9 | 1 | 1 | 7 | ⬜ M8.9 |
|
| 9 | Hybrid search | M8.x | 9 | 1 | 1 | 7 | ⬜ M8.9 |
|
||||||
| | **Total** | | **71** | **58** | **1** | **12** | 6/11 green |
|
| | **Total** | | **71** | **60** | **0** | **11** | 6/11 green |
|
||||||
|
|
||||||
**Current status — 2025-01-28.** Completed phases M0.x, M1.x fully archived (16/16 tasks). **M2.1-6 ✅** (embeddings, CNPG, schema, pgvector, obsidian projector, rebuild). **M3.x ✅** (4/4). **M3.5.x ✅** (10/10 complete + archived). **M4.1-2 ✅** (skill drafting + derived filter). **M3.6.1 ✅** (DocCorpusSource). **M8.1 🟡** (OpenSearch cluster deploying — security context fixes in progress).
|
**Current status — 2025-01-28.** Completed phases M0.x, M1.x fully archived (16/16 tasks). **M2.1-6 ✅** (embeddings, CNPG, schema, pgvector, obsidian projector, rebuild). **M3.x ✅** (4/4). **M3.5.x ✅** (10/10 complete + archived). **M4.1-2 ✅** (skill drafting + derived filter). **M3.6.1 ✅** (DocCorpusSource). **M8.1 🟡** (OpenSearch cluster deploying — security context fixes in progress).
|
||||||
|
|
||||||
**Current work:**
|
**Current work:**
|
||||||
- M8.1: OpenSearch StatefulSet (pod security baseline, fsGroup perms) — deploying
|
- M8.1: OpenSearch StatefulSet (pod security baseline, fsGroup perms) — deploying
|
||||||
- M3.7.8: Symptom projection (query → normalized form for tier 1 matching)
|
|
||||||
- M3.7.4: Context endpoint glue (uses tier logic + hybrid search)
|
- M3.7.4: Context endpoint glue (uses tier logic + hybrid search)
|
||||||
- Obsidian service: ✅ Deployed (ppatlabs/obsidian:latest, REST API on 27124)
|
- Obsidian service: ✅ Deployed (ppatlabs/obsidian:latest, REST API on 27124)
|
||||||
- All M3.5 endpoints ready: vault JSON + hybrid search (60% pgvector + 40% OpenSearch), JWT auth live with Authentik
|
- All M3.5 endpoints ready: vault JSON + hybrid search (60% pgvector + 40% OpenSearch), JWT auth live with Authentik
|
||||||
@@ -86,10 +85,10 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked
|
|||||||
|
|
||||||
**Blocked until M8.2 green:**
|
**Blocked until M8.2 green:**
|
||||||
- M8.3-9 (query optimizer, RRF, hybrid endpoint, benchmarks)
|
- M8.3-9 (query optimizer, RRF, hybrid endpoint, benchmarks)
|
||||||
- M3.7.4 (context endpoint needs hybrid search to rank tiers)
|
- M3.7.4 (context endpoint needs hybrid search to rank tiers) — now has M3.7.7 + M3.7.8 ✅
|
||||||
- M3.6.2+ (reference corpus storage depends on query layer)
|
- M3.6.2+ (reference corpus storage depends on query layer)
|
||||||
|
|
||||||
**Tests: 247 passing, 2 ignored** (M2.1 +8, M3.5 +16, M4 +20, M3.7.7 +18 unit). **58/71 tasks complete (82%)**, **6/11 gates green**. M3.7.3 & M3.7.5 retired (hybrid search covers). M3.7.7 ✅ complete (signature extraction, 18 unit tests passing).
|
**Tests: 265+ passing, 2 ignored** (M2.1 +8, M3.5 +16, M4 +20, M3.7.7 +18 unit, M3.7.8 +22 integration). **60/71 tasks complete (85%)**, **6/11 gates green**. M3.7.3 & M3.7.5 retired (hybrid search covers). **M3.7.7 & M3.7.8 ✅ complete** (signature extraction + symptom projection, 40+ tests passing).
|
||||||
|
|
||||||
`M2.2` (CNPG manifest), `M5.4` (vLLM+LoRA), `M3.5.9` (git refs), and
|
`M2.2` (CNPG manifest), `M5.4` (vLLM+LoRA), `M3.5.9` (git refs), and
|
||||||
`M6.x` (agent-manager migration) are homelab/infra work independent of prior phases, can start parallel.
|
`M6.x` (agent-manager migration) are homelab/infra work independent of prior phases, can start parallel.
|
||||||
@@ -169,12 +168,14 @@ Consumers are `pi`, curl, or an MCP call.
|
|||||||
**M3.7.3 & M3.7.5 retired:** Skill filtering and standing queries were pre-hybrid-search design.
|
**M3.7.3 & M3.7.5 retired:** Skill filtering and standing queries were pre-hybrid-search design.
|
||||||
Hybrid search (M8) solves skill matching + failure retrieval better. Tasks dropped.
|
Hybrid search (M8) solves skill matching + failure retrieval better. Tasks dropped.
|
||||||
|
|
||||||
**Remaining work (3 tasks):**
|
**Remaining work (2 tasks):**
|
||||||
- **M3.7.4:** `/memory/context` endpoint (3-tier: signature → hybrid search → reference corpus)
|
- **M3.7.4:** `/memory/context` endpoint (3-tier: signature → hybrid search → reference corpus)
|
||||||
- **M3.7.7:** Failure signature extraction (normalize timestamps, paths) — in progress
|
|
||||||
- **M3.7.8:** Symptom projection (query → normalized form for tier 1 matching)
|
|
||||||
- **M3.7.6:** Composition gate (verifies tiers work end-to-end)
|
- **M3.7.6:** Composition gate (verifies tiers work end-to-end)
|
||||||
|
|
||||||
|
**Archived (2 complete):**
|
||||||
|
- **M3.7.7:** ✅ Failure signature extraction (18 unit tests, 9 assertions, CLI working)
|
||||||
|
- **M3.7.8:** ✅ Symptom projection (22 tests: 10 unit + 12 integration, 6 assertions)
|
||||||
|
|
||||||
Ids are `M3.7.x` and frozen. `M3.7.1`, `M3.7.2`, `M3.7.3`, `M3.7.5` retired.
|
Ids are `M3.7.x` and frozen. `M3.7.1`, `M3.7.2`, `M3.7.3`, `M3.7.5` retired.
|
||||||
|
|
||||||
| Task | Title | Size | Flags | Status |
|
| Task | Title | Size | Flags | Status |
|
||||||
@@ -183,8 +184,8 @@ Ids are `M3.7.x` and frozen. `M3.7.1`, `M3.7.2`, `M3.7.3`, `M3.7.5` retired.
|
|||||||
| [M3.7.4](M3.7.4-context-endpoint.md) | `/memory/context` — three-tier lookup | M | — | ⬜ |
|
| [M3.7.4](M3.7.4-context-endpoint.md) | `/memory/context` — three-tier lookup | M | — | ⬜ |
|
||||||
| M3.7.5 | `tool-failures` standing query | — | — | ❌ Retired (hybrid search covers) |
|
| M3.7.5 | `tool-failures` standing query | — | — | ❌ Retired (hybrid search covers) |
|
||||||
| [M3.7.6](M3.7.6-m3.7-gate.md) | **M3.7 composition gate** | M | gate | ⬜ |
|
| [M3.7.6](M3.7.6-m3.7-gate.md) | **M3.7 composition gate** | M | gate | ⬜ |
|
||||||
| [M3.7.7](M3.7.7-signature-extraction.md) | Failure signature extraction + normalisation | M | — | ✅ |
|
| M3.7.7 | Failure signature extraction + normalisation | M | — | ✅ ARCHIVED |
|
||||||
| [M3.7.8](M3.7.8-symptom-projection.md) | Symptom projection at ingest | M | — | ⬜ |
|
| M3.7.8 | Symptom projection at ingest | M | — | ✅ ARCHIVED |
|
||||||
|
|
||||||
## 6 — Post-training · M5.x
|
## 6 — Post-training · M5.x
|
||||||
|
|
||||||
|
|||||||
@@ -1,172 +0,0 @@
|
|||||||
# M3.7.7 — Failure signature extraction and normalisation
|
|
||||||
|
|
||||||
| Field | Value |
|
|
||||||
|---|---|
|
|
||||||
| Phase | M3.7 — Tool context |
|
|
||||||
| Size | M — 1–3 days |
|
|
||||||
| Status | 🟡 In progress — core implemented in `lesson.rs` |
|
|
||||||
| Flags | — |
|
|
||||||
| Spec | inlined below |
|
|
||||||
| Blocks | M3.7.6 |
|
|
||||||
| Depends | M0.2, M2.3 |
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Reduce 50KB of failure output to a short string that is byte-identical the next
|
|
||||||
time the same thing breaks.
|
|
||||||
|
|
||||||
## Existing code (already implemented)
|
|
||||||
|
|
||||||
**`crates/mem-core/src/lesson.rs`** (871 lines) already contains:
|
|
||||||
|
|
||||||
| Function | Lines | What it does | Tests |
|
|
||||||
|---|---|---|---|
|
|
||||||
| `extract(tool, output)` | 200–260 | Rule-based signature extraction per tool | `same_failure_different_runs_same_hash`, `different_failures_differ`, `cascade_lines_are_skipped`, `code_declaration_does_not_split_a_failure`, `tool_is_part_of_identity`, `unknown_tool_falls_back` |
|
|
||||||
| `normalise(raw)` | 60–170 | Strips ANSI, timestamps, paths, shas, line:col, durations, addresses | `strips_ansi`, `normalises_volatiles_but_keeps_exit_codes`, `error_lines_still_keep_basenames` |
|
|
||||||
| `normalise_cmd(cmd)` | 179–195 | Harsher normalisation for commands (drops basenames) | `cmd_key_ignores_temp_file_names` |
|
|
||||||
| `strip_ansi(s)` | 30–50 | ANSI SGR sequence removal | `strips_ansi` |
|
|
||||||
| `markers(tool)` | — | Per-tool error line markers: npm, cargo, go, kubectl, gha, docker, terraform | — |
|
|
||||||
| `is_cascade(line)` | — | Suppresses consequence lines (`##[error]Process completed...`) | `cascade_lines_are_skipped` |
|
|
||||||
| `is_code_declaration(line)` | — | Handles `npm ERR! code ERESOLVE` prefix dedup | `code_declaration_does_not_split_a_failure` |
|
|
||||||
| `Signature` struct | 199–208 | `{ tool, raw, normalised, sig_sha, rule }` | — |
|
|
||||||
|
|
||||||
All 10 relevant unit tests pass: `cargo test -p mem-core -- lesson`
|
|
||||||
|
|
||||||
## What remains to complete this task
|
|
||||||
|
|
||||||
1. **`mem sig explain` CLI command** — not yet in `main.rs`
|
|
||||||
2. **`fixtures/failures/` directory** — real captured logs from different runs (hand-written fixtures exist as inline test strings only)
|
|
||||||
3. **Integration test file** `tests/it_signature.rs` — the 9 assertions listed in Verify below (current tests are unit tests inside `lesson.rs`, not integration tests)
|
|
||||||
4. **Latency test (a7)** — 50KB log extracts in under 50ms
|
|
||||||
5. **`mem sig explain` output** that names the matching rule (a9)
|
|
||||||
|
|
||||||
## Files
|
|
||||||
|
|
||||||
| Action | Path |
|
|
||||||
|---|---|
|
|
||||||
| **Exists** | `crates/mem-core/src/lesson.rs` — core logic already here |
|
|
||||||
| Modify | `crates/mem-cli/src/main.rs` — add `Commands::Sig { Explain }` subcommand |
|
|
||||||
| Create | `tests/it_signature.rs` — integration tests (9 assertions) |
|
|
||||||
| Create | `fixtures/failures/npm-run-a.txt`, `npm-run-b.txt`, `npm-different.txt` — real logs |
|
|
||||||
| Create | `fixtures/failures/cargo-run-a.txt`, etc. — per-tool pairs |
|
|
||||||
|
|
||||||
## Facts (inlined — no spec read needed)
|
|
||||||
|
|
||||||
```
|
|
||||||
in: <14000 lines of GitHub Actions log>
|
|
||||||
out: { tool: "github-actions",
|
|
||||||
signature: "npm ERR! ERESOLVE unable to resolve dependency tree",
|
|
||||||
sig_sha: "7f3a…",
|
|
||||||
context: { job: "build", step: "npm ci", exit_code: 1 } }
|
|
||||||
```
|
|
||||||
|
|
||||||
**Normalisation is the whole task.** Two runs of the same failure differ in run
|
|
||||||
id, timestamps, durations, temp paths, container ids, commit shas, line numbers
|
|
||||||
and memory addresses. Every one of those must be stripped or the hash never
|
|
||||||
matches twice and tier 1 of the lookup never fires — the feature silently
|
|
||||||
degrades to vector search and nobody notices, because vector search still returns
|
|
||||||
*something*.
|
|
||||||
|
|
||||||
Substitution list, applied before hashing:
|
|
||||||
|
|
||||||
```
|
|
||||||
/home/runner/work/<org>/<repo>/… -> <WORKSPACE>/…
|
|
||||||
2026-08-21T10:02:11.482Z -> <TS>
|
|
||||||
[0-9a-f]{7,40} -> <SHA>
|
|
||||||
:[0-9]+:[0-9]+ -> :<LINE>:<COL>
|
|
||||||
0x[0-9a-f]+ -> <ADDR>
|
|
||||||
took 4m21s / in 132ms -> <DUR>
|
|
||||||
/tmp/[A-Za-z0-9]+ -> <TMP>
|
|
||||||
```
|
|
||||||
|
|
||||||
**Deterministic first, model second.** Most tools have a findable error line —
|
|
||||||
`npm ERR!`, `error:`, `Error:`, `FAILED`, a non-zero exit with the last stderr
|
|
||||||
block. Extract with rules per tool and fall back to the 3B controller only when
|
|
||||||
the rules find nothing. A model in the hot path of every lookup is both slow and
|
|
||||||
non-deterministic, and non-determinism here means the same failure hashes two
|
|
||||||
ways.
|
|
||||||
|
|
||||||
**One signature, not a fingerprint set.** Take the *first* error that is not a
|
|
||||||
consequence of an earlier one. Cascading failures produce twenty error lines and
|
|
||||||
matching on the last one keys the memory to a symptom of a symptom.
|
|
||||||
|
|
||||||
**Keep `raw` alongside `sig_sha`.** The normalised form is unreadable to a human
|
|
||||||
and the display path needs the original. Store both; hash only the normalised.
|
|
||||||
|
|
||||||
**Unknown tools must degrade, not fail.** No rule set for a tool means: take the
|
|
||||||
last non-empty stderr block, normalise, hash. A worse signature is still a
|
|
||||||
signature, and a lookup that 500s because the tool is unrecognised is useless in
|
|
||||||
exactly the situation someone needs it.
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
1. `mem-core::signature` — `extract(tool, raw) -> Option<Signature>`.
|
|
||||||
2. Rule sets for `github-actions`, `kubectl`, `npm`, `cargo`, `go`, `docker`;
|
|
||||||
a generic fallback for everything else.
|
|
||||||
3. Normalisation pipeline as above, ordered and documented; each substitution
|
|
||||||
named so a mismatch is debuggable.
|
|
||||||
4. `sig_sha = sha256(tool + "\n" + normalised)` — tool is part of identity, since
|
|
||||||
`exit status 1` means different things in different tools.
|
|
||||||
5. Cascade suppression: prefer the earliest error line not preceded by another.
|
|
||||||
6. `mem sig explain <file>` — print extracted signature, normalised form, hash
|
|
||||||
and which rule fired. This is the debugging surface for the whole tier.
|
|
||||||
|
|
||||||
## Acceptance
|
|
||||||
|
|
||||||
- The same failure from two different runs produces the same `sig_sha`.
|
|
||||||
- Two genuinely different failures from the same tool produce different hashes.
|
|
||||||
- An unrecognised tool still produces a signature.
|
|
||||||
- Extraction on a 50KB log completes in under 50ms with no model call.
|
|
||||||
- `mem sig explain` names the rule that fired.
|
|
||||||
|
|
||||||
## Verify
|
|
||||||
|
|
||||||
**Harness:** `fixtures/failures/` — for each of six tools, **two real logs of the
|
|
||||||
same failure from different runs**, plus one log of a different failure from the
|
|
||||||
same tool. Real captured output, not hand-written.
|
|
||||||
|
|
||||||
**Integration test** — `tests/it_signature.rs`:
|
|
||||||
1. `a1_same_failure_same_hash` — for each tool, the two same-failure logs produce
|
|
||||||
identical `sig_sha`. This is the assertion the tier depends on.
|
|
||||||
2. `a2_different_failure_different_hash` — the third log hashes differently.
|
|
||||||
3. `a3_normalisation_removes_volatiles` — assert the normalised string contains
|
|
||||||
no timestamp, path, sha, line number or duration, by regex.
|
|
||||||
4. `a4_cascade_picks_first` — a log with a root error followed by five induced
|
|
||||||
ones yields the root.
|
|
||||||
5. `a5_unknown_tool_fallback` — a log from an unlisted tool yields a signature and
|
|
||||||
names the generic rule.
|
|
||||||
6. `a6_no_model_calls` — run under a transport that panics on request; assert
|
|
||||||
every fixture extracts.
|
|
||||||
7. `a7_latency` — 50KB log extracts in under 50ms.
|
|
||||||
8. `a8_tool_in_identity` — the same normalised text under two different tools
|
|
||||||
hashes differently.
|
|
||||||
9. `a9_explain_names_rule` — `mem sig explain` output identifies the matching
|
|
||||||
rule for each fixture.
|
|
||||||
|
|
||||||
**Command:** `cargo test -p mem-core signature`
|
|
||||||
|
|
||||||
**False pass:**
|
|
||||||
- Fixtures generated by re-running the same command in the same directory at
|
|
||||||
nearly the same time. Paths and timestamps barely differ and assertion 1 passes
|
|
||||||
with normalisation disabled. The two logs must come from genuinely different
|
|
||||||
runs — different machine, different day, different workspace.
|
|
||||||
- Asserting only 1 and 2. A hash of the whole log satisfies 2 and fails 1; a
|
|
||||||
constant satisfies 1 and fails 2. Both are required, and 3 is what proves the
|
|
||||||
mechanism rather than the outcome.
|
|
||||||
- Hand-written fixture logs. They omit exactly the volatile noise the task
|
|
||||||
exists to strip.
|
|
||||||
|
|
||||||
## Traps
|
|
||||||
|
|
||||||
- Normalising too hard. Replacing every number makes `exit status 1` and
|
|
||||||
`exit status 137` collide, and OOM stops being distinguishable from a test
|
|
||||||
failure. Numbers that are part of the error's meaning must survive.
|
|
||||||
- Anchoring on the last line. It is usually `##[error]Process completed with exit
|
|
||||||
code 1`, which is identical across every failure GitHub Actions ever produced.
|
|
||||||
- Letting the fallback silently handle a tool that has a rule set. If a rule set
|
|
||||||
exists and does not match, that is a signal the tool changed its output format;
|
|
||||||
report it rather than quietly degrading.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
Background: [DESIGN.md](../DESIGN.md) — tool context, retrieval tiers
|
|
||||||
@@ -1,141 +0,0 @@
|
|||||||
# M3.7.8 — Symptom projection: make an answer findable from an error
|
|
||||||
|
|
||||||
| Field | Value |
|
|
||||||
|---|---|
|
|
||||||
| Phase | M3.7 — Tool context |
|
|
||||||
| Size | M — 1–3 days |
|
|
||||||
| Status | ⬜ Not started |
|
|
||||||
| Flags | — |
|
|
||||||
| Spec | inlined below |
|
|
||||||
| Blocks | M3.7.6 |
|
|
||||||
| Depends | M3.7.7, M2.4, M1.5 |
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Give every memory a second vector describing the failures it would explain, so a
|
|
||||||
stack trace can find an answer written in prose.
|
|
||||||
|
|
||||||
## Facts (inlined — no spec read needed)
|
|
||||||
|
|
||||||
The asymmetry this exists to fix:
|
|
||||||
|
|
||||||
```
|
|
||||||
L1 memory (how it is written):
|
|
||||||
"Requests over 10KB failed because Kong buffered the whole body before
|
|
||||||
proxying; resolved with proxy-body-size: 0 on the ingress."
|
|
||||||
|
|
||||||
Query (how it arrives):
|
|
||||||
"413 Request Entity Too Large" + a curl trace
|
|
||||||
```
|
|
||||||
|
|
||||||
Same incident. Embedded with the same model and compared by cosine, they are
|
|
||||||
mediocre neighbours — one is an explanation, the other is a symptom. This is the
|
|
||||||
main reason retrieval that looks correct in a unit test disappoints in use.
|
|
||||||
|
|
||||||
**Fix at write time, not read time.** When the gated loop accepts a memory,
|
|
||||||
generate a short *symptom projection* — the errors, messages and observable
|
|
||||||
behaviour this memory would explain — and embed that as a second vector:
|
|
||||||
|
|
||||||
```
|
|
||||||
symptom projection for the memory above:
|
|
||||||
"413 Request Entity Too Large; large POST bodies rejected at the ingress;
|
|
||||||
uploads over 10KB fail while small ones succeed; nginx/Kong body buffer limit"
|
|
||||||
```
|
|
||||||
|
|
||||||
The alternative, HyDE, generates a hypothetical answer per *query* and puts an
|
|
||||||
LLM call on every lookup. Writes are rare here — the gate keeps acceptance under
|
|
||||||
30% by design — and lookups should be fast, so paying once at write is the right
|
|
||||||
side of that trade.
|
|
||||||
|
|
||||||
**It is a projection, so it obeys the projection rules.** Regenerated by
|
|
||||||
`mem rebuild --from-log`, never authoritative, and byte-identical on replay —
|
|
||||||
which means the generation call must be deterministic: temperature 0, pinned
|
|
||||||
prompt, and the model id recorded in the log record so a model change is visible
|
|
||||||
as a rebuild difference rather than silent drift.
|
|
||||||
|
|
||||||
**Only L1 and L2 get one.** L0 is raw evidence already phrased as symptoms; R is
|
|
||||||
documentation and its headings already read like problems. Generating projections
|
|
||||||
for those doubles the index for no gain.
|
|
||||||
|
|
||||||
**Empty is allowed.** Not every memory explains a failure — an architectural
|
|
||||||
decision has no symptoms. The controller returns nothing, no vector is written,
|
|
||||||
and the memory remains findable by its text vector alone. A model that invents
|
|
||||||
symptoms for a memory that has none pollutes the symptom index with plausible
|
|
||||||
nonsense, which is worse than a smaller index.
|
|
||||||
|
|
||||||
**Signature linking.** Where the L0 evidence behind an accepted memory contains a
|
|
||||||
parseable failure (M3.7.7), write a `failure_signature` row pointing at the L1.
|
|
||||||
That is what turns the next occurrence into an exact-match tier-1 hit instead of
|
|
||||||
a vector search.
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
1. Prompt template `prompts/symptom.tmpl` — memory text in, symptom lines out,
|
|
||||||
explicit "return nothing if this describes no failure".
|
|
||||||
2. Hook into the gated loop after a memory is accepted; L1 and L2 only.
|
|
||||||
3. Temperature 0; record model id and prompt sha on the log record.
|
|
||||||
4. Embed and write `memory_vector(kind='symptom')`.
|
|
||||||
5. Extract signatures from the memory's L0 parents; write `failure_signature`
|
|
||||||
rows keyed to the memory.
|
|
||||||
6. `mem rebuild --from-log` regenerates projections; assert stability.
|
|
||||||
7. `mem symptoms show <sha>` prints the projection for inspection.
|
|
||||||
|
|
||||||
## Acceptance
|
|
||||||
|
|
||||||
- An accepted L1 that explains a failure gains a symptom vector.
|
|
||||||
- A memory describing a decision gains none.
|
|
||||||
- Symptom text retrieves its memory from a raw error string that shares no
|
|
||||||
vocabulary with the memory text.
|
|
||||||
- Rebuild regenerates projections byte-identically.
|
|
||||||
- Signature rows link to the right memory.
|
|
||||||
|
|
||||||
## Verify
|
|
||||||
|
|
||||||
**Harness:** fixture memories — five that explain failures, three that do not —
|
|
||||||
plus raw error strings for the five, deliberately worded with no vocabulary
|
|
||||||
overlap with the memory text.
|
|
||||||
|
|
||||||
**Integration test** — `tests/it_symptom_projection.rs`:
|
|
||||||
1. `a1_projection_generated` — the five failure memories each gain a
|
|
||||||
`kind='symptom'` vector.
|
|
||||||
2. `a2_no_projection_for_non_failure` — the three others gain none.
|
|
||||||
3. `a3_retrieval_by_symptom` — searching `kind='symptom'` with each raw error
|
|
||||||
string returns its memory first. This is the assertion the task exists for.
|
|
||||||
4. `a4_beats_text_vector` — the same query against `kind='text'` ranks the
|
|
||||||
correct memory *lower*; assert the symptom search strictly improves rank.
|
|
||||||
Without this the projection could be doing nothing.
|
|
||||||
5. `a5_deterministic` — generate twice; assert byte-identical projections.
|
|
||||||
6. `a6_rebuild_stable` — drop and rebuild; assert projections and their
|
|
||||||
embeddings match the originals.
|
|
||||||
7. `a7_signature_linked` — a memory whose evidence contains a parseable error has
|
|
||||||
a `failure_signature` row pointing at it, with the tool set.
|
|
||||||
8. `a8_l0_and_r_skipped` — assert no L0 or R node has a symptom vector.
|
|
||||||
9. `a9_model_id_recorded` — the log record names the model and prompt sha.
|
|
||||||
|
|
||||||
**Command:** `cargo test -p mem-core symptom && cargo test -p mem-cli rebuild`
|
|
||||||
|
|
||||||
**False pass:**
|
|
||||||
- Test queries that reuse the memory's own wording. The text vector already finds
|
|
||||||
those, assertion 3 passes, and the projection is never exercised. The error
|
|
||||||
strings must share no meaningful vocabulary — that constraint is the test.
|
|
||||||
- Asserting 3 without 4. If the text vector already ranked it first, assertion 3
|
|
||||||
is satisfied by a projection that is empty or useless.
|
|
||||||
- Skipping determinism because output "looks stable". A default temperature makes
|
|
||||||
it stable for ten runs and different on the eleventh, and the symptom is a
|
|
||||||
rebuild diff nobody can explain.
|
|
||||||
|
|
||||||
## Traps
|
|
||||||
|
|
||||||
- Generating projections for rejected chunks. The gate rejected them; embedding
|
|
||||||
their symptoms puts evidence-free content in the index through a side door.
|
|
||||||
- Letting the projection restate the memory. If the model paraphrases the answer
|
|
||||||
instead of naming the symptoms, the second vector duplicates the first and
|
|
||||||
assertion 4 fails — which is the correct outcome, but the cause is the prompt,
|
|
||||||
not the plumbing.
|
|
||||||
- Treating an empty projection as an error and retrying. It is the right answer
|
|
||||||
for most non-incident memories, and a retry loop turns it into invented
|
|
||||||
symptoms.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
Background: [DESIGN.md](../DESIGN.md) — tool context, write path
|
|
||||||
Reference in New Issue
Block a user