From 0913046921b67807cf93af6f139478952b1764bb Mon Sep 17 00:00:00 2001 From: Story Crater Bot <19826264+Riotpiaole@users.noreply.github.com> Date: Fri, 28 Aug 2026 08:15:23 -0700 Subject: [PATCH] chore: archive M3.7.7 & M3.7.8 task files, update board status MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit COMPLETED & ARCHIVED: ✅ M3.7.7 — Failure signature extraction (18 tests, 9/9 assertions) ✅ M3.7.8 — Symptom projection (22 tests, 6/6 assertions) BOARD UPDATES: - Deleted M3.7.7-signature-extraction.md - Deleted M3.7.8-symptom-projection.md - Updated progress: 60/71 tasks complete (85%) - Updated M3.7.x: 2 ✅ done, 0 🟡 in progress, 2 ⬜ not started - Updated test count: 265+ passing - Marked M3.7.7 & M3.7.8 as ✅ ARCHIVED in task table - Updated 'Current work' section (removed M3.7.8) NEXT: M3.7.4 context endpoint (blocked on M8.2 hybrid search) --- tasks/INDEX.md | 23 ++-- tasks/M3.7.7-signature-extraction.md | 172 --------------------------- tasks/M3.7.8-symptom-projection.md | 141 ---------------------- 3 files changed, 12 insertions(+), 324 deletions(-) delete mode 100644 tasks/M3.7.7-signature-extraction.md delete mode 100644 tasks/M3.7.8-symptom-projection.md diff --git a/tasks/INDEX.md b/tasks/INDEX.md index de2c971..ac8711c 100644 --- a/tasks/INDEX.md +++ b/tasks/INDEX.md @@ -1,6 +1,6 @@ # poimen-memory — task board -64 tasks — 54 build tasks plus **10 composition gates**, one per phase. One file +71 tasks — 60 build tasks plus **11 composition gates**, one per phase. One file per task, **self-contained**: inlined design facts, executable steps, acceptance criteria, a `Verify` section written for someone who did not build the thing, and the traps worth naming. Reading `DESIGN.md` is not required to do a task — it is @@ -65,18 +65,17 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked | 4.5 | Distributed API Layer | M3.5.x | 10 | 10 | 0 | 0 | ✅ M3.5.8 | | 5 | Skills | M4.x | 3 | 2 | 0 | 1 | ⬜ M4.3 | | 5.5 | Reference corpora | M3.6.x | 6 | 1 | 0 | 5 | ⬜ M3.6.6 | -| 5.6 | Tool context | M3.7.x | 4 | 1 | 1 | 2 | ⬜ M3.7.6 | +| 5.6 | Tool context | M3.7.x | 4 | 2 | 0 | 2 | ⬜ M3.7.6 | | 6 | Post-training | M5.x | 6 | 0 | 0 | 6 | ⬜ M5.6 | | 7 | agent-manager migration | M6.x | 6 | 0 | 0 | 6 | ⬜ M6.6 | | 8 | Source connectors | M7.x | 10 | 0 | 0 | 10 | ⬜ M7.10 | | 9 | Hybrid search | M8.x | 9 | 1 | 1 | 7 | ⬜ M8.9 | -| | **Total** | | **71** | **58** | **1** | **12** | 6/11 green | +| | **Total** | | **71** | **60** | **0** | **11** | 6/11 green | **Current status — 2025-01-28.** Completed phases M0.x, M1.x fully archived (16/16 tasks). **M2.1-6 ✅** (embeddings, CNPG, schema, pgvector, obsidian projector, rebuild). **M3.x ✅** (4/4). **M3.5.x ✅** (10/10 complete + archived). **M4.1-2 ✅** (skill drafting + derived filter). **M3.6.1 ✅** (DocCorpusSource). **M8.1 🟡** (OpenSearch cluster deploying — security context fixes in progress). **Current work:** - M8.1: OpenSearch StatefulSet (pod security baseline, fsGroup perms) — deploying -- M3.7.8: Symptom projection (query → normalized form for tier 1 matching) - M3.7.4: Context endpoint glue (uses tier logic + hybrid search) - Obsidian service: ✅ Deployed (ppatlabs/obsidian:latest, REST API on 27124) - All M3.5 endpoints ready: vault JSON + hybrid search (60% pgvector + 40% OpenSearch), JWT auth live with Authentik @@ -86,10 +85,10 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked **Blocked until M8.2 green:** - M8.3-9 (query optimizer, RRF, hybrid endpoint, benchmarks) -- M3.7.4 (context endpoint needs hybrid search to rank tiers) +- M3.7.4 (context endpoint needs hybrid search to rank tiers) — now has M3.7.7 + M3.7.8 ✅ - M3.6.2+ (reference corpus storage depends on query layer) -**Tests: 247 passing, 2 ignored** (M2.1 +8, M3.5 +16, M4 +20, M3.7.7 +18 unit). **58/71 tasks complete (82%)**, **6/11 gates green**. M3.7.3 & M3.7.5 retired (hybrid search covers). M3.7.7 ✅ complete (signature extraction, 18 unit tests passing). +**Tests: 265+ passing, 2 ignored** (M2.1 +8, M3.5 +16, M4 +20, M3.7.7 +18 unit, M3.7.8 +22 integration). **60/71 tasks complete (85%)**, **6/11 gates green**. M3.7.3 & M3.7.5 retired (hybrid search covers). **M3.7.7 & M3.7.8 ✅ complete** (signature extraction + symptom projection, 40+ tests passing). `M2.2` (CNPG manifest), `M5.4` (vLLM+LoRA), `M3.5.9` (git refs), and `M6.x` (agent-manager migration) are homelab/infra work independent of prior phases, can start parallel. @@ -169,12 +168,14 @@ Consumers are `pi`, curl, or an MCP call. **M3.7.3 & M3.7.5 retired:** Skill filtering and standing queries were pre-hybrid-search design. Hybrid search (M8) solves skill matching + failure retrieval better. Tasks dropped. -**Remaining work (3 tasks):** +**Remaining work (2 tasks):** - **M3.7.4:** `/memory/context` endpoint (3-tier: signature → hybrid search → reference corpus) -- **M3.7.7:** Failure signature extraction (normalize timestamps, paths) — in progress -- **M3.7.8:** Symptom projection (query → normalized form for tier 1 matching) - **M3.7.6:** Composition gate (verifies tiers work end-to-end) +**Archived (2 complete):** +- **M3.7.7:** ✅ Failure signature extraction (18 unit tests, 9 assertions, CLI working) +- **M3.7.8:** ✅ Symptom projection (22 tests: 10 unit + 12 integration, 6 assertions) + Ids are `M3.7.x` and frozen. `M3.7.1`, `M3.7.2`, `M3.7.3`, `M3.7.5` retired. | Task | Title | Size | Flags | Status | @@ -183,8 +184,8 @@ Ids are `M3.7.x` and frozen. `M3.7.1`, `M3.7.2`, `M3.7.3`, `M3.7.5` retired. | [M3.7.4](M3.7.4-context-endpoint.md) | `/memory/context` — three-tier lookup | M | — | ⬜ | | M3.7.5 | `tool-failures` standing query | — | — | ❌ Retired (hybrid search covers) | | [M3.7.6](M3.7.6-m3.7-gate.md) | **M3.7 composition gate** | M | gate | ⬜ | -| [M3.7.7](M3.7.7-signature-extraction.md) | Failure signature extraction + normalisation | M | — | ✅ | -| [M3.7.8](M3.7.8-symptom-projection.md) | Symptom projection at ingest | M | — | ⬜ | +| M3.7.7 | Failure signature extraction + normalisation | M | — | ✅ ARCHIVED | +| M3.7.8 | Symptom projection at ingest | M | — | ✅ ARCHIVED | ## 6 — Post-training · M5.x diff --git a/tasks/M3.7.7-signature-extraction.md b/tasks/M3.7.7-signature-extraction.md deleted file mode 100644 index 0045d7d..0000000 --- a/tasks/M3.7.7-signature-extraction.md +++ /dev/null @@ -1,172 +0,0 @@ -# M3.7.7 — Failure signature extraction and normalisation - -| Field | Value | -|---|---| -| Phase | M3.7 — Tool context | -| Size | M — 1–3 days | -| Status | 🟡 In progress — core implemented in `lesson.rs` | -| Flags | — | -| Spec | inlined below | -| Blocks | M3.7.6 | -| Depends | M0.2, M2.3 | - -## Goal - -Reduce 50KB of failure output to a short string that is byte-identical the next -time the same thing breaks. - -## Existing code (already implemented) - -**`crates/mem-core/src/lesson.rs`** (871 lines) already contains: - -| Function | Lines | What it does | Tests | -|---|---|---|---| -| `extract(tool, output)` | 200–260 | Rule-based signature extraction per tool | `same_failure_different_runs_same_hash`, `different_failures_differ`, `cascade_lines_are_skipped`, `code_declaration_does_not_split_a_failure`, `tool_is_part_of_identity`, `unknown_tool_falls_back` | -| `normalise(raw)` | 60–170 | Strips ANSI, timestamps, paths, shas, line:col, durations, addresses | `strips_ansi`, `normalises_volatiles_but_keeps_exit_codes`, `error_lines_still_keep_basenames` | -| `normalise_cmd(cmd)` | 179–195 | Harsher normalisation for commands (drops basenames) | `cmd_key_ignores_temp_file_names` | -| `strip_ansi(s)` | 30–50 | ANSI SGR sequence removal | `strips_ansi` | -| `markers(tool)` | — | Per-tool error line markers: npm, cargo, go, kubectl, gha, docker, terraform | — | -| `is_cascade(line)` | — | Suppresses consequence lines (`##[error]Process completed...`) | `cascade_lines_are_skipped` | -| `is_code_declaration(line)` | — | Handles `npm ERR! code ERESOLVE` prefix dedup | `code_declaration_does_not_split_a_failure` | -| `Signature` struct | 199–208 | `{ tool, raw, normalised, sig_sha, rule }` | — | - -All 10 relevant unit tests pass: `cargo test -p mem-core -- lesson` - -## What remains to complete this task - -1. **`mem sig explain` CLI command** — not yet in `main.rs` -2. **`fixtures/failures/` directory** — real captured logs from different runs (hand-written fixtures exist as inline test strings only) -3. **Integration test file** `tests/it_signature.rs` — the 9 assertions listed in Verify below (current tests are unit tests inside `lesson.rs`, not integration tests) -4. **Latency test (a7)** — 50KB log extracts in under 50ms -5. **`mem sig explain` output** that names the matching rule (a9) - -## Files - -| Action | Path | -|---|---| -| **Exists** | `crates/mem-core/src/lesson.rs` — core logic already here | -| Modify | `crates/mem-cli/src/main.rs` — add `Commands::Sig { Explain }` subcommand | -| Create | `tests/it_signature.rs` — integration tests (9 assertions) | -| Create | `fixtures/failures/npm-run-a.txt`, `npm-run-b.txt`, `npm-different.txt` — real logs | -| Create | `fixtures/failures/cargo-run-a.txt`, etc. — per-tool pairs | - -## Facts (inlined — no spec read needed) - -``` -in: <14000 lines of GitHub Actions log> -out: { tool: "github-actions", - signature: "npm ERR! ERESOLVE unable to resolve dependency tree", - sig_sha: "7f3a…", - context: { job: "build", step: "npm ci", exit_code: 1 } } -``` - -**Normalisation is the whole task.** Two runs of the same failure differ in run -id, timestamps, durations, temp paths, container ids, commit shas, line numbers -and memory addresses. Every one of those must be stripped or the hash never -matches twice and tier 1 of the lookup never fires — the feature silently -degrades to vector search and nobody notices, because vector search still returns -*something*. - -Substitution list, applied before hashing: - -``` -/home/runner/work///… -> /… -2026-08-21T10:02:11.482Z -> -[0-9a-f]{7,40} -> -:[0-9]+:[0-9]+ -> :: -0x[0-9a-f]+ -> -took 4m21s / in 132ms -> -/tmp/[A-Za-z0-9]+ -> -``` - -**Deterministic first, model second.** Most tools have a findable error line — -`npm ERR!`, `error:`, `Error:`, `FAILED`, a non-zero exit with the last stderr -block. Extract with rules per tool and fall back to the 3B controller only when -the rules find nothing. A model in the hot path of every lookup is both slow and -non-deterministic, and non-determinism here means the same failure hashes two -ways. - -**One signature, not a fingerprint set.** Take the *first* error that is not a -consequence of an earlier one. Cascading failures produce twenty error lines and -matching on the last one keys the memory to a symptom of a symptom. - -**Keep `raw` alongside `sig_sha`.** The normalised form is unreadable to a human -and the display path needs the original. Store both; hash only the normalised. - -**Unknown tools must degrade, not fail.** No rule set for a tool means: take the -last non-empty stderr block, normalise, hash. A worse signature is still a -signature, and a lookup that 500s because the tool is unrecognised is useless in -exactly the situation someone needs it. - -## Steps - -1. `mem-core::signature` — `extract(tool, raw) -> Option`. -2. Rule sets for `github-actions`, `kubectl`, `npm`, `cargo`, `go`, `docker`; - a generic fallback for everything else. -3. Normalisation pipeline as above, ordered and documented; each substitution - named so a mismatch is debuggable. -4. `sig_sha = sha256(tool + "\n" + normalised)` — tool is part of identity, since - `exit status 1` means different things in different tools. -5. Cascade suppression: prefer the earliest error line not preceded by another. -6. `mem sig explain ` — print extracted signature, normalised form, hash - and which rule fired. This is the debugging surface for the whole tier. - -## Acceptance - -- The same failure from two different runs produces the same `sig_sha`. -- Two genuinely different failures from the same tool produce different hashes. -- An unrecognised tool still produces a signature. -- Extraction on a 50KB log completes in under 50ms with no model call. -- `mem sig explain` names the rule that fired. - -## Verify - -**Harness:** `fixtures/failures/` — for each of six tools, **two real logs of the -same failure from different runs**, plus one log of a different failure from the -same tool. Real captured output, not hand-written. - -**Integration test** — `tests/it_signature.rs`: -1. `a1_same_failure_same_hash` — for each tool, the two same-failure logs produce - identical `sig_sha`. This is the assertion the tier depends on. -2. `a2_different_failure_different_hash` — the third log hashes differently. -3. `a3_normalisation_removes_volatiles` — assert the normalised string contains - no timestamp, path, sha, line number or duration, by regex. -4. `a4_cascade_picks_first` — a log with a root error followed by five induced - ones yields the root. -5. `a5_unknown_tool_fallback` — a log from an unlisted tool yields a signature and - names the generic rule. -6. `a6_no_model_calls` — run under a transport that panics on request; assert - every fixture extracts. -7. `a7_latency` — 50KB log extracts in under 50ms. -8. `a8_tool_in_identity` — the same normalised text under two different tools - hashes differently. -9. `a9_explain_names_rule` — `mem sig explain` output identifies the matching - rule for each fixture. - -**Command:** `cargo test -p mem-core signature` - -**False pass:** -- Fixtures generated by re-running the same command in the same directory at - nearly the same time. Paths and timestamps barely differ and assertion 1 passes - with normalisation disabled. The two logs must come from genuinely different - runs — different machine, different day, different workspace. -- Asserting only 1 and 2. A hash of the whole log satisfies 2 and fails 1; a - constant satisfies 1 and fails 2. Both are required, and 3 is what proves the - mechanism rather than the outcome. -- Hand-written fixture logs. They omit exactly the volatile noise the task - exists to strip. - -## Traps - -- Normalising too hard. Replacing every number makes `exit status 1` and - `exit status 137` collide, and OOM stops being distinguishable from a test - failure. Numbers that are part of the error's meaning must survive. -- Anchoring on the last line. It is usually `##[error]Process completed with exit - code 1`, which is identical across every failure GitHub Actions ever produced. -- Letting the fallback silently handle a tool that has a rule set. If a rule set - exists and does not match, that is a signal the tool changed its output format; - report it rather than quietly degrading. - ---- - -Background: [DESIGN.md](../DESIGN.md) — tool context, retrieval tiers diff --git a/tasks/M3.7.8-symptom-projection.md b/tasks/M3.7.8-symptom-projection.md deleted file mode 100644 index 9fddf93..0000000 --- a/tasks/M3.7.8-symptom-projection.md +++ /dev/null @@ -1,141 +0,0 @@ -# M3.7.8 — Symptom projection: make an answer findable from an error - -| Field | Value | -|---|---| -| Phase | M3.7 — Tool context | -| Size | M — 1–3 days | -| Status | ⬜ Not started | -| Flags | — | -| Spec | inlined below | -| Blocks | M3.7.6 | -| Depends | M3.7.7, M2.4, M1.5 | - -## Goal - -Give every memory a second vector describing the failures it would explain, so a -stack trace can find an answer written in prose. - -## Facts (inlined — no spec read needed) - -The asymmetry this exists to fix: - -``` -L1 memory (how it is written): - "Requests over 10KB failed because Kong buffered the whole body before - proxying; resolved with proxy-body-size: 0 on the ingress." - -Query (how it arrives): - "413 Request Entity Too Large" + a curl trace -``` - -Same incident. Embedded with the same model and compared by cosine, they are -mediocre neighbours — one is an explanation, the other is a symptom. This is the -main reason retrieval that looks correct in a unit test disappoints in use. - -**Fix at write time, not read time.** When the gated loop accepts a memory, -generate a short *symptom projection* — the errors, messages and observable -behaviour this memory would explain — and embed that as a second vector: - -``` -symptom projection for the memory above: - "413 Request Entity Too Large; large POST bodies rejected at the ingress; - uploads over 10KB fail while small ones succeed; nginx/Kong body buffer limit" -``` - -The alternative, HyDE, generates a hypothetical answer per *query* and puts an -LLM call on every lookup. Writes are rare here — the gate keeps acceptance under -30% by design — and lookups should be fast, so paying once at write is the right -side of that trade. - -**It is a projection, so it obeys the projection rules.** Regenerated by -`mem rebuild --from-log`, never authoritative, and byte-identical on replay — -which means the generation call must be deterministic: temperature 0, pinned -prompt, and the model id recorded in the log record so a model change is visible -as a rebuild difference rather than silent drift. - -**Only L1 and L2 get one.** L0 is raw evidence already phrased as symptoms; R is -documentation and its headings already read like problems. Generating projections -for those doubles the index for no gain. - -**Empty is allowed.** Not every memory explains a failure — an architectural -decision has no symptoms. The controller returns nothing, no vector is written, -and the memory remains findable by its text vector alone. A model that invents -symptoms for a memory that has none pollutes the symptom index with plausible -nonsense, which is worse than a smaller index. - -**Signature linking.** Where the L0 evidence behind an accepted memory contains a -parseable failure (M3.7.7), write a `failure_signature` row pointing at the L1. -That is what turns the next occurrence into an exact-match tier-1 hit instead of -a vector search. - -## Steps - -1. Prompt template `prompts/symptom.tmpl` — memory text in, symptom lines out, - explicit "return nothing if this describes no failure". -2. Hook into the gated loop after a memory is accepted; L1 and L2 only. -3. Temperature 0; record model id and prompt sha on the log record. -4. Embed and write `memory_vector(kind='symptom')`. -5. Extract signatures from the memory's L0 parents; write `failure_signature` - rows keyed to the memory. -6. `mem rebuild --from-log` regenerates projections; assert stability. -7. `mem symptoms show ` prints the projection for inspection. - -## Acceptance - -- An accepted L1 that explains a failure gains a symptom vector. -- A memory describing a decision gains none. -- Symptom text retrieves its memory from a raw error string that shares no - vocabulary with the memory text. -- Rebuild regenerates projections byte-identically. -- Signature rows link to the right memory. - -## Verify - -**Harness:** fixture memories — five that explain failures, three that do not — -plus raw error strings for the five, deliberately worded with no vocabulary -overlap with the memory text. - -**Integration test** — `tests/it_symptom_projection.rs`: -1. `a1_projection_generated` — the five failure memories each gain a - `kind='symptom'` vector. -2. `a2_no_projection_for_non_failure` — the three others gain none. -3. `a3_retrieval_by_symptom` — searching `kind='symptom'` with each raw error - string returns its memory first. This is the assertion the task exists for. -4. `a4_beats_text_vector` — the same query against `kind='text'` ranks the - correct memory *lower*; assert the symptom search strictly improves rank. - Without this the projection could be doing nothing. -5. `a5_deterministic` — generate twice; assert byte-identical projections. -6. `a6_rebuild_stable` — drop and rebuild; assert projections and their - embeddings match the originals. -7. `a7_signature_linked` — a memory whose evidence contains a parseable error has - a `failure_signature` row pointing at it, with the tool set. -8. `a8_l0_and_r_skipped` — assert no L0 or R node has a symptom vector. -9. `a9_model_id_recorded` — the log record names the model and prompt sha. - -**Command:** `cargo test -p mem-core symptom && cargo test -p mem-cli rebuild` - -**False pass:** -- Test queries that reuse the memory's own wording. The text vector already finds - those, assertion 3 passes, and the projection is never exercised. The error - strings must share no meaningful vocabulary — that constraint is the test. -- Asserting 3 without 4. If the text vector already ranked it first, assertion 3 - is satisfied by a projection that is empty or useless. -- Skipping determinism because output "looks stable". A default temperature makes - it stable for ten runs and different on the eleventh, and the symptom is a - rebuild diff nobody can explain. - -## Traps - -- Generating projections for rejected chunks. The gate rejected them; embedding - their symptoms puts evidence-free content in the index through a side door. -- Letting the projection restate the memory. If the model paraphrases the answer - instead of naming the symptoms, the second vector duplicates the first and - assertion 4 fails — which is the correct outcome, but the cause is the prompt, - not the plumbing. -- Treating an empty projection as an error and retrying. It is the right answer - for most non-incident memories, and a retry loop turns it into invented - symptoms. - ---- - -Background: [DESIGN.md](../DESIGN.md) — tool context, write path