Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)
ci / markdown (push) Waiting to run
ci / markdown (push) Waiting to run
This commit is contained in:
Story Crater Bot
@@ -0,0 +1,172 @@
|
||||
# M3.7.7 — Failure signature extraction and normalisation
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | M3.7 — Tool context |
|
||||
| Size | M — 1–3 days |
|
||||
| Status | 🟡 In progress — core implemented in `lesson.rs` |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | M3.7.6 |
|
||||
| Depends | M0.2, M2.3 |
|
||||
|
||||
## Goal
|
||||
|
||||
Reduce 50KB of failure output to a short string that is byte-identical the next
|
||||
time the same thing breaks.
|
||||
|
||||
## Existing code (already implemented)
|
||||
|
||||
**`crates/mem-core/src/lesson.rs`** (871 lines) already contains:
|
||||
|
||||
| Function | Lines | What it does | Tests |
|
||||
|---|---|---|---|
|
||||
| `extract(tool, output)` | 200–260 | Rule-based signature extraction per tool | `same_failure_different_runs_same_hash`, `different_failures_differ`, `cascade_lines_are_skipped`, `code_declaration_does_not_split_a_failure`, `tool_is_part_of_identity`, `unknown_tool_falls_back` |
|
||||
| `normalise(raw)` | 60–170 | Strips ANSI, timestamps, paths, shas, line:col, durations, addresses | `strips_ansi`, `normalises_volatiles_but_keeps_exit_codes`, `error_lines_still_keep_basenames` |
|
||||
| `normalise_cmd(cmd)` | 179–195 | Harsher normalisation for commands (drops basenames) | `cmd_key_ignores_temp_file_names` |
|
||||
| `strip_ansi(s)` | 30–50 | ANSI SGR sequence removal | `strips_ansi` |
|
||||
| `markers(tool)` | — | Per-tool error line markers: npm, cargo, go, kubectl, gha, docker, terraform | — |
|
||||
| `is_cascade(line)` | — | Suppresses consequence lines (`##[error]Process completed...`) | `cascade_lines_are_skipped` |
|
||||
| `is_code_declaration(line)` | — | Handles `npm ERR! code ERESOLVE` prefix dedup | `code_declaration_does_not_split_a_failure` |
|
||||
| `Signature` struct | 199–208 | `{ tool, raw, normalised, sig_sha, rule }` | — |
|
||||
|
||||
All 10 relevant unit tests pass: `cargo test -p mem-core -- lesson`
|
||||
|
||||
## What remains to complete this task
|
||||
|
||||
1. **`mem sig explain` CLI command** — not yet in `main.rs`
|
||||
2. **`fixtures/failures/` directory** — real captured logs from different runs (hand-written fixtures exist as inline test strings only)
|
||||
3. **Integration test file** `tests/it_signature.rs` — the 9 assertions listed in Verify below (current tests are unit tests inside `lesson.rs`, not integration tests)
|
||||
4. **Latency test (a7)** — 50KB log extracts in under 50ms
|
||||
5. **`mem sig explain` output** that names the matching rule (a9)
|
||||
|
||||
## Files
|
||||
|
||||
| Action | Path |
|
||||
|---|---|
|
||||
| **Exists** | `crates/mem-core/src/lesson.rs` — core logic already here |
|
||||
| Modify | `crates/mem-cli/src/main.rs` — add `Commands::Sig { Explain }` subcommand |
|
||||
| Create | `tests/it_signature.rs` — integration tests (9 assertions) |
|
||||
| Create | `fixtures/failures/npm-run-a.txt`, `npm-run-b.txt`, `npm-different.txt` — real logs |
|
||||
| Create | `fixtures/failures/cargo-run-a.txt`, etc. — per-tool pairs |
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
```
|
||||
in: <14000 lines of GitHub Actions log>
|
||||
out: { tool: "github-actions",
|
||||
signature: "npm ERR! ERESOLVE unable to resolve dependency tree",
|
||||
sig_sha: "7f3a…",
|
||||
context: { job: "build", step: "npm ci", exit_code: 1 } }
|
||||
```
|
||||
|
||||
**Normalisation is the whole task.** Two runs of the same failure differ in run
|
||||
id, timestamps, durations, temp paths, container ids, commit shas, line numbers
|
||||
and memory addresses. Every one of those must be stripped or the hash never
|
||||
matches twice and tier 1 of the lookup never fires — the feature silently
|
||||
degrades to vector search and nobody notices, because vector search still returns
|
||||
*something*.
|
||||
|
||||
Substitution list, applied before hashing:
|
||||
|
||||
```
|
||||
/home/runner/work/<org>/<repo>/… -> <WORKSPACE>/…
|
||||
2026-08-21T10:02:11.482Z -> <TS>
|
||||
[0-9a-f]{7,40} -> <SHA>
|
||||
:[0-9]+:[0-9]+ -> :<LINE>:<COL>
|
||||
0x[0-9a-f]+ -> <ADDR>
|
||||
took 4m21s / in 132ms -> <DUR>
|
||||
/tmp/[A-Za-z0-9]+ -> <TMP>
|
||||
```
|
||||
|
||||
**Deterministic first, model second.** Most tools have a findable error line —
|
||||
`npm ERR!`, `error:`, `Error:`, `FAILED`, a non-zero exit with the last stderr
|
||||
block. Extract with rules per tool and fall back to the 3B controller only when
|
||||
the rules find nothing. A model in the hot path of every lookup is both slow and
|
||||
non-deterministic, and non-determinism here means the same failure hashes two
|
||||
ways.
|
||||
|
||||
**One signature, not a fingerprint set.** Take the *first* error that is not a
|
||||
consequence of an earlier one. Cascading failures produce twenty error lines and
|
||||
matching on the last one keys the memory to a symptom of a symptom.
|
||||
|
||||
**Keep `raw` alongside `sig_sha`.** The normalised form is unreadable to a human
|
||||
and the display path needs the original. Store both; hash only the normalised.
|
||||
|
||||
**Unknown tools must degrade, not fail.** No rule set for a tool means: take the
|
||||
last non-empty stderr block, normalise, hash. A worse signature is still a
|
||||
signature, and a lookup that 500s because the tool is unrecognised is useless in
|
||||
exactly the situation someone needs it.
|
||||
|
||||
## Steps
|
||||
|
||||
1. `mem-core::signature` — `extract(tool, raw) -> Option<Signature>`.
|
||||
2. Rule sets for `github-actions`, `kubectl`, `npm`, `cargo`, `go`, `docker`;
|
||||
a generic fallback for everything else.
|
||||
3. Normalisation pipeline as above, ordered and documented; each substitution
|
||||
named so a mismatch is debuggable.
|
||||
4. `sig_sha = sha256(tool + "\n" + normalised)` — tool is part of identity, since
|
||||
`exit status 1` means different things in different tools.
|
||||
5. Cascade suppression: prefer the earliest error line not preceded by another.
|
||||
6. `mem sig explain <file>` — print extracted signature, normalised form, hash
|
||||
and which rule fired. This is the debugging surface for the whole tier.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- The same failure from two different runs produces the same `sig_sha`.
|
||||
- Two genuinely different failures from the same tool produce different hashes.
|
||||
- An unrecognised tool still produces a signature.
|
||||
- Extraction on a 50KB log completes in under 50ms with no model call.
|
||||
- `mem sig explain` names the rule that fired.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** `fixtures/failures/` — for each of six tools, **two real logs of the
|
||||
same failure from different runs**, plus one log of a different failure from the
|
||||
same tool. Real captured output, not hand-written.
|
||||
|
||||
**Integration test** — `tests/it_signature.rs`:
|
||||
1. `a1_same_failure_same_hash` — for each tool, the two same-failure logs produce
|
||||
identical `sig_sha`. This is the assertion the tier depends on.
|
||||
2. `a2_different_failure_different_hash` — the third log hashes differently.
|
||||
3. `a3_normalisation_removes_volatiles` — assert the normalised string contains
|
||||
no timestamp, path, sha, line number or duration, by regex.
|
||||
4. `a4_cascade_picks_first` — a log with a root error followed by five induced
|
||||
ones yields the root.
|
||||
5. `a5_unknown_tool_fallback` — a log from an unlisted tool yields a signature and
|
||||
names the generic rule.
|
||||
6. `a6_no_model_calls` — run under a transport that panics on request; assert
|
||||
every fixture extracts.
|
||||
7. `a7_latency` — 50KB log extracts in under 50ms.
|
||||
8. `a8_tool_in_identity` — the same normalised text under two different tools
|
||||
hashes differently.
|
||||
9. `a9_explain_names_rule` — `mem sig explain` output identifies the matching
|
||||
rule for each fixture.
|
||||
|
||||
**Command:** `cargo test -p mem-core signature`
|
||||
|
||||
**False pass:**
|
||||
- Fixtures generated by re-running the same command in the same directory at
|
||||
nearly the same time. Paths and timestamps barely differ and assertion 1 passes
|
||||
with normalisation disabled. The two logs must come from genuinely different
|
||||
runs — different machine, different day, different workspace.
|
||||
- Asserting only 1 and 2. A hash of the whole log satisfies 2 and fails 1; a
|
||||
constant satisfies 1 and fails 2. Both are required, and 3 is what proves the
|
||||
mechanism rather than the outcome.
|
||||
- Hand-written fixture logs. They omit exactly the volatile noise the task
|
||||
exists to strip.
|
||||
|
||||
## Traps
|
||||
|
||||
- Normalising too hard. Replacing every number makes `exit status 1` and
|
||||
`exit status 137` collide, and OOM stops being distinguishable from a test
|
||||
failure. Numbers that are part of the error's meaning must survive.
|
||||
- Anchoring on the last line. It is usually `##[error]Process completed with exit
|
||||
code 1`, which is identical across every failure GitHub Actions ever produced.
|
||||
- Letting the fallback silently handle a tool that has a rule set. If a rule set
|
||||
exists and does not match, that is a signal the tool changed its output format;
|
||||
report it rather than quietly degrading.
|
||||
|
||||
---
|
||||
|
||||
Background: [DESIGN.md](../DESIGN.md) — tool context, retrieval tiers
|
||||
Reference in New Issue
Block a user