poimen-memory/tasks/M3.7.7-signature-extraction.md

M3.7.7 — Failure signature extraction and normalisation

Field	Value
Phase	M3.7 — Tool context
Size	M — 1–3 days
Status	🟡 In progress — core implemented in lesson.rs
Flags	—
Spec	inlined below
Blocks	M3.7.6
Depends	M0.2, M2.3

Goal

Reduce 50KB of failure output to a short string that is byte-identical the next time the same thing breaks.

Existing code (already implemented)

crates/mem-core/src/lesson.rs (871 lines) already contains:

Function	Lines	What it does	Tests
extract(tool, output)	200–260	Rule-based signature extraction per tool	same_failure_different_runs_same_hash, different_failures_differ, cascade_lines_are_skipped, code_declaration_does_not_split_a_failure, tool_is_part_of_identity, unknown_tool_falls_back
normalise(raw)	60–170	Strips ANSI, timestamps, paths, shas, line:col, durations, addresses	strips_ansi, normalises_volatiles_but_keeps_exit_codes, error_lines_still_keep_basenames
normalise_cmd(cmd)	179–195	Harsher normalisation for commands (drops basenames)	cmd_key_ignores_temp_file_names
strip_ansi(s)	30–50	ANSI SGR sequence removal	strips_ansi
markers(tool)	—	Per-tool error line markers: npm, cargo, go, kubectl, gha, docker, terraform	—
is_cascade(line)	—	Suppresses consequence lines (##[error]Process completed...)	cascade_lines_are_skipped
is_code_declaration(line)	—	Handles npm ERR! code ERESOLVE prefix dedup	code_declaration_does_not_split_a_failure
Signature struct	199–208	{ tool, raw, normalised, sig_sha, rule }	—

All 10 relevant unit tests pass: cargo test -p mem-core -- lesson

What remains to complete this task

1. mem sig explain CLI command — not yet in main.rs
2. fixtures/failures/ directory — real captured logs from different runs (hand-written fixtures exist as inline test strings only)
3. Integration test file tests/it_signature.rs — the 9 assertions listed in Verify below (current tests are unit tests inside lesson.rs, not integration tests)
4. Latency test (a7) — 50KB log extracts in under 50ms
5. mem sig explain output that names the matching rule (a9)

Files

Action	Path
Exists	crates/mem-core/src/lesson.rs — core logic already here
Modify	crates/mem-cli/src/main.rs — add Commands::Sig { Explain } subcommand
Create	tests/it_signature.rs — integration tests (9 assertions)
Create	fixtures/failures/npm-run-a.txt, npm-run-b.txt, npm-different.txt — real logs
Create	fixtures/failures/cargo-run-a.txt, etc. — per-tool pairs

Facts (inlined — no spec read needed)

in:  <14000 lines of GitHub Actions log>
out: { tool: "github-actions",
       signature: "npm ERR! ERESOLVE unable to resolve dependency tree",
       sig_sha: "7f3a…",
       context: { job: "build", step: "npm ci", exit_code: 1 } }

Normalisation is the whole task. Two runs of the same failure differ in run id, timestamps, durations, temp paths, container ids, commit shas, line numbers and memory addresses. Every one of those must be stripped or the hash never matches twice and tier 1 of the lookup never fires — the feature silently degrades to vector search and nobody notices, because vector search still returns something.

Substitution list, applied before hashing:

/home/runner/work/<org>/<repo>/…   ->  <WORKSPACE>/…
2026-08-21T10:02:11.482Z           ->  <TS>
[0-9a-f]{7,40}                     ->  <SHA>
:[0-9]+:[0-9]+                     ->  :<LINE>:<COL>
0x[0-9a-f]+                        ->  <ADDR>
took 4m21s / in 132ms              ->  <DUR>
/tmp/[A-Za-z0-9]+                  ->  <TMP>

Deterministic first, model second. Most tools have a findable error line — npm ERR!, error:, Error:, FAILED, a non-zero exit with the last stderr block. Extract with rules per tool and fall back to the 3B controller only when the rules find nothing. A model in the hot path of every lookup is both slow and non-deterministic, and non-determinism here means the same failure hashes two ways.

One signature, not a fingerprint set. Take the first error that is not a consequence of an earlier one. Cascading failures produce twenty error lines and matching on the last one keys the memory to a symptom of a symptom.

Keep raw alongside sig_sha. The normalised form is unreadable to a human and the display path needs the original. Store both; hash only the normalised.

Unknown tools must degrade, not fail. No rule set for a tool means: take the last non-empty stderr block, normalise, hash. A worse signature is still a signature, and a lookup that 500s because the tool is unrecognised is useless in exactly the situation someone needs it.

Steps

1. mem-core::signature — extract(tool, raw) -> Option<Signature>.
2. Rule sets for github-actions, kubectl, npm, cargo, go, docker; a generic fallback for everything else.
3. Normalisation pipeline as above, ordered and documented; each substitution named so a mismatch is debuggable.
4. sig_sha = sha256(tool + "\n" + normalised) — tool is part of identity, since exit status 1 means different things in different tools.
5. Cascade suppression: prefer the earliest error line not preceded by another.
6. mem sig explain <file> — print extracted signature, normalised form, hash and which rule fired. This is the debugging surface for the whole tier.

Acceptance

• The same failure from two different runs produces the same sig_sha.
• Two genuinely different failures from the same tool produce different hashes.
• An unrecognised tool still produces a signature.
• Extraction on a 50KB log completes in under 50ms with no model call.
• mem sig explain names the rule that fired.

Verify

Harness: fixtures/failures/ — for each of six tools, two real logs of the same failure from different runs, plus one log of a different failure from the same tool. Real captured output, not hand-written.

Integration test — tests/it_signature.rs:

1. a1_same_failure_same_hash — for each tool, the two same-failure logs produce identical sig_sha. This is the assertion the tier depends on.
2. a2_different_failure_different_hash — the third log hashes differently.
3. a3_normalisation_removes_volatiles — assert the normalised string contains no timestamp, path, sha, line number or duration, by regex.
4. a4_cascade_picks_first — a log with a root error followed by five induced ones yields the root.
5. a5_unknown_tool_fallback — a log from an unlisted tool yields a signature and names the generic rule.
6. a6_no_model_calls — run under a transport that panics on request; assert every fixture extracts.
7. a7_latency — 50KB log extracts in under 50ms.
8. a8_tool_in_identity — the same normalised text under two different tools hashes differently.
9. a9_explain_names_rule — mem sig explain output identifies the matching rule for each fixture.

Command: cargo test -p mem-core signature

False pass:

• Fixtures generated by re-running the same command in the same directory at nearly the same time. Paths and timestamps barely differ and assertion 1 passes with normalisation disabled. The two logs must come from genuinely different runs — different machine, different day, different workspace.
• Asserting only 1 and 2. A hash of the whole log satisfies 2 and fails 1; a constant satisfies 1 and fails 2. Both are required, and 3 is what proves the mechanism rather than the outcome.
• Hand-written fixture logs. They omit exactly the volatile noise the task exists to strip.

Traps

• Normalising too hard. Replacing every number makes exit status 1 and exit status 137 collide, and OOM stops being distinguishable from a test failure. Numbers that are part of the error's meaning must survive.
• Anchoring on the last line. It is usually ##[error]Process completed with exit code 1, which is identical across every failure GitHub Actions ever produced.
• Letting the fallback silently handle a tool that has a rule set. If a rule set exists and does not match, that is a signal the tool changed its output format; report it rather than quietly degrading.

---

Background: DESIGN.md — tool context, retrieval tiers