poimen-memory/tasks/M3.6.4-reference-cycle-guard.md

M3.6.4 — Reference text cannot re-enter as evidence

Field	Value
Phase	M3.6 — Reference corpora
Size	M — 1–3 days
Status	✅ COMPLETE
Flags	—
Spec	inlined below
Blocks	M3.6.6
Depends	M3.6.3, M4.2

Goal

Stop a retrieved manual page from coming back through the front door as a project finding.

Facts (inlined — no spec read needed)

The cycle is M4.2's, with documentation substituted for emitted skills:

agent queries memory, gets an R section
        │
        ▼
section is pasted into the agent's context
        │
        ▼
appears verbatim in that session's transcript
        │
        ▼
transcript ingested; the gate sees upstream doc text
        │
        ▼
"kubectl describe shows events" becomes an L1 project memory

The gate is right to accept it — the chunk genuinely contains information about the question. That is what makes this dangerous rather than merely noisy: no threshold tuning catches it, because the text really is relevant. Only knowing that the system emitted the text itself distinguishes the two cases.

Reuse M4.2, do not rebuild it. M4.2 already computes normalised shingle overlap against an artifact manifest and tags matching records derived: true, excluding them from evidence while keeping them in the log so the exclusion is auditable. R chunks are a second artifact kind in that same manifest. A parallel matcher would drift from it and double the tuning surface.

{"kind":"reference","name":"kubectl/common-issues","sha256":"cd34…","shingles":[…],"emitted_at":"…"}
{"kind":"skill","name":"infra-root-causes","sha256":"ab12…","shingles":[…],"emitted_at":"…"}

Threshold pressure differs by kind and this is the real work. A skill is emitted once and quoted rarely. Documentation is quoted constantly and partially — one command from a fifty-line cheatsheet. Shingle overlap against a whole R chunk will sit far below M4.2's 0.8 default for exactly the case that matters, so matching must be at section granularity with its own threshold, tuned and logged separately. One shared matcher, two configured thresholds.

A false positive here is costly and must stay visible. Excluding a genuine discussion about kubectl because it quotes two lines of the cheatsheet silently drops real evidence. Every exclusion emits derived_excluded naming the matched artifact, and mem verify can list them for audit.

Steps

1. Generalise M4.2's manifest to vault/.artifacts.jsonl with a kind field; keep skills writing to it unchanged.
2. mem ref add/sync append kind: "reference" entries per R chunk; tombstones remove them.
3. Add per-kind thresholds to the matcher config; default reference threshold lower than the skill threshold, and record the value in the exclusion event.
4. Extend mem verify --derived-filter to assert no L0 evidence node matches a live R artifact.
5. mem verify --exclusions lists recent derived_excluded events with the matched artifact and overlap score, for false-positive review.

Acceptance

• A session transcript containing a verbatim R section is excluded from evidence.
• The same transcript still appears in the log, tagged, with the match named.
• A session that merely mentions the tool without quoting it is not excluded.
• Skill exclusion behaviour from M4.2 is unchanged.
• Removing a corpus removes its manifest entries; previously excluded text is not retroactively rewritten in the log.

Verify

Harness: fixture corpus ingested as R, plus three synthetic transcripts — one quoting a section verbatim, one paraphrasing it heavily, one discussing the tool without quoting. Deterministic embedder.

Integration test — tests/it_reference_cycle.rs:

1. a1_verbatim_quote_excluded — the quoting transcript produces zero L0 evidence nodes; assert a derived_excluded event naming the R artifact.
2. a2_discussion_not_excluded — the non-quoting transcript produces evidence normally. This is the false-positive guard and it is the assertion that fails when the threshold is set too low.
3. a3_partial_quote_caught — the transcript quoting ~10 lines of a 50-line section is excluded, proving section-granularity matching rather than whole-chunk overlap.
4. a4_skill_path_unchanged — run M4.2's own test fixtures; assert identical results before and after the manifest generalisation.
5. a5_exclusion_is_auditable — every exclusion event carries artifact name, overlap score and the threshold in force.
6. a6_tombstone_removes_manifest_entry — mem ref rm, then assert the R entries are gone from the manifest and the same transcript now ingests normally.
7. a7_verify_catches_leak — hand-insert an L0 node whose text matches an R artifact; assert mem verify --derived-filter exits non-zero and names it.
8. a8_no_retroactive_log_edit — after rm, assert prior derived_excluded records are still present and unmodified.

Command: cargo test -p mem-ingest reference_cycle && cargo test -p mem-cli verify

False pass:

• Testing only the verbatim case. Verbatim is easy and a whole-chunk hash catches it; assertion 3 is the one that distinguishes a working matcher, and assertion 2 is the one that proves it is not simply excluding everything that mentions the tool.
• Asserting exclusion by checking evidence count is zero. A filter that is accidentally excluding all records also yields zero; assertion 2 has to run in the same test binary.
• Reusing M4.2's threshold unchanged and declaring it done. The default is tuned for whole-artifact quoting; assertion 3 fails against it, which is the point.

Traps

• Registering R chunks in the manifest before they are committed to the log. A failed ingest then leaves manifest entries that exclude evidence for a corpus that does not exist, and the symptom is missing memories with no obvious cause.
• Normalising differently in the two paths. If the shingler treats markdown tables differently at emit-time and at ingest-time, overlap collapses and the filter silently stops firing — same failure M4.2 already warns about, now with two producers to keep in step.
• Letting the exclusion event omit the threshold. A tuning change makes every historical exclusion uninterpretable, and this filter will be tuned.

---

Background: DESIGN.md — reference corpora, skills · M4.2