poimen-memory/tasks/M3.6.4-reference-cycle-guard.md

# M3.6.4 — Reference text cannot re-enter as evidence

| Field | Value |
|---|---|
| Phase | M3.6 — Reference corpora |
| Size | M — 1–3 days |
| Status | ⬜ Not started |
| Flags | — |
| Spec | inlined below |
| Blocks | M3.6.6 |
| Depends | M3.6.3, M4.2 |

## Goal

Stop a retrieved manual page from coming back through the front door as a project
finding.

## Facts (inlined — no spec read needed)

The cycle is M4.2's, with documentation substituted for emitted skills:

```
agent queries memory, gets an R section
        │
        ▼
section is pasted into the agent's context
        │
        ▼
appears verbatim in that session's transcript
        │
        ▼
transcript ingested; the gate sees upstream doc text
        │
        ▼
"kubectl describe shows events" becomes an L1 project memory
```

The gate is *right* to accept it — the chunk genuinely contains information about
the question. That is what makes this dangerous rather than merely noisy: no
threshold tuning catches it, because the text really is relevant. Only knowing
that the system emitted the text itself distinguishes the two cases.

**Reuse M4.2, do not rebuild it.** M4.2 already computes normalised shingle
overlap against an artifact manifest and tags matching records `derived: true`,
excluding them from evidence while keeping them in the log so the exclusion is
auditable. R chunks are a second artifact kind in that same manifest. A parallel
matcher would drift from it and double the tuning surface.

```jsonl
{"kind":"reference","name":"kubectl/common-issues","sha256":"cd34…","shingles":[…],"emitted_at":"…"}
{"kind":"skill","name":"infra-root-causes","sha256":"ab12…","shingles":[…],"emitted_at":"…"}
```

**Threshold pressure differs by kind and this is the real work.** A skill is
emitted once and quoted rarely. Documentation is quoted constantly and partially
— one command from a fifty-line cheatsheet. Shingle overlap against a whole R
chunk will sit far below M4.2's 0.8 default for exactly the case that matters, so
matching must be at section granularity with its own threshold, tuned and logged
separately. One shared matcher, two configured thresholds.

**A false positive here is costly and must stay visible.** Excluding a genuine
discussion *about* `kubectl` because it quotes two lines of the cheatsheet
silently drops real evidence. Every exclusion emits `derived_excluded` naming the
matched artifact, and `mem verify` can list them for audit.

## Steps

1. Generalise M4.2's manifest to `vault/.artifacts.jsonl` with a `kind` field;
   keep skills writing to it unchanged.
2. `mem ref add`/`sync` append `kind: "reference"` entries per R chunk;
   tombstones remove them.
3. Add per-kind thresholds to the matcher config; default reference threshold
   lower than the skill threshold, and record the value in the exclusion event.
4. Extend `mem verify --derived-filter` to assert no L0 evidence node matches a
   live R artifact.
5. `mem verify --exclusions` lists recent `derived_excluded` events with the
   matched artifact and overlap score, for false-positive review.

## Acceptance

- A session transcript containing a verbatim R section is excluded from evidence.
- The same transcript still appears in the log, tagged, with the match named.
- A session that merely *mentions* the tool without quoting it is not excluded.
- Skill exclusion behaviour from M4.2 is unchanged.
- Removing a corpus removes its manifest entries; previously excluded text is not
  retroactively rewritten in the log.

## Verify

**Harness:** fixture corpus ingested as R, plus three synthetic transcripts — one
quoting a section verbatim, one paraphrasing it heavily, one discussing the tool
without quoting. Deterministic embedder.

**Integration test** — `tests/it_reference_cycle.rs`:
1. `a1_verbatim_quote_excluded` — the quoting transcript produces zero L0
   evidence nodes; assert a `derived_excluded` event naming the R artifact.
2. `a2_discussion_not_excluded` — the non-quoting transcript produces evidence
   normally. This is the false-positive guard and it is the assertion that fails
   when the threshold is set too low.
3. `a3_partial_quote_caught` — the transcript quoting ~10 lines of a 50-line
   section is excluded, proving section-granularity matching rather than
   whole-chunk overlap.
4. `a4_skill_path_unchanged` — run M4.2's own test fixtures; assert identical
   results before and after the manifest generalisation.
5. `a5_exclusion_is_auditable` — every exclusion event carries artifact name,
   overlap score and the threshold in force.
6. `a6_tombstone_removes_manifest_entry` — `mem ref rm`, then assert the R
   entries are gone from the manifest and the same transcript now ingests
   normally.
7. `a7_verify_catches_leak` — hand-insert an L0 node whose text matches an R
   artifact; assert `mem verify --derived-filter` exits non-zero and names it.
8. `a8_no_retroactive_log_edit` — after `rm`, assert prior `derived_excluded`
   records are still present and unmodified.

**Command:** `cargo test -p mem-ingest reference_cycle && cargo test -p mem-cli verify`

**False pass:**
- Testing only the verbatim case. Verbatim is easy and a whole-chunk hash catches
  it; assertion 3 is the one that distinguishes a working matcher, and assertion
  2 is the one that proves it is not simply excluding everything that mentions
  the tool.
- Asserting exclusion by checking evidence count is zero. A filter that is
  accidentally excluding *all* records also yields zero; assertion 2 has to run
  in the same test binary.
- Reusing M4.2's threshold unchanged and declaring it done. The default is tuned
  for whole-artifact quoting; assertion 3 fails against it, which is the point.

## Traps

- Registering R chunks in the manifest before they are committed to the log. A
  failed ingest then leaves manifest entries that exclude evidence for a corpus
  that does not exist, and the symptom is missing memories with no obvious cause.
- Normalising differently in the two paths. If the shingler treats markdown
  tables differently at emit-time and at ingest-time, overlap collapses and the
  filter silently stops firing — same failure M4.2 already warns about, now with
  two producers to keep in step.
- Letting the exclusion event omit the threshold. A tuning change makes every
  historical exclusion uninterpretable, and this filter will be tuned.

---

Background: [DESIGN.md](../DESIGN.md) — reference corpora, skills · [M4.2](M4.2-derived-filter.md)