(plan) system review and break down plans

2026-08-19 09:52:07 -07:00
commit 9d28b63ff2
46 changed files with 5620 additions and 0 deletions
@@ -0,0 +1,87 @@
+# M4.3 — M4 composition gate
+
+| Field | Value |
+|---|---|
+| Phase | M4 — Skills |
+| Size | M — 1–3 days |
+| Status | ⬜ Not started |
+| Flags | gate |
+| Spec | inlined below |
+| Blocks | all of M4 |
+
+## Goal
+
+Prove the loop stays open — that a generated skill cannot silently become
+training data for the memory that generated it.
+
+## Facts (inlined — no spec read needed)
+
+Two properties, and both must hold. Either one alone is insufficient:
+
+1. **A draft is not loadable.** It lives in `_drafts/`, and pointing a real agent
+   at `vault/skills/` must not surface it.
+2. **A promoted skill's text never enters evidence.** Even after a human promotes
+   it and it appears in a session, the derived filter keeps it out of L0.
+
+The gate exercises the full cycle deliberately: draft a skill, promote it, run a
+session that loads it, ingest that session, and assert the skill's content did
+not become evidence.
+
+Property 1 is tested with the real loader, not by inspecting paths — the
+question is what an agent actually sees.
+
+## Steps
+
+1. Draft a skill from a real note.
+2. `pi --skill vault/skills/ --list-skills` (or the equivalent enumeration);
+   assert the draft is absent.
+3. Promote it with `git mv` out of `_drafts/`; assert it now appears.
+4. Synthesize a session transcript that quotes the promoted skill, in three
+   grades: verbatim, reformatted, and merely referenced.
+5. Ingest it; assert the first two are excluded and the third is kept.
+6. Run `mem verify --derived-filter`; assert clean.
+7. Commit `expected/m4.3.txt`; diff.
+
+## Acceptance
+
+- Draft absent from a real loader's skill enumeration.
+- Promoted skill present.
+- Verbatim and reformatted quotes excluded from evidence; a bare mention kept.
+- `mem verify --derived-filter` reports zero leaks.
+
+## Verify
+
+**Harness:** the real `pi` binary for enumeration, seeded vault and log.
+
+**Integration test** — `verify/m4.3.sh` diffed against `expected/m4.3.txt`:
+1. `a1_draft_not_loadable` — enumerate skills; assert the draft name is absent.
+2. `a2_promoted_is_loadable` — after `git mv`, assert present. Proves assertion 1
+   is not passing because the loader is broken.
+3. `a3_verbatim_excluded` — ingest, assert no L0 node matches.
+4. `a4_reformatted_excluded` — same.
+5. `a5_mention_kept` — the referencing record survives as evidence.
+6. `a6_verify_clean` — `mem verify --derived-filter` exits 0.
+7. `a7_exclusions_auditable` — assert `derived_excluded` events name the artifact.
+
+**Command:** `bash verify/m4.3.sh | diff - expected/m4.3.txt`
+
+**False pass:**
+- Asserting the draft is absent without assertion 2. If the loader silently fails
+  to enumerate anything, assertion 1 passes trivially and the guarantee is
+  untested.
+- Testing exclusion only on the draft. The draft is not the risk — a *promoted*
+  skill is the one that actually reaches sessions, and it is the one the filter
+  must catch.
+
+## Traps
+
+- Automating promotion inside the gate script and leaving it there. The gate
+  needs to promote something to test the promoted path; make it obvious that the
+  script is the only place it happens, and that production has no such path.
+- Reading a passing gate as "the cycle is impossible". It means the two guards
+  hold today. Adding a new emitted artifact type without adding it to the
+  manifest reopens the loop, and only assertion 6 will notice.
+
+---
+
+Background: [DESIGN.md](../DESIGN.md) — Skills, Risks