88 lines
3.4 KiB
Markdown
88 lines
3.4 KiB
Markdown
# M4.3 — M4 composition gate
|
||
|
||
| Field | Value |
|
||
|---|---|
|
||
| Phase | M4 — Skills |
|
||
| Size | M — 1–3 days |
|
||
| Status | ⬜ Not started |
|
||
| Flags | gate |
|
||
| Spec | inlined below |
|
||
| Blocks | all of M4 |
|
||
|
||
## Goal
|
||
|
||
Prove the loop stays open — that a generated skill cannot silently become
|
||
training data for the memory that generated it.
|
||
|
||
## Facts (inlined — no spec read needed)
|
||
|
||
Two properties, and both must hold. Either one alone is insufficient:
|
||
|
||
1. **A draft is not loadable.** It lives in `_drafts/`, and pointing a real agent
|
||
at `vault/skills/` must not surface it.
|
||
2. **A promoted skill's text never enters evidence.** Even after a human promotes
|
||
it and it appears in a session, the derived filter keeps it out of L0.
|
||
|
||
The gate exercises the full cycle deliberately: draft a skill, promote it, run a
|
||
session that loads it, ingest that session, and assert the skill's content did
|
||
not become evidence.
|
||
|
||
Property 1 is tested with the real loader, not by inspecting paths — the
|
||
question is what an agent actually sees.
|
||
|
||
## Steps
|
||
|
||
1. Draft a skill from a real note.
|
||
2. `pi --skill vault/skills/ --list-skills` (or the equivalent enumeration);
|
||
assert the draft is absent.
|
||
3. Promote it with `git mv` out of `_drafts/`; assert it now appears.
|
||
4. Synthesize a session transcript that quotes the promoted skill, in three
|
||
grades: verbatim, reformatted, and merely referenced.
|
||
5. Ingest it; assert the first two are excluded and the third is kept.
|
||
6. Run `mem verify --derived-filter`; assert clean.
|
||
7. Commit `expected/m4.3.txt`; diff.
|
||
|
||
## Acceptance
|
||
|
||
- Draft absent from a real loader's skill enumeration.
|
||
- Promoted skill present.
|
||
- Verbatim and reformatted quotes excluded from evidence; a bare mention kept.
|
||
- `mem verify --derived-filter` reports zero leaks.
|
||
|
||
## Verify
|
||
|
||
**Harness:** the real `pi` binary for enumeration, seeded vault and log.
|
||
|
||
**Integration test** — `verify/m4.3.sh` diffed against `expected/m4.3.txt`:
|
||
1. `a1_draft_not_loadable` — enumerate skills; assert the draft name is absent.
|
||
2. `a2_promoted_is_loadable` — after `git mv`, assert present. Proves assertion 1
|
||
is not passing because the loader is broken.
|
||
3. `a3_verbatim_excluded` — ingest, assert no L0 node matches.
|
||
4. `a4_reformatted_excluded` — same.
|
||
5. `a5_mention_kept` — the referencing record survives as evidence.
|
||
6. `a6_verify_clean` — `mem verify --derived-filter` exits 0.
|
||
7. `a7_exclusions_auditable` — assert `derived_excluded` events name the artifact.
|
||
|
||
**Command:** `bash verify/m4.3.sh | diff - expected/m4.3.txt`
|
||
|
||
**False pass:**
|
||
- Asserting the draft is absent without assertion 2. If the loader silently fails
|
||
to enumerate anything, assertion 1 passes trivially and the guarantee is
|
||
untested.
|
||
- Testing exclusion only on the draft. The draft is not the risk — a *promoted*
|
||
skill is the one that actually reaches sessions, and it is the one the filter
|
||
must catch.
|
||
|
||
## Traps
|
||
|
||
- Automating promotion inside the gate script and leaving it there. The gate
|
||
needs to promote something to test the promoted path; make it obvious that the
|
||
script is the only place it happens, and that production has no such path.
|
||
- Reading a passing gate as "the cycle is impossible". It means the two guards
|
||
hold today. Adding a new emitted artifact type without adding it to the
|
||
manifest reopens the loop, and only assertion 6 will notice.
|
||
|
||
---
|
||
|
||
Background: [DESIGN.md](../DESIGN.md) — Skills, Risks
|