88 lines
3.4 KiB
Markdown
88 lines
3.4 KiB
Markdown
|
# M4.3 — M4 composition gate
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
|---|---|
|
|||
|
|
| Phase | M4 — Skills |
|
|||
|
|
| Size | M — 1–3 days |
|
|||
|
|
| Status | ⬜ Not started |
|
|||
|
|
| Flags | gate |
|
|||
|
|
| Spec | inlined below |
|
|||
|
|
| Blocks | all of M4 |
|
|||
|
|
|
|||
|
|
## Goal
|
|||
|
|
|
|||
|
|
Prove the loop stays open — that a generated skill cannot silently become
|
|||
|
|
training data for the memory that generated it.
|
|||
|
|
|
|||
|
|
## Facts (inlined — no spec read needed)
|
|||
|
|
|
|||
|
|
Two properties, and both must hold. Either one alone is insufficient:
|
|||
|
|
|
|||
|
|
1. **A draft is not loadable.** It lives in `_drafts/`, and pointing a real agent
|
|||
|
|
at `vault/skills/` must not surface it.
|
|||
|
|
2. **A promoted skill's text never enters evidence.** Even after a human promotes
|
|||
|
|
it and it appears in a session, the derived filter keeps it out of L0.
|
|||
|
|
|
|||
|
|
The gate exercises the full cycle deliberately: draft a skill, promote it, run a
|
|||
|
|
session that loads it, ingest that session, and assert the skill's content did
|
|||
|
|
not become evidence.
|
|||
|
|
|
|||
|
|
Property 1 is tested with the real loader, not by inspecting paths — the
|
|||
|
|
question is what an agent actually sees.
|
|||
|
|
|
|||
|
|
## Steps
|
|||
|
|
|
|||
|
|
1. Draft a skill from a real note.
|
|||
|
|
2. `pi --skill vault/skills/ --list-skills` (or the equivalent enumeration);
|
|||
|
|
assert the draft is absent.
|
|||
|
|
3. Promote it with `git mv` out of `_drafts/`; assert it now appears.
|
|||
|
|
4. Synthesize a session transcript that quotes the promoted skill, in three
|
|||
|
|
grades: verbatim, reformatted, and merely referenced.
|
|||
|
|
5. Ingest it; assert the first two are excluded and the third is kept.
|
|||
|
|
6. Run `mem verify --derived-filter`; assert clean.
|
|||
|
|
7. Commit `expected/m4.3.txt`; diff.
|
|||
|
|
|
|||
|
|
## Acceptance
|
|||
|
|
|
|||
|
|
- Draft absent from a real loader's skill enumeration.
|
|||
|
|
- Promoted skill present.
|
|||
|
|
- Verbatim and reformatted quotes excluded from evidence; a bare mention kept.
|
|||
|
|
- `mem verify --derived-filter` reports zero leaks.
|
|||
|
|
|
|||
|
|
## Verify
|
|||
|
|
|
|||
|
|
**Harness:** the real `pi` binary for enumeration, seeded vault and log.
|
|||
|
|
|
|||
|
|
**Integration test** — `verify/m4.3.sh` diffed against `expected/m4.3.txt`:
|
|||
|
|
1. `a1_draft_not_loadable` — enumerate skills; assert the draft name is absent.
|
|||
|
|
2. `a2_promoted_is_loadable` — after `git mv`, assert present. Proves assertion 1
|
|||
|
|
is not passing because the loader is broken.
|
|||
|
|
3. `a3_verbatim_excluded` — ingest, assert no L0 node matches.
|
|||
|
|
4. `a4_reformatted_excluded` — same.
|
|||
|
|
5. `a5_mention_kept` — the referencing record survives as evidence.
|
|||
|
|
6. `a6_verify_clean` — `mem verify --derived-filter` exits 0.
|
|||
|
|
7. `a7_exclusions_auditable` — assert `derived_excluded` events name the artifact.
|
|||
|
|
|
|||
|
|
**Command:** `bash verify/m4.3.sh | diff - expected/m4.3.txt`
|
|||
|
|
|
|||
|
|
**False pass:**
|
|||
|
|
- Asserting the draft is absent without assertion 2. If the loader silently fails
|
|||
|
|
to enumerate anything, assertion 1 passes trivially and the guarantee is
|
|||
|
|
untested.
|
|||
|
|
- Testing exclusion only on the draft. The draft is not the risk — a *promoted*
|
|||
|
|
skill is the one that actually reaches sessions, and it is the one the filter
|
|||
|
|
must catch.
|
|||
|
|
|
|||
|
|
## Traps
|
|||
|
|
|
|||
|
|
- Automating promotion inside the gate script and leaving it there. The gate
|
|||
|
|
needs to promote something to test the promoted path; make it obvious that the
|
|||
|
|
script is the only place it happens, and that production has no such path.
|
|||
|
|
- Reading a passing gate as "the cycle is impossible". It means the two guards
|
|||
|
|
hold today. Adding a new emitted artifact type without adding it to the
|
|||
|
|
manifest reopens the loop, and only assertion 6 will notice.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
Background: [DESIGN.md](../DESIGN.md) — Skills, Risks
|