(tasks) add tasks for harness

2026-08-17 23:05:20 -07:00
parent e2d678b118
commit 5a30d0ffc6
74 changed files with 8204 additions and 1 deletions
@@ -0,0 +1,108 @@
+# T4.5 — P4 composition gate
+
+| Field | Value |
+|---|---|
+| Phase | P4 — Verification |
+| Size | M — 1 to 3 days |
+| Status | Not started |
+| Flags | gate |
+| Spec | inlined below |
+| Blocks | P5 |
+
+## Goal
+
+Prove the verifier port, the snapshot barrier, lazy blob access and the retention
+guard hold **simultaneously**, and that any verifier implementation is
+interchangeable.
+
+**Phase gate criterion:** fail-closed proven by fault injection.
+
+## Facts (inlined — no spec read needed)
+
+- **The framework cannot break the agent it runs.** Verification failures degrade
+  the record, never the work. Any path where a verifier can fail into an agent's
+  execution is a defect — and the composition is where such a path appears, since
+  no single verifier test runs a full fleet.
+- The properties no single P4 task owns:
+  - a **panicking** verifier must not disturb a **concurrent** healthy verifier's
+    snapshot, nor the run;
+  - the barrier must hold while **all three** external mutators fire, including
+    at once;
+  - laziness must survive fan-out — one text-reading verifier must not cause the
+    exit-code verifiers beside it to fetch anything;
+  - retention must refuse while **any** verifier is outstanding, which is the
+    `Verifying` case T4.4 checks and the composition makes real.
+- Verification decides, grading explains. A gate that lets a verifier outcome be
+  influenced by grading state has collapsed the two.
+
+## Steps
+
+1. Assemble a verifier fleet on one attempt: exit-code, text-reading, panicking,
+   hanging, erroring — five implementations, one fan-out.
+2. Fire all three snapshot mutators concurrently while the fleet is mid-flight.
+3. Assert every fail-closed outcome, every unaffected outcome, the blob call
+   count, and the retention refusal, in one test.
+4. Swap the fleet for a completely different set of verifier implementations;
+   assert the run's outcome shape is unchanged.
+5. Make this the required CI job gating P5.
+
+## Acceptance
+
+- With the five-verifier fleet on one attempt: the three bad verifiers resolve to
+  `Fail`, the two good ones return their own results, and **the run completes**.
+- The snapshot holds across all three mutators fired concurrently.
+- Exit-code verifiers record **zero** blob reads even while a sibling reads text.
+- Reduction is refused for the whole duration of `Verifying`.
+
+## Verify
+
+**Harness:** five verifier implementations; a counting `BlobStore` wrapper per
+verifier, not one shared counter; rendezvous channels so mutators fire only once
+every verifier has demonstrably parked.
+
+**Integration test** — `tests/it_p4_composition.rs`:
+1. **Fleet fail-closed:** assert panic → `Fail`, hang → `Fail`, error → `Fail`,
+   and that the exit-code and text verifiers return their **own** correct
+   results. A fleet where one bad verifier poisons the others is the failure.
+2. **Run survives:** assert the run reaches a terminal state and that agent work
+   was never blocked — compare run duration against a no-verifier baseline.
+3. **Concurrent mutators:** with the fleet parked, fire a rewind, a cancel and an
+   intent recovery **at once**. Assert every held `EpisodeView` is byte-identical
+   to its entry snapshot.
+4. **Per-verifier laziness:** assert the exit-code verifiers' counters are
+   **exactly 0** while the text verifier's is exactly the number of refs it
+   requested. A single shared counter hides this.
+5. **Retention interlock:** attempt reduction at three points — during
+   `Verifying`, after `Verifying` but during `Grading`, and after terminal
+   `Ungraded`. Assert refuse, refuse, permit.
+6. **Verifier interchange:** re-run the entire test with a second, unrelated set
+   of five verifier implementations. Assert identical run-level outcomes.
+7. **Isolation after panic:** run a healthy verifier in the same process **after**
+   the panicking one; assert `Pass`. A poisoned mutex passes every earlier step.
+8. **Regression:** re-run P0–P3 gates and all P4 suites.
+
+**Command:** `cargo test -p verify --test it_p4_composition -- --test-threads=1`
+
+**False pass:**
+- Running the five verifiers **sequentially**. Fan-out is the declared
+  concurrency shape, and cross-contamination between verifiers only exists
+  concurrently.
+- Step 3 firing the mutators one at a time. Each is already covered by T4.2; the
+  gate is about them arriving together.
+- One shared blob counter in step 4 — the sibling's reads mask the exit-code
+  verifiers' zero.
+- Step 5 testing only the `Grading` refusal, leaving the `Ungraded`-is-permitted
+  half untested. That is the half that strands storage in production.
+
+## Traps
+
+- Catching the verifier panic in the test rather than asserting the framework
+  catches it.
+- Asserting the run "did not error" instead of asserting it reached a terminal
+  state with a complete log.
+
+---
+
+Background (not required to do this task):
+[rust-agentic-sys.md](../../../rust-agentic-sys.md) §1, §10 ·
+[rust-agentic-task.md](../../../rust-agentic-task.md)