# T4.1 — `Verifier` port | Field | Value | |---|---| | Phase | P4 — Verification | | Size | S — under 1 day | | Status | Not started | | Flags | — | | Spec | inlined below | | Blocks | T4.2 | ## Goal The verification port with a fail-closed contract enforced by the framework, not by the verifier author. ## Facts (inlined — no spec read needed) ```rust #[async_trait] pub trait Verifier: Send + Sync { fn id(&self) -> VerifierId; /// Any error, panic or timeout MUST resolve to `Fail`. A verifier that /// throws or hangs can never report `Pass`. async fn verify(&self, cx: &VerifierCtx) -> VerifierOutcome; } ``` - **Verification returns ground truth. Grading attributes cause and ranks.** Collapse them and the system grades its own homework. - **The framework cannot break the agent it runs.** Observation, verification and grading failures degrade the record, never the work. Any code path where a verifier or grader can fail into an agent's execution is a defect. - Fail-closed means the *framework* converts panic, error and timeout into `Fail`. A contract documented in a doc comment and implemented by each author is not a contract. - Never let a rubric judge what a verifier can check. Every criterion that can be made mechanical should be a `Verifier` — deterministic, cheap, and not subject to judge drift. ## Steps 1. Define `VerifierId`, `VerifierOutcome` (pass/fail plus detail) and the trait above. 2. Write the framework-side runner that wraps every `verify` call: - `tokio::time::timeout` at `cx.deadline` → `Fail` - `FutureExt::catch_unwind` (or a spawn-and-join boundary) for panics → `Fail` - any `Err` → `Fail` 3. Make the runner the only call path. `Verifier::verify` is never invoked directly from the executor. 4. Record which failure mode produced a `Fail` — timeout, panic and error are the same outcome but different operational signals. 5. Run verifiers for one attempt as a fan-out/join; they are independent checks. 6. Fault-inject all three modes in tests with real verifiers that panic, hang and error. ## Acceptance - Verifiers that panic, hang past deadline, and return an error **all resolve to `Fail`** — fault-injected, not asserted by comment. ## Verify **Harness:** three real misbehaving verifiers — one that panics, one that loops past the deadline, one that returns `Err`. Fault-injected, not mocked. **Integration test** — `tests/it_verifier_fail_closed.rs`: 1. Run a real run with each bad verifier in turn. 2. Assert each resolves to `Fail`. 3. Assert **the run itself completes** and reaches a terminal state — the framework must not break the agent it runs. A panicking verifier taking the run down is the primary failure being tested for. 4. Assert the recorded failure mode distinguishes timeout / panic / error, since they are the same outcome but different operational signals. 5. Assert a verifier that panics does **not** poison shared state: run a second, healthy verifier afterwards in the same process and assert it returns `Pass`. 6. Fan-out: three verifiers on one attempt, one of them panicking; assert the other two still return their own results. 7. Assert no code path can produce `Pass` from an `Err` — a `trybuild` case if `verify` is only reachable through the framework runner. **Command:** `cargo test -p verify fail_closed` **False pass:** - Testing the runner in isolation with a mock verifier rather than in a real run. Step 3 — the run surviving — only means something end to end. - Catching the panic at the test boundary with `catch_unwind` in the *test*, which makes a framework that does not catch it look fine. - Step 5 omitted: a panic that leaves a mutex poisoned passes steps 1–4 and breaks every subsequent verification in the process. ## Traps - Letting a panic unwind into the executor task. It takes the run with it, which is the exact inversion the first principle forbids. - A `Result` signature that pushes the decision onto callers. One caller will map `Err` to `Pass` by accident. --- Background (not required to do this task): [rust-agentic-sys.md](../../../rust-agentic-sys.md) §1, §10, §11.7 · [rust-agentic-task.md](../../../rust-agentic-task.md)