poimen/tasks/T4.1-verifier-port.md

T4.1 — Verifier port

Field	Value
Phase	P4 — Verification
Size	S — under 1 day
Status	Not started
Flags	—
Spec	inlined below
Blocks	T4.2

Goal

The verification port with a fail-closed contract enforced by the framework, not by the verifier author.

Facts (inlined — no spec read needed)

#[async_trait]
pub trait Verifier: Send + Sync {
    fn id(&self) -> VerifierId;
    /// Any error, panic or timeout MUST resolve to `Fail`. A verifier that
    /// throws or hangs can never report `Pass`.
    async fn verify(&self, cx: &VerifierCtx) -> VerifierOutcome;
}

• Verification returns ground truth. Grading attributes cause and ranks. Collapse them and the system grades its own homework.
• The framework cannot break the agent it runs. Observation, verification and grading failures degrade the record, never the work. Any code path where a verifier or grader can fail into an agent's execution is a defect.
• Fail-closed means the framework converts panic, error and timeout into Fail. A contract documented in a doc comment and implemented by each author is not a contract.
• Never let a rubric judge what a verifier can check. Every criterion that can be made mechanical should be a Verifier — deterministic, cheap, and not subject to judge drift.

Steps

1. Define VerifierId, VerifierOutcome (pass/fail plus detail) and the trait above.
2. Write the framework-side runner that wraps every verify call:
   • tokio::time::timeout at cx.deadline → Fail
   • FutureExt::catch_unwind (or a spawn-and-join boundary) for panics → Fail
   • any Err → Fail
3. Make the runner the only call path. Verifier::verify is never invoked directly from the executor.
4. Record which failure mode produced a Fail — timeout, panic and error are the same outcome but different operational signals.
5. Run verifiers for one attempt as a fan-out/join; they are independent checks.
6. Fault-inject all three modes in tests with real verifiers that panic, hang and error.

Acceptance

• Verifiers that panic, hang past deadline, and return an error all resolve to Fail — fault-injected, not asserted by comment.

Verify

Harness: three real misbehaving verifiers — one that panics, one that loops past the deadline, one that returns Err. Fault-injected, not mocked.

Integration test — tests/it_verifier_fail_closed.rs:

1. Run a real run with each bad verifier in turn.
2. Assert each resolves to Fail.
3. Assert the run itself completes and reaches a terminal state — the framework must not break the agent it runs. A panicking verifier taking the run down is the primary failure being tested for.
4. Assert the recorded failure mode distinguishes timeout / panic / error, since they are the same outcome but different operational signals.
5. Assert a verifier that panics does not poison shared state: run a second, healthy verifier afterwards in the same process and assert it returns Pass.
6. Fan-out: three verifiers on one attempt, one of them panicking; assert the other two still return their own results.
7. Assert no code path can produce Pass from an Err — a trybuild case if verify is only reachable through the framework runner.

Command: cargo test -p verify fail_closed

False pass:

• Testing the runner in isolation with a mock verifier rather than in a real run. Step 3 — the run surviving — only means something end to end.
• Catching the panic at the test boundary with catch_unwind in the test, which makes a framework that does not catch it look fine.
• Step 5 omitted: a panic that leaves a mutex poisoned passes steps 1–4 and breaks every subsequent verification in the process.

Traps

• Letting a panic unwind into the executor task. It takes the run with it, which is the exact inversion the first principle forbids.
• A Result<VerifierOutcome, E> signature that pushes the decision onto callers. One caller will map Err to Pass by accident.

---

Background (not required to do this task): rust-agentic-sys.md §1, §10, §11.7 · rust-agentic-task.md