T5.10 — Attempt tournaments

Goal

Grade the attempts of one step as a comparison group. On by default, even though T5.8/T5.9 are not, because it costs no extra agent runs.

Where comparison groups come from:

Source	Group	Cost	Needs
Attempts	attempts 1..N of one step, identical context	free, already recorded	nothing
Recurring tasks	runs sharing a `TaskId` over time	free, slow to fill	`TaskId`
Replay	one task re-executed under N variants	N full runs	blobs, sandbox

Attempt tournaments first. Retries are already on disk and attempt N+1 never mutates attempt N, so the attempts of one step are a group on identical context — the cheapest per-step credit signal available, and the only source of per-step credit the system has.
Two paths, not one:
- Same-outcome attempts go to the judge. The question there is "which failure got further" — which no verifier can answer.
- A fail→pass pair is never judged. The verifier has already ordered it; asking a judge which is better asks it to re-decide what verification decided. The pair is consumed structurally: what differed between attempt N and N+1 — context partition, tool selection, prompt — is attributed to the StepId as the change that turned a fail into a pass.
The bracketing rule has no exemptions. If a pairing crosses an outcome class, it is evidence for attribution, never input to a judge. This is the place the rule is easiest to break, because the two attempts sit side by side on disk and look like a free comparison.
Per-step credit attributes to StepId, which is why the StepId stability contract (T3.4) is load-bearing rather than cosmetic.

Build the group: all attempts of one (BranchKey, StepId), partitioned by verifier outcome class.
Route by class before any judge call. Same-class subsets go to the comparison path; cross-class pairs go to the attribution path. Make this a single branch point so it cannot be bypassed.
Comparison path: hand the same-class subset to the configured strategy.
Attribution path: diff attempt N against N+1 across context partition, tool selection and prompt ref; emit the delta attributed to StepId. No judge call.
Persist per-step credit keyed by (TenantId, WorkflowVersion, StepId).
Instrument the Judge::compare counter and assert zero cross-class calls — count calls rather than inspecting pairs, since a pair-inspection test passes against an implementation that builds the pair and then filters it late.

A run with three attempts produces per-step credit attributed to StepId.
A group containing both a failed and a passed attempt issues zero Judge::compare calls across the outcome boundary — asserted by counting calls, not by inspecting pairs.

Harness: a run with three attempts, plus a counting mock judge. The counter is the instrument — pair inspection is not.

Integration test — tests/it_attempt_tournament.rs:

Three attempts on one step, all failing. Assert compare is called on the same-outcome subset and per-step credit is attributed to the StepId.
Mixed group — the guard: attempts 1 and 2 fail, attempt 3 passes. Assert zero Judge::compare calls cross the outcome boundary, counted at the judge. Assert the fail→pass pair produced a structural attribution record instead.
Assert the attribution names the actual delta — plant a context-partition difference between attempts 2 and 3 and assert it appears in the record.
Assert credit is keyed (TenantId, WorkflowVersion, StepId) and survives a workflow version bump that retains the StepId (T3.4).
Assert this path runs with the tournament flags off — it is on by default while T5.8/T5.9 are not.
Zero-cost assertion: no agent runs are spawned by grading; assert the run counter is unchanged.
Single-attempt step: assert no comparison and no spurious credit record.

Command: cargo test -p grading attempt_tournament

False pass:

Inspecting the pair list rather than counting judge calls. An implementation that builds the cross-class pair and filters it just before dispatch passes pair inspection and still sends it under a later refactor. Count at the judge boundary.
Step 2 written with all-failing attempts, where no boundary exists to cross. The mixed group is the whole test.
Step 3 omitted: emitting an empty attribution record satisfies "credit produced" while carrying no signal.

Pairing the failure with its successful retry. It is free, it looks like signal, and it makes the judge re-decide what the verifier already decided.
Attributing credit to step index rather than StepId, which breaks at the next workflow version.

Background (not required to do this task): rust-agentic-sys.md §11.4, §11.5, §11.7, §11.8 · rust-agentic-task.md