(tasks) add tasks for harness

2026-08-17 23:05:20 -07:00
parent e2d678b118
commit 5a30d0ffc6
74 changed files with 8204 additions and 1 deletions
@@ -0,0 +1,129 @@
+# T1.3 — Run executor
+
+| Field | Value |
+|---|---|
+| Phase | P1 — Walking skeleton |
+| Size | L — over 3 days |
+| Status | Not started |
+| Flags | — |
+| Spec | inlined below |
+| Blocks | — |
+
+## Goal
+
+Drive a hardcoded three-step workflow through the full run lifecycle, emitting
+every kernel transition to the log. T3.5 later replaces the hardcoded workflow
+with the IR interpreter.
+
+## Facts (inlined — no spec read needed)
+
+Run lifecycle — the states and the legal moves:
+
+```
+        spawn
+          │
+          v
+      Scheduled ─────────────────┐
+          │ admitted             │
+          v                      │
+       Running ⇄ Suspended ──────┤
+          │ all steps terminal   │ cancel
+          v                      │
+      Verifying ─────────────────┤   ◄── rests here while verifier
+          │                      │       futures are outstanding
+          v                      │
+      Verified{pass|fail}        │
+          │                      │
+          v                      v
+       Grading ──────────────► Cancelled ●
+          │   ◄── may wait for a tournament group to fill
+          │
+          ├──► Graded ●   ──┐
+          │                 ├──► Archived ●
+          └──► Ungraded ● ──┘
+```
+
+- **Cancel is reachable from every non-terminal state**, not only `Scheduled`. A
+  run cancelled mid-step is the ordinary case — it is what a user clicking stop
+  does. Cancelling during `Verifying` or `Grading` is rarer and still legal.
+- **`Verifying` and `Grading` are states the run rests in**, not synchronous
+  branches. Resolving pass/fail inside `verify()` collapsed this in the prior
+  implementation and blocked three separate features at once: async verifiers,
+  mid-run UI, and the snapshot barrier (T4.2).
+- **`Ungraded` is terminal and sits beside `Graded`, not below it.** Grading can
+  legitimately end with no score. Without a terminal state saying so, those runs
+  rest in `Grading` forever and become permanently irreducible (T4.4).
+- `Suspended` releases the worker: a run awaiting human approval or a webhook
+  must not hold an executor slot across a human decision.
+- Concurrency shape: unbounded across runs; **serial** steps within a run;
+  strictly serial attempts within a step; fan-out only for declared `Parallel`
+  branches and for verifiers of one attempt. The serial spine is
+  `(TenantId, RunId)`.
+
+## Steps
+
+1. Define the run-state enum and its transition function, exhaustive like T0.2's.
+   Include `Cancelled` as reachable from every non-terminal state.
+2. Build the executor loop: take the hardcoded three-step workflow, run steps
+   serially, each step producing one or more attempts (T1.4).
+3. Emit a `WorkEvent` for **every** transition through `EventLog::commit` — run
+   level and attempt level. One commit per transition, not one per run.
+4. Enter `Verifying` only when every step is terminal. Park there; do not resolve
+   verification inline.
+5. Enter `Grading` as a resting state likewise; allow it to terminate as
+   `Graded` **or** `Ungraded`.
+6. Run everything inside a `RunScope` (T1.1) so cancel propagates and no task
+   outlives the run.
+7. Add `Suspended` with lease release wired as a no-op stub in embedded mode —
+   the state must exist now, since T7.3 depends on it.
+
+## Acceptance
+
+- One run completes end to end against the stub model.
+- `Verifying` and `Grading` are observable as **distinct states in the log**, not
+  skipped or collapsed.
+
+## Verify
+
+**Harness:** embedded `redb`, stub model (T1.2), one throwaway verifier, one stub
+grader. This is the first test that exercises T0.5, T0.6, T0.8, T1.1 and T1.2
+together — treat it as the P1 integration point.
+
+**Integration test** — `tests/it_full_run.rs`:
+1. Spawn one run of the three-step workflow; drive to completion.
+2. Read the log back and assert the **run-state sequence** contains
+   `Scheduled, Running, Verifying, Grading` and a terminal state, in that order.
+   Assert `Verifying` and `Grading` each appear as their own record — not
+   inferred, not skipped.
+3. Assert every step produced attempt records and that no state appears out of
+   order against T0.2's table.
+4. Call `assert_refold_identical` on the finished run.
+5. Cancel matrix: start a run, cancel it from each of `Scheduled`, `Running`,
+   `Verifying`, `Grading` in turn; assert each reaches `Cancelled`.
+6. Ordering: launch 20 runs concurrently, assert each run's own records are
+   totally ordered and that no two steps of one run overlap in time.
+
+**Command:** `cargo test -p executor --test it_full_run`
+
+**False pass:**
+- Asserting `Verifying` "happened" by checking a boolean on the run record. That
+  passes against a synchronous `verify()` call, which is the collapse this task
+  exists to prevent. The **log record** is the assertion.
+- A cancel test that only covers `Scheduled`. That is the one case a broken
+  implementation gets right.
+- Step 6 with a single run, where serial execution is indistinguishable from
+  accidental parallelism.
+
+## Traps
+
+- Making cancel legal only from `Scheduled`. Nobody would ship that lifecycle.
+- Treating `Verifying` as a function call. It removes the seam T4.2 needs.
+- Interleaving steps within a run "since they look independent". The serial spine
+  is `(TenantId, RunId)`; parallelism lives between runs and inside declared
+  fan-out only.
+
+---
+
+Background (not required to do this task):
+[rust-agentic-sys.md](../../../rust-agentic-sys.md) §5.2, §5.3 ·
+[rust-agentic-task.md](../../../rust-agentic-task.md)