(tasks) add tasks for harness

2026-08-17 23:05:20 -07:00
parent e2d678b118
commit 5a30d0ffc6
74 changed files with 8204 additions and 1 deletions
@@ -0,0 +1,123 @@
+# T3.5 — Interpreter over the IR
+
+| Field | Value |
+|---|---|
+| Phase | P3 — Workflow as data |
+| Size | L — over 3 days |
+| Status | Not started |
+| Flags | — |
+| Spec | inlined below |
+| Blocks | — |
+
+## Goal
+
+Execute a validated `WorkflowDef` directly. Replaces T1.3's hardcoded workflow.
+
+## Facts (inlined — no spec read needed)
+
+Five step kinds to support:
+
+```rust
+pub enum StepKind {
+    Model { prompt: PromptTemplate, effort: ReasoningEffort },
+    Tool { tool: ToolId, args: ArgTemplate },
+    Parallel { branches: Vec<StepId>, join: JoinPolicy },
+    Conditional { on: Predicate, then: StepId, otherwise: Option<StepId> },
+    SubWorkflow { workflow: WorkflowId, version: VersionSelector },
+}
+```
+
+- Concurrency shape is fixed and the interpreter must not widen it:
+
+| Scope | Parallel |
+|---|---|
+| across runs | unbounded |
+| steps within a run | **serial by default** — step N+1 reads N's output |
+| `Parallel` step branches | fan-out/join, declared explicitly in the IR |
+| attempts within a step | **strictly serial** — a retry needs the prior failure |
+| verifiers for one attempt | fan-out/join |
+
+- The serial spine is `(TenantId, RunId)`. Parallelism lives between runs and
+  inside declared fan-out. Nothing else may interleave.
+- **Tool calls appear in the episode as first-class steps.** The hostcall
+  boundary already knows the identity, arguments and capabilities; recording them
+  as opaque invocations discards information the runtime is holding anyway.
+- Every external effect goes through the write-ahead intent wrapper (T2.1) — the
+  interpreter is where that is easy to bypass.
+- `SubWorkflow` version resolution is T3.6's task; the interpreter consumes an
+  already-resolved hash.
+
+## Steps
+
+1. Replace T1.3's hardcoded step list with a walk over `WorkflowDef.steps` driven
+   by `transitions`. Keep the run lifecycle from T1.3 unchanged.
+2. `Model`: render `PromptTemplate`, capture the context partition (T1.5), put
+   prompt and output blobs (T1.6), record `Usage`.
+3. `Tool`: resolve `ToolId`, render `ArgTemplate`, call through the intent
+   wrapper (T2.1), record the tool call as its own step with identity, arguments
+   and capabilities.
+4. `Parallel`: fan out inside the `RunScope` (T1.1) so cancellation still
+   propagates; apply `JoinPolicy` at the join.
+5. `Conditional`: evaluate `Predicate` against recorded state only — no ambient
+   inputs, or the fold stops being reproducible.
+6. `SubWorkflow`: execute against the resolved version, within the kernel's
+   recursion bound, recording the child run's linkage.
+7. Port P1's hardcoded workflow to YAML and diff the resulting episode structure
+   against the P1 fixture.
+
+## Acceptance
+
+- P1's hardcoded workflow, re-expressed as YAML, produces an **identical episode
+  structure**.
+
+## Verify
+
+**Harness:** P1's episode fixture as the oracle, plus the stub model and the
+external side-effect ledger from T2.2.
+
+**Integration test** — `tests/it_interpreter_parity.rs`:
+1. Express P1's hardcoded workflow as YAML. Run it through the interpreter.
+2. Assert the resulting **episode structure** matches the P1 fixture: same steps,
+   same attempt counts, same event sequence. Compare the serialized episode with
+   volatile fields (ids, timestamps) normalized — and list the normalized fields
+   explicitly, so the normalizer cannot quietly grow.
+3. One test per `StepKind`:
+   - `Model` — prompt rendered, partition and blobs captured.
+   - `Tool` — appears as a **first-class step** with identity, arguments and
+     capabilities recorded; the effect went through the intent wrapper (assert 3
+     intent records exist).
+   - `Parallel` — branches run concurrently, `JoinPolicy` applied; cancel the run
+     mid-fan-out and assert every branch task terminated.
+   - `Conditional` — both arms exercised; run the same workflow twice and assert
+     identical predicate outcomes.
+   - `SubWorkflow` — child linkage recorded; recursion past the bound rejected.
+4. **Serialization guard:** instrument step start/end times; assert no two
+   non-`Parallel` steps of one run overlap.
+5. **Bypass audit:** assert every tool invocation in the run has a matching
+   `Pending`/`Dispatched`/`Committed` intent triple. A count mismatch means a
+   direct call path exists.
+6. `assert_refold_identical` on the interpreted run.
+
+**Command:** `cargo test -p executor interpreter`
+
+**False pass:**
+- Step 2 with an over-broad normalizer that blanks anything differing. Then any
+  two episodes "match". Enumerate the normalized fields.
+- Step 5 omitted: calling a tool directly works, produces a correct-looking
+  episode, and fails T2.6's crash matrix later in a way that looks unrelated.
+- `Parallel` tested with branches that complete instantly, where concurrency is
+  indistinguishable from serial execution.
+
+## Traps
+
+- Running independent-looking steps concurrently. Serial-by-default is the
+  contract; fan-out is declared, never inferred.
+- Calling a tool directly rather than through the intent wrapper. It works, and
+  T2.6's crash matrix then fails somewhere unrelated-looking.
+- A predicate that reads the clock or the environment.
+
+---
+
+Background (not required to do this task):
+[rust-agentic-sys.md](../../../rust-agentic-sys.md) §4.1, §5.3, §13.1 ·
+[rust-agentic-task.md](../../../rust-agentic-task.md)