(tasks) add tasks for harness

2026-08-17 23:05:20 -07:00
parent e2d678b118
commit 5a30d0ffc6
74 changed files with 8204 additions and 1 deletions
@@ -0,0 +1,90 @@
+# T5.1 — `TaskId` at spawn
+
+| Field | Value |
+|---|---|
+| Phase | P5 — Grading |
+| Size | S — under 1 day |
+| Status | Not started |
+| Flags | — |
+| Spec | inlined below |
+| Blocks | T5.5 onward. Cannot be retrofitted |
+
+## Goal
+
+Every run carries a `TaskId` from spawn. Required, no default, no derivation from
+`RunId`.
+
+## Facts (inlined — no spec read needed)
+
+- `TaskId` is the **comparison-group key**: a content hash of the task input
+  **before any workflow touches it**.
+- Hashing the prompt does not work — the workflow changes the prompt by
+  construction, which is the entire point of a variant.
+- **`TaskId` cannot be backfilled.** A run recorded without one is permanently
+  ungroupable and useless to the learning loop. Hence: required at spawn, no
+  `Default`, no `From<RunId>` (T0.1).
+- **Land this during P1**, even though nothing consumes it until P5. It is a
+  small task deep in P5 and it is the one item on the critical path that a later
+  phase cannot repair.
+- What defines `TaskId` for a given user is open — ticket id, input fixture, or a
+  hash of the pre-workflow goal. The framework provides the type and a default
+  hasher and lets it be overridden. Settle it before any run is recorded.
+
+## Steps
+
+1. Add `task: TaskId` as a **required positional field** of the spawn parameters.
+   Not `Option`, not a builder method that can be omitted.
+2. Provide a default hasher over the pre-workflow task input, and a hook to
+   override it. Document that the choice is permanent for existing data.
+3. Record `TaskId` on the run's spawn event so it lands in the log, not only in
+   materialized state.
+4. Expose it on `EpisodeView` for the grading path.
+5. Add the `trybuild` compile-fail case: a spawn call omitting `TaskId`.
+
+## Acceptance
+
+- **Spawning without a `TaskId` fails to compile.**
+
+## Verify
+
+**Harness:** `trybuild`, plus a full run to prove the id reaches storage.
+
+**Integration test** — `tests/it_taskid_required.rs`:
+1. `trybuild` compile-fail: a spawn call omitting `TaskId`. Assert the expected
+   stderr **names the missing field**, not merely "does not compile".
+2. Run a real run; read the **log** (not materialized state) and assert the spawn
+   event carries the `TaskId`.
+3. Assert `EpisodeView` exposes it, so the grading path can group on it.
+4. Determinism: hash the same task input twice, in two processes; assert equal
+   `TaskId`.
+5. Independence: assert two runs of the **same** task input share a `TaskId`, and
+   that two different inputs do not. Both directions.
+6. Assert the `TaskId` is **not** derived from the prompt: change the workflow
+   version (which changes the prompt) and assert the `TaskId` is unchanged for
+   the same input.
+7. `trybuild`: `TaskId::default()` and `TaskId::from(run_id)` both fail.
+
+**Command:** `cargo test -p kernel taskid && cargo test -p kernel --test compile_fail`
+
+**False pass:**
+- Step 2 read from materialized state. If the id lives only in state it cannot be
+  re-derived after a cold fold, and it cannot be backfilled — read the log.
+- Step 6 omitted: hashing the prompt passes steps 1–5 completely and makes every
+  variant its own group, which is the exact failure that renders the learning
+  loop inert.
+- A builder with a runtime `expect("task_id required")`. It passes a test that
+  checks for a panic and still allows runs to be recorded without one in any code
+  path the test did not cover.
+
+## Traps
+
+- A builder with `.task_id(...)` optional and a runtime check. Runs recorded
+  before someone notices are unrecoverable.
+- Deriving it from the prompt or from `RunId`. Both compile; both make every run
+  its own group of one.
+
+---
+
+Background (not required to do this task):
+[rust-agentic-sys.md](../../../rust-agentic-sys.md) §3, §11.5, §18 ·
+[rust-agentic-task.md](../../../rust-agentic-task.md)