91 lines
3.7 KiB
Markdown
91 lines
3.7 KiB
Markdown
|
# T5.1 — `TaskId` at spawn
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
|---|---|
|
|||
|
|
| Phase | P5 — Grading |
|
|||
|
|
| Size | S — under 1 day |
|
|||
|
|
| Status | Not started |
|
|||
|
|
| Flags | — |
|
|||
|
|
| Spec | inlined below |
|
|||
|
|
| Blocks | T5.5 onward. Cannot be retrofitted |
|
|||
|
|
|
|||
|
|
## Goal
|
|||
|
|
|
|||
|
|
Every run carries a `TaskId` from spawn. Required, no default, no derivation from
|
|||
|
|
`RunId`.
|
|||
|
|
|
|||
|
|
## Facts (inlined — no spec read needed)
|
|||
|
|
|
|||
|
|
- `TaskId` is the **comparison-group key**: a content hash of the task input
|
|||
|
|
**before any workflow touches it**.
|
|||
|
|
- Hashing the prompt does not work — the workflow changes the prompt by
|
|||
|
|
construction, which is the entire point of a variant.
|
|||
|
|
- **`TaskId` cannot be backfilled.** A run recorded without one is permanently
|
|||
|
|
ungroupable and useless to the learning loop. Hence: required at spawn, no
|
|||
|
|
`Default`, no `From<RunId>` (T0.1).
|
|||
|
|
- **Land this during P1**, even though nothing consumes it until P5. It is a
|
|||
|
|
small task deep in P5 and it is the one item on the critical path that a later
|
|||
|
|
phase cannot repair.
|
|||
|
|
- What defines `TaskId` for a given user is open — ticket id, input fixture, or a
|
|||
|
|
hash of the pre-workflow goal. The framework provides the type and a default
|
|||
|
|
hasher and lets it be overridden. Settle it before any run is recorded.
|
|||
|
|
|
|||
|
|
## Steps
|
|||
|
|
|
|||
|
|
1. Add `task: TaskId` as a **required positional field** of the spawn parameters.
|
|||
|
|
Not `Option`, not a builder method that can be omitted.
|
|||
|
|
2. Provide a default hasher over the pre-workflow task input, and a hook to
|
|||
|
|
override it. Document that the choice is permanent for existing data.
|
|||
|
|
3. Record `TaskId` on the run's spawn event so it lands in the log, not only in
|
|||
|
|
materialized state.
|
|||
|
|
4. Expose it on `EpisodeView` for the grading path.
|
|||
|
|
5. Add the `trybuild` compile-fail case: a spawn call omitting `TaskId`.
|
|||
|
|
|
|||
|
|
## Acceptance
|
|||
|
|
|
|||
|
|
- **Spawning without a `TaskId` fails to compile.**
|
|||
|
|
|
|||
|
|
## Verify
|
|||
|
|
|
|||
|
|
**Harness:** `trybuild`, plus a full run to prove the id reaches storage.
|
|||
|
|
|
|||
|
|
**Integration test** — `tests/it_taskid_required.rs`:
|
|||
|
|
1. `trybuild` compile-fail: a spawn call omitting `TaskId`. Assert the expected
|
|||
|
|
stderr **names the missing field**, not merely "does not compile".
|
|||
|
|
2. Run a real run; read the **log** (not materialized state) and assert the spawn
|
|||
|
|
event carries the `TaskId`.
|
|||
|
|
3. Assert `EpisodeView` exposes it, so the grading path can group on it.
|
|||
|
|
4. Determinism: hash the same task input twice, in two processes; assert equal
|
|||
|
|
`TaskId`.
|
|||
|
|
5. Independence: assert two runs of the **same** task input share a `TaskId`, and
|
|||
|
|
that two different inputs do not. Both directions.
|
|||
|
|
6. Assert the `TaskId` is **not** derived from the prompt: change the workflow
|
|||
|
|
version (which changes the prompt) and assert the `TaskId` is unchanged for
|
|||
|
|
the same input.
|
|||
|
|
7. `trybuild`: `TaskId::default()` and `TaskId::from(run_id)` both fail.
|
|||
|
|
|
|||
|
|
**Command:** `cargo test -p kernel taskid && cargo test -p kernel --test compile_fail`
|
|||
|
|
|
|||
|
|
**False pass:**
|
|||
|
|
- Step 2 read from materialized state. If the id lives only in state it cannot be
|
|||
|
|
re-derived after a cold fold, and it cannot be backfilled — read the log.
|
|||
|
|
- Step 6 omitted: hashing the prompt passes steps 1–5 completely and makes every
|
|||
|
|
variant its own group, which is the exact failure that renders the learning
|
|||
|
|
loop inert.
|
|||
|
|
- A builder with a runtime `expect("task_id required")`. It passes a test that
|
|||
|
|
checks for a panic and still allows runs to be recorded without one in any code
|
|||
|
|
path the test did not cover.
|
|||
|
|
|
|||
|
|
## Traps
|
|||
|
|
|
|||
|
|
- A builder with `.task_id(...)` optional and a runtime check. Runs recorded
|
|||
|
|
before someone notices are unrecoverable.
|
|||
|
|
- Deriving it from the prompt or from `RunId`. Both compile; both make every run
|
|||
|
|
its own group of one.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
Background (not required to do this task):
|
|||
|
|
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §3, §11.5, §18 ·
|
|||
|
|
[rust-agentic-task.md](../../../rust-agentic-task.md)
|