91 lines
3.7 KiB
Markdown
91 lines
3.7 KiB
Markdown
# T5.1 — `TaskId` at spawn
|
||
|
||
| Field | Value |
|
||
|---|---|
|
||
| Phase | P5 — Grading |
|
||
| Size | S — under 1 day |
|
||
| Status | Not started |
|
||
| Flags | — |
|
||
| Spec | inlined below |
|
||
| Blocks | T5.5 onward. Cannot be retrofitted |
|
||
|
||
## Goal
|
||
|
||
Every run carries a `TaskId` from spawn. Required, no default, no derivation from
|
||
`RunId`.
|
||
|
||
## Facts (inlined — no spec read needed)
|
||
|
||
- `TaskId` is the **comparison-group key**: a content hash of the task input
|
||
**before any workflow touches it**.
|
||
- Hashing the prompt does not work — the workflow changes the prompt by
|
||
construction, which is the entire point of a variant.
|
||
- **`TaskId` cannot be backfilled.** A run recorded without one is permanently
|
||
ungroupable and useless to the learning loop. Hence: required at spawn, no
|
||
`Default`, no `From<RunId>` (T0.1).
|
||
- **Land this during P1**, even though nothing consumes it until P5. It is a
|
||
small task deep in P5 and it is the one item on the critical path that a later
|
||
phase cannot repair.
|
||
- What defines `TaskId` for a given user is open — ticket id, input fixture, or a
|
||
hash of the pre-workflow goal. The framework provides the type and a default
|
||
hasher and lets it be overridden. Settle it before any run is recorded.
|
||
|
||
## Steps
|
||
|
||
1. Add `task: TaskId` as a **required positional field** of the spawn parameters.
|
||
Not `Option`, not a builder method that can be omitted.
|
||
2. Provide a default hasher over the pre-workflow task input, and a hook to
|
||
override it. Document that the choice is permanent for existing data.
|
||
3. Record `TaskId` on the run's spawn event so it lands in the log, not only in
|
||
materialized state.
|
||
4. Expose it on `EpisodeView` for the grading path.
|
||
5. Add the `trybuild` compile-fail case: a spawn call omitting `TaskId`.
|
||
|
||
## Acceptance
|
||
|
||
- **Spawning without a `TaskId` fails to compile.**
|
||
|
||
## Verify
|
||
|
||
**Harness:** `trybuild`, plus a full run to prove the id reaches storage.
|
||
|
||
**Integration test** — `tests/it_taskid_required.rs`:
|
||
1. `trybuild` compile-fail: a spawn call omitting `TaskId`. Assert the expected
|
||
stderr **names the missing field**, not merely "does not compile".
|
||
2. Run a real run; read the **log** (not materialized state) and assert the spawn
|
||
event carries the `TaskId`.
|
||
3. Assert `EpisodeView` exposes it, so the grading path can group on it.
|
||
4. Determinism: hash the same task input twice, in two processes; assert equal
|
||
`TaskId`.
|
||
5. Independence: assert two runs of the **same** task input share a `TaskId`, and
|
||
that two different inputs do not. Both directions.
|
||
6. Assert the `TaskId` is **not** derived from the prompt: change the workflow
|
||
version (which changes the prompt) and assert the `TaskId` is unchanged for
|
||
the same input.
|
||
7. `trybuild`: `TaskId::default()` and `TaskId::from(run_id)` both fail.
|
||
|
||
**Command:** `cargo test -p kernel taskid && cargo test -p kernel --test compile_fail`
|
||
|
||
**False pass:**
|
||
- Step 2 read from materialized state. If the id lives only in state it cannot be
|
||
re-derived after a cold fold, and it cannot be backfilled — read the log.
|
||
- Step 6 omitted: hashing the prompt passes steps 1–5 completely and makes every
|
||
variant its own group, which is the exact failure that renders the learning
|
||
loop inert.
|
||
- A builder with a runtime `expect("task_id required")`. It passes a test that
|
||
checks for a panic and still allows runs to be recorded without one in any code
|
||
path the test did not cover.
|
||
|
||
## Traps
|
||
|
||
- A builder with `.task_id(...)` optional and a runtime check. Runs recorded
|
||
before someone notices are unrecoverable.
|
||
- Deriving it from the prompt or from `RunId`. Both compile; both make every run
|
||
its own group of one.
|
||
|
||
---
|
||
|
||
Background (not required to do this task):
|
||
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §3, §11.5, §18 ·
|
||
[rust-agentic-task.md](../../../rust-agentic-task.md)
|