(tasks) add tasks for harness

This commit is contained in:
Story Crater Bot
2026-08-17 23:05:20 -07:00
parent e2d678b118
commit 5a30d0ffc6
74 changed files with 8204 additions and 1 deletions
@@ -0,0 +1,125 @@
# T3.1 — `WorkflowDef` IR + canonicalization
| Field | Value |
|---|---|
| Phase | P3 — Workflow as data |
| Size | L — over 3 days |
| Status | Not started |
| Flags | — |
| Spec | inlined below |
| Blocks | T3.2T3.6 |
## Goal
The validated intermediate representation every workflow format parses into, and
the canonicalization that makes `WorkflowVersion` a content hash of *meaning*
rather than of source text.
## Facts (inlined — no spec read needed)
```rust
pub struct WorkflowDef {
pub id: WorkflowId,
pub schema: SchemaVersion,
pub steps: Vec<StepDef>,
pub transitions: Vec<Transition>,
pub rubric: RubricDef,
pub budget: BudgetDef,
}
pub struct StepDef {
/// Author-assigned, stable across versions.
pub id: StepId,
pub kind: StepKind,
pub tools: ToolSelector,
pub verify: Vec<VerifierRef>,
pub retry: RetryPolicy,
pub timeout: Duration,
}
pub enum StepKind {
Model { prompt: PromptTemplate, effort: ReasoningEffort },
Tool { tool: ToolId, args: ArgTemplate },
Parallel { branches: Vec<StepId>, join: JoinPolicy },
Conditional { on: Predicate, then: StepId, otherwise: Option<StepId> },
SubWorkflow { workflow: WorkflowId, version: VersionSelector },
}
```
- **`WorkflowVersion` is the Blake3 hash of the canonicalized IR, not of the
source text.** Two YAML files differing only in key order produce the same
version — which is what makes "did this change affect results" answerable.
- Validation and canonicalization live in the **kernel** and run on the IR, so a
new format (T3.2) inherits every check without reimplementing one.
- Domain states are open: a workflow declares its own step names, ordering and
transitions as data. The kernel validates the declaration, then executes it.
Adding a domain state is a config change, never a recompile.
- Versions form a DAG: content-addressed, parent-pointered, **never edited**.
Editing a version in place destroys every result already attributed to it.
## Steps
1. Define the IR types above. Keep them free of parser concerns — no source
spans, no format-specific fields.
2. Write the canonicalizer: sort every unordered collection by a defined key,
normalize whitespace inside templates only where semantically irrelevant,
drop optional fields that equal their default, and emit a deterministic byte
encoding.
3. Hash the canonical bytes with Blake3 into `WorkflowVersion`.
4. Add the parent pointer: a new version records its parent's hash. Nothing
mutates an existing version record.
5. Decide and document what is *not* canonicalized — prompt template text is
semantic and must hash as written. Put that decision in a comment at the
canonicalizer, since the next person will otherwise "improve" it.
6. Test: build two structurally identical IRs from different field orders and
assert equal hashes; change one prompt character and assert different hashes.
## Acceptance
- Two YAML files differing only in key order and whitespace produce identical
`WorkflowVersion`.
## Verify
**Harness:** pairs of source files that differ only in ways that must not matter,
and pairs that differ in ways that must.
**Integration test**`tests/it_canonical_version.rs`:
1. **Must match:** two YAML files with reordered top-level keys, reordered map
entries, different indentation, and trailing whitespace. Assert identical
`WorkflowVersion`.
2. **Must differ:** change one character inside a prompt template. Assert a
different version. This is the boundary — prompt text is semantic.
3. **Must differ:** change a `timeout`, a `retry` count, a `StepId`. One test per
field, table-driven, so a canonicalizer that drops a field is caught by the
field it drops.
4. Round trip: canonicalize twice; assert the byte output is stable.
5. Parent pointer: build v2 from v1; assert v2 records v1's hash and that v1's
record is unchanged on disk.
6. Property test: generate random IRs, permute the order of every unordered
collection, assert hash invariance.
**Command:** `cargo test -p workflow canonical`
**False pass:**
- Only testing step 1. A canonicalizer that discards fields it does not
understand passes every "these should match" test and silently makes two
different workflows the same version. Step 3's per-field table is the guard.
- Hashing the source bytes, which passes step 2 and fails step 1 — check both
directions.
- A property test that permutes only the collection the implementation already
sorts.
## Traps
- Hashing the source bytes. It is one line simpler and makes every reformat look
like a behaviour change.
- Canonicalizing prompt text. A whitespace change inside a prompt *is* a
behaviour change.
- Allowing a version record to be updated in place.
---
Background (not required to do this task):
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §2, §4.1, §12.2 ·
[rust-agentic-task.md](../../../rust-agentic-task.md)