3.1 KiB
3.1 KiB
T1.2 — Stub model provider
| Field | Value |
|---|---|
| Phase | P1 — Walking skeleton |
| Size | S — under 1 day |
| Status | Not started |
| Flags | — |
| Spec | inlined below |
| Blocks | — |
Goal
A deterministic scripted model provider so the walking skeleton runs end to end with no network and no spend.
Facts (inlined — no spec read needed)
- Nothing is built before the record is trustworthy: no dashboards and no learning loop until one full run works end to end against a stub model.
- The stub is not a test fixture that gets deleted — it is the fixed point every later determinism assertion is measured against, including T0.8's re-fold property and T5.x's grading tests.
- Failure injection here is what T1.4 (retry), T2.1 (crash windows) and T2.6 (crash matrix) drive their scenarios with.
Steps
- Define the model port the executor calls: request in, response plus
Usageout,async, cancellation-aware. - Implement the stub over a script: an ordered list of responses, optionally keyed by step id so a workflow with branches stays readable.
- Add configurable latency per response, driven by
tokio::time::sleepsotokio::time::pausein tests makes it instant. - Add failure injection: error responses, timeouts, and a hang that outlives the deadline. Each selectable per script entry.
- Make every field of the emitted response deterministic — no timestamps from the wall clock, no generated ids that are not seeded.
- Test: run the same script twice, serialize both episodes, assert equal bytes.
Acceptance
- The same script yields byte-identical episodes across runs.
Verify
Harness: the stub itself plus T0.8's assert_refold_identical.
Integration test — tests/it_stub_determinism.rs:
- Run the same script twice, in two fresh databases, in the same process.
- Serialize both episodes; assert byte equality.
- Run a third time in a separate process and compare against a committed expected serialization — catches anything seeded from process state.
- With
tokio::time::pause, assert a script declaring 30s latency completes in near-zero wall time. If it does not, the stub is sleeping off the tokio clock. - Failure injection: assert an error entry produces a failed attempt, a timeout
entry produces
TimedOut, and a hang entry is cut off by the deadline.
Command: cargo test -p testkit stub
False pass:
- Steps 1–2 in one process with a lazily-initialized global seed: both runs share it and agree, while a fresh process differs. Step 3 is the guard.
- Comparing episodes with a
PartialEqthat skips timestamps. That hides exactly the nondeterminism being hunted — compare the full serialized bytes.
Traps
- A wall-clock timestamp or a random id in the stub response. It defeats every byte-identity assertion downstream and looks like a fold bug.
- Sleeping with
std::thread::sleep, whichtokio::time::pausecannot skip.
Background (not required to do this task): rust-agentic-sys.md §16 · rust-agentic-task.md