104 lines
4.5 KiB
Markdown
104 lines
4.5 KiB
Markdown
# T1.8 — P1 composition gate
|
||
|
||
| Field | Value |
|
||
|---|---|
|
||
| Phase | P1 — Walking skeleton |
|
||
| Size | M — 1 to 3 days |
|
||
| Status | Not started |
|
||
| Flags | gate |
|
||
| Spec | inlined below |
|
||
| Blocks | P2 |
|
||
|
||
## Goal
|
||
|
||
Prove T1.1–T1.7 compose into one recorded run that re-derives byte-identically,
|
||
and that the pluggable parts of the skeleton are genuinely pluggable.
|
||
|
||
**Phase gate criterion:** one full run is recorded and re-derives
|
||
byte-identically.
|
||
|
||
## Facts (inlined — no spec read needed)
|
||
|
||
- The walking skeleton is **one full run of one workflow against a stub model, in
|
||
embedded mode, with no network — recorded, verified, and re-derivable from the
|
||
log.** Nothing after it is worth starting until it runs.
|
||
- Nothing is built before the record is trustworthy: no dashboards and no
|
||
learning loop until this passes.
|
||
- The composition properties no single P1 task owns:
|
||
- cancellation from **any** state leaves a legal attempt state and a complete
|
||
log (T1.1 + T1.3 + T0.2);
|
||
- a retry, a cancel and a normal completion all produce episodes that survive
|
||
drop-and-re-fold (T1.4 + T1.7 + T0.8);
|
||
- context partitions and blob refs are present on **every** model step of
|
||
**every** attempt, including failed ones (T1.5 + T1.6 + T1.4).
|
||
- The serial spine is `(TenantId, RunId)`. Parallelism lives between runs. A
|
||
composition test with one run cannot see a violation of this.
|
||
|
||
## Steps
|
||
|
||
1. Build the run matrix: `{clean, one retry, two retries, cancelled mid-step,
|
||
cancelled during Verifying}` × `{served from state, served from cold re-fold}`.
|
||
2. Assert every cell produces a consistent episode and a legal state sequence.
|
||
3. Swap the stub model script and the verifier implementation without touching
|
||
executor code — the skeleton must not know which is installed.
|
||
4. Run 50 runs concurrently across 2 tenants; assert per-run ordering and
|
||
cross-run independence.
|
||
5. Make this the required CI job gating P2.
|
||
|
||
## Acceptance
|
||
|
||
- Every matrix cell is green.
|
||
- A full run is recorded and re-derives **byte-identically**.
|
||
- Swapping the stub script or the verifier requires no executor change.
|
||
|
||
## Verify
|
||
|
||
**Harness:** the P0 in-memory ports (T0.9) for speed, plus the real `redb` path
|
||
for at least one cell — a composition that only works in memory is not a
|
||
composition.
|
||
|
||
**Integration test** — `tests/it_p1_composition.rs`:
|
||
1. **Matrix:** for each of the 10 cells, assert the episode from materialized
|
||
state and from cold re-fold are **byte-identical**, and that the run's state
|
||
sequence is legal under T0.2's table.
|
||
2. **Cancel sweep:** cancel from `Scheduled`, `Running`, `Verifying`, `Grading`.
|
||
For each, assert the attempt lands in `Cancelled` (no dispatched intent),
|
||
every spawned task terminated before `RunScope` returned, and the log is
|
||
complete up to the cancel.
|
||
3. **Capture completeness:** across all cells, assert **every** model-step
|
||
attempt — including failed ones — carries a context partition and prompt and
|
||
output refs. Iterate, do not sample.
|
||
4. **Immutability under composition:** in the two-retry cell, snapshot attempts 1
|
||
and 2, run to completion, assert byte equality. Then drop state, re-fold, and
|
||
assert the attempts are still identical.
|
||
5. **Isolation:** 50 concurrent runs across 2 tenants. Assert each run's records
|
||
are totally ordered, no two steps of one run overlap in time, and no record
|
||
carries the wrong tenant.
|
||
6. **Pluggability:** run the same matrix with a second stub script and a second
|
||
verifier. Assert zero changes to executor code — enforce by keeping the
|
||
executor crate's test-only surface empty.
|
||
7. **Regression:** re-run every P1 and P0 task suite in the same job.
|
||
|
||
**Command:** `cargo test -p skeleton --test it_p1_composition`
|
||
|
||
**False pass:**
|
||
- Testing only the clean cell. Retry and cancel are where the fold, the scope and
|
||
the capture paths interact, and each is individually green already.
|
||
- Step 3 sampling one attempt. A capture path that skips failed attempts passes a
|
||
sample and discards the entire learning signal.
|
||
- Step 5 with one run, where serial execution and accidental parallelism are
|
||
indistinguishable.
|
||
- Step 1 comparing episodes with `PartialEq` rather than serialized bytes.
|
||
|
||
## Traps
|
||
|
||
- Declaring the gate green with `Verifying`/`Grading` collapsed into function
|
||
calls. The states must appear in the log, or T4.2's barrier has nowhere to live.
|
||
- Building any dashboard or grading work before this is green.
|
||
|
||
---
|
||
|
||
Background (not required to do this task):
|
||
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §5.2, §5.3, §16, §20 ·
|
||
[rust-agentic-task.md](../../../rust-agentic-task.md)
|