104 lines
4.5 KiB
Markdown
104 lines
4.5 KiB
Markdown
|
# T1.8 — P1 composition gate
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
|---|---|
|
|||
|
|
| Phase | P1 — Walking skeleton |
|
|||
|
|
| Size | M — 1 to 3 days |
|
|||
|
|
| Status | Not started |
|
|||
|
|
| Flags | gate |
|
|||
|
|
| Spec | inlined below |
|
|||
|
|
| Blocks | P2 |
|
|||
|
|
|
|||
|
|
## Goal
|
|||
|
|
|
|||
|
|
Prove T1.1–T1.7 compose into one recorded run that re-derives byte-identically,
|
|||
|
|
and that the pluggable parts of the skeleton are genuinely pluggable.
|
|||
|
|
|
|||
|
|
**Phase gate criterion:** one full run is recorded and re-derives
|
|||
|
|
byte-identically.
|
|||
|
|
|
|||
|
|
## Facts (inlined — no spec read needed)
|
|||
|
|
|
|||
|
|
- The walking skeleton is **one full run of one workflow against a stub model, in
|
|||
|
|
embedded mode, with no network — recorded, verified, and re-derivable from the
|
|||
|
|
log.** Nothing after it is worth starting until it runs.
|
|||
|
|
- Nothing is built before the record is trustworthy: no dashboards and no
|
|||
|
|
learning loop until this passes.
|
|||
|
|
- The composition properties no single P1 task owns:
|
|||
|
|
- cancellation from **any** state leaves a legal attempt state and a complete
|
|||
|
|
log (T1.1 + T1.3 + T0.2);
|
|||
|
|
- a retry, a cancel and a normal completion all produce episodes that survive
|
|||
|
|
drop-and-re-fold (T1.4 + T1.7 + T0.8);
|
|||
|
|
- context partitions and blob refs are present on **every** model step of
|
|||
|
|
**every** attempt, including failed ones (T1.5 + T1.6 + T1.4).
|
|||
|
|
- The serial spine is `(TenantId, RunId)`. Parallelism lives between runs. A
|
|||
|
|
composition test with one run cannot see a violation of this.
|
|||
|
|
|
|||
|
|
## Steps
|
|||
|
|
|
|||
|
|
1. Build the run matrix: `{clean, one retry, two retries, cancelled mid-step,
|
|||
|
|
cancelled during Verifying}` × `{served from state, served from cold re-fold}`.
|
|||
|
|
2. Assert every cell produces a consistent episode and a legal state sequence.
|
|||
|
|
3. Swap the stub model script and the verifier implementation without touching
|
|||
|
|
executor code — the skeleton must not know which is installed.
|
|||
|
|
4. Run 50 runs concurrently across 2 tenants; assert per-run ordering and
|
|||
|
|
cross-run independence.
|
|||
|
|
5. Make this the required CI job gating P2.
|
|||
|
|
|
|||
|
|
## Acceptance
|
|||
|
|
|
|||
|
|
- Every matrix cell is green.
|
|||
|
|
- A full run is recorded and re-derives **byte-identically**.
|
|||
|
|
- Swapping the stub script or the verifier requires no executor change.
|
|||
|
|
|
|||
|
|
## Verify
|
|||
|
|
|
|||
|
|
**Harness:** the P0 in-memory ports (T0.9) for speed, plus the real `redb` path
|
|||
|
|
for at least one cell — a composition that only works in memory is not a
|
|||
|
|
composition.
|
|||
|
|
|
|||
|
|
**Integration test** — `tests/it_p1_composition.rs`:
|
|||
|
|
1. **Matrix:** for each of the 10 cells, assert the episode from materialized
|
|||
|
|
state and from cold re-fold are **byte-identical**, and that the run's state
|
|||
|
|
sequence is legal under T0.2's table.
|
|||
|
|
2. **Cancel sweep:** cancel from `Scheduled`, `Running`, `Verifying`, `Grading`.
|
|||
|
|
For each, assert the attempt lands in `Cancelled` (no dispatched intent),
|
|||
|
|
every spawned task terminated before `RunScope` returned, and the log is
|
|||
|
|
complete up to the cancel.
|
|||
|
|
3. **Capture completeness:** across all cells, assert **every** model-step
|
|||
|
|
attempt — including failed ones — carries a context partition and prompt and
|
|||
|
|
output refs. Iterate, do not sample.
|
|||
|
|
4. **Immutability under composition:** in the two-retry cell, snapshot attempts 1
|
|||
|
|
and 2, run to completion, assert byte equality. Then drop state, re-fold, and
|
|||
|
|
assert the attempts are still identical.
|
|||
|
|
5. **Isolation:** 50 concurrent runs across 2 tenants. Assert each run's records
|
|||
|
|
are totally ordered, no two steps of one run overlap in time, and no record
|
|||
|
|
carries the wrong tenant.
|
|||
|
|
6. **Pluggability:** run the same matrix with a second stub script and a second
|
|||
|
|
verifier. Assert zero changes to executor code — enforce by keeping the
|
|||
|
|
executor crate's test-only surface empty.
|
|||
|
|
7. **Regression:** re-run every P1 and P0 task suite in the same job.
|
|||
|
|
|
|||
|
|
**Command:** `cargo test -p skeleton --test it_p1_composition`
|
|||
|
|
|
|||
|
|
**False pass:**
|
|||
|
|
- Testing only the clean cell. Retry and cancel are where the fold, the scope and
|
|||
|
|
the capture paths interact, and each is individually green already.
|
|||
|
|
- Step 3 sampling one attempt. A capture path that skips failed attempts passes a
|
|||
|
|
sample and discards the entire learning signal.
|
|||
|
|
- Step 5 with one run, where serial execution and accidental parallelism are
|
|||
|
|
indistinguishable.
|
|||
|
|
- Step 1 comparing episodes with `PartialEq` rather than serialized bytes.
|
|||
|
|
|
|||
|
|
## Traps
|
|||
|
|
|
|||
|
|
- Declaring the gate green with `Verifying`/`Grading` collapsed into function
|
|||
|
|
calls. The states must appear in the log, or T4.2's barrier has nowhere to live.
|
|||
|
|
- Building any dashboard or grading work before this is green.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
Background (not required to do this task):
|
|||
|
|
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §5.2, §5.3, §16, §20 ·
|
|||
|
|
[rust-agentic-task.md](../../../rust-agentic-task.md)
|