110 lines
4.9 KiB
Markdown
110 lines
4.9 KiB
Markdown
# T2.7 — P2 composition gate
|
||
|
||
| Field | Value |
|
||
|---|---|
|
||
| Phase | P2 — Durability hard parts |
|
||
| Size | L — over 3 days |
|
||
| Status | Not started |
|
||
| Flags | gate |
|
||
| Spec | inlined below |
|
||
| Blocks | P3 |
|
||
|
||
## Goal
|
||
|
||
Prove intents, effect classes, forks, checkpoints and schema evolution survive
|
||
each other — not just crashes in isolation. T2.6 crashes a simple workload; this
|
||
crashes a **compound** one.
|
||
|
||
**Phase gate criterion:** `kill -9` at every transition leaves recoverable state.
|
||
|
||
## Facts (inlined — no spec read needed)
|
||
|
||
- Each P2 task was verified against a simple run. The interactions are where the
|
||
cost lives, and each is individually plausible-looking:
|
||
- a **crash during a rewind** — a fork event committed, the new branch's LSN 0
|
||
not yet written;
|
||
- a **`Dispatched` intent on a branch that was subsequently forked** — recovery
|
||
must resolve it on the branch that owns it, not the live one;
|
||
- a **checkpoint written before a schema upgrade** — the checkpoint holds v1
|
||
state, the log above it holds v2 records;
|
||
- **reduction ordering** interacting with a fork — dead-branch text is
|
||
reducible, the live branch's is not yet.
|
||
- Recovery is defined jointly: legal states (T0.2), intents resolved by class
|
||
(T2.2), byte-identical re-fold (T0.8), and exactly-once effects measured
|
||
**externally**.
|
||
- The discarded branch is evidence. A recovery path that "cleans up" a partial
|
||
fork destroys the failure that motivated the rewind.
|
||
|
||
## Steps
|
||
|
||
1. Build the compound workload: a run that retries, dispatches one intent per
|
||
effect class, rewinds at least once, and crosses a checkpoint boundary.
|
||
2. Extend T2.6's crash-point enumeration to cover the compound path, including
|
||
points **inside** a rewind and **inside** recovery itself.
|
||
3. Add the schema dimension: run half the seeds with a v1→v2 upgrade applied
|
||
between crash and restart.
|
||
4. Assert the four recovery properties jointly after every restart.
|
||
5. Run recovery **twice** per seed — recovery must itself be idempotent, since a
|
||
crash during recovery is an ordinary event.
|
||
6. Make this the required CI job gating P3.
|
||
|
||
## Acceptance
|
||
|
||
- 500 seeded crash points on the **compound** workload: zero unrecoverable
|
||
states, zero double-applied effects, byte-identical re-fold, all intents
|
||
resolved by class.
|
||
- Recovery run twice produces the same state as recovery run once.
|
||
- Half the seeds pass with a schema upgrade interposed.
|
||
|
||
## Verify
|
||
|
||
**Harness:** T2.6's child-process `kill -9` rig, the external side-effect ledger,
|
||
the three effect-class tools, and a seed printed on every run.
|
||
|
||
**Integration test** — `tests/it_p2_composition.rs`:
|
||
1. **Crash inside a rewind:** abort between the fork event commit and the new
|
||
branch's first record. On restart assert the parent branch is **byte-identical
|
||
to its pre-rewind snapshot** and the run is in a legal state — either the fork
|
||
completed or it did not, never half.
|
||
2. **Intent on a forked branch:** dispatch an `Unsafe` intent on branch 0, fork
|
||
to branch 1, crash. Assert recovery marks the **branch-0** attempt
|
||
`Indeterminate` and does not touch branch 1.
|
||
3. **Checkpoint across schema versions:** write a checkpoint at v1, append v2
|
||
records, crash, restart under the v2 binary. Assert rebuild from that
|
||
checkpoint equals a rebuild from LSN 0.
|
||
4. **Recovery idempotence:** run recovery, snapshot state, run recovery again,
|
||
assert byte equality and assert the external ledger count is unchanged.
|
||
5. **Effect exactness under compounding:** across all seeds, assert the ledger
|
||
holds exactly one entry per intended effect — including effects issued before
|
||
a fork.
|
||
6. **Crash during recovery:** abort partway through the recovery pass itself;
|
||
restart; assert convergence.
|
||
7. **Regression:** re-run T2.1–T2.6 plus P0/P1 gates in the same job.
|
||
|
||
**Command:**
|
||
`cargo test -p durability --features test-hooks --test it_p2_composition -- --nocapture`
|
||
|
||
**False pass:**
|
||
- Running the compound matrix on the simple workload. T2.6 already covers that;
|
||
this gate exists for the interactions, and every one of them is green in
|
||
isolation.
|
||
- Step 4 omitted. A recovery pass that is correct once and destructive twice
|
||
passes every test in P2 and fails the first time a machine crashes while
|
||
recovering — which is exactly when it runs.
|
||
- Step 1 asserting only that the run recovered. Recovering by truncating the
|
||
parent branch also recovers, and destroys the evidence.
|
||
- Measuring effect counts in the framework's log rather than the external ledger.
|
||
|
||
## Traps
|
||
|
||
- Treating a crash during recovery as out of scope. It is the ordinary case in a
|
||
crash loop, and a crash loop is what a nonzero `Indeterminate` rate means.
|
||
- Letting the compound workload drift out of sync with new transitions. Assert
|
||
the enumerated crash-point count against a constant, as T2.6 does.
|
||
|
||
---
|
||
|
||
Background (not required to do this task):
|
||
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §8.4, §8.5, §8.6, §8.7 ·
|
||
[rust-agentic-task.md](../../../rust-agentic-task.md)
|