5.4 KiB
5.4 KiB
T8.7 — P8 composition gate
| Field | Value |
|---|---|
| Phase | P8 — Operability |
| Size | L — over 3 days |
| Status | Not started |
| Flags | gate |
| Spec | inlined below |
| Blocks | — release gate |
Goal
Prove the operability layer composes with everything before it — and that turning all of P8 on does not break the zero-infrastructure embedded product.
Phase gate criterion: metering, metrics, sandbox enforcement and capacity admission all active, with the embedded smoke crate still green.
Facts (inlined — no spec read needed)
- P8's features all sit across the rest of the system rather than beside it,
so each one's own test is necessarily narrow:
- metering attributes calls the executor and grader make;
- metrics observe every stage;
- keyed capability constrains every tool call;
- reduction rewrites what grading and verification later read;
- admission control gates work the scheduler wants to run.
- The properties no single P8 task owns:
- a reduced episode is still gradeable — reduction and grading are built by different tasks and read the same blobs;
- budget and capacity refusals are distinguishable end to end, since they look identical in a queue-depth graph and have opposite fixes;
- metering totals reconcile with the model provider's own call count — attribution that misses a call path understates spend silently;
- the embedded smoke crate still passes with every P8 feature enabled. If it does not, an operability feature has leaked a service dependency into the default path.
- The retention ceiling is derived from hardware, not chosen: where
max_context_tokens / 2is tighter than the configured ceiling, the bound wins. That interaction spans T5.2 and T8.4 and belongs to neither.
Steps
- Enable every P8 feature and run the full behavioural suite from T7.8.
- Run the reduce-then-grade path: reduce a run, then grade the reduced episode.
- Reconcile metering totals against the provider's own counter.
- Run the embedded smoke crate with all features on.
- Drive both refusal reasons and assert they are separable everywhere.
- Make this the release gate.
Acceptance
- The full behavioural suite passes with all P8 features enabled, in both deployment modes.
- A reduced episode is still gradeable and still produces a valid
Score. - Metering totals reconcile exactly with the provider's call count.
- The embedded smoke crate passes with every feature on.
- Capacity and budget refusals are distinguishable end to end.
Verify
Harness: T7.8's mode-parameterized suite; the provider call counter; the external side-effect ledger; the out-of-workspace smoke crate; a scrapeable metrics recorder.
Integration test — tests/it_p8_composition.rs:
- Everything on: run the T7.8 behavioural suite with metering, metrics, keyed capability, reduction and admission control all enabled, in both modes.
- Reduce then grade: reduce a run past the ceiling, then grade it. Assert a
valid
Score, and assert the grading path handledget→Noneon reduced originals by following theReducedevent. Then drop state, re-fold, and assert byte-identical. - Derived ceiling interaction: configure a 200k retention ceiling with a hardware-derived bound of 80k. Assert reduction targets 80k and that the judge's context request is admitted.
- Metering reconciliation: assert
sum(metered calls)equals the provider's own counter, and that every call carries a non-defaultPurpose. A mismatch means a call path is unattributed. - Refusal separability: drive one capacity refusal and one budget refusal.
Assert two distinct metric series moved, that each error names its limit, and
that the budget-refused run reaches terminal
Ungradedwhile the capacity-refused work was never queued. - Keyed capability under the real interpreter: assert every tool call in the suite carried a kernel-supplied key or was denied and audited — count against the intent records, not against the tool's own reporting.
- Zero swaps across the whole gate.
- Smoke crate: run it, outside the workspace, default features only, with
every P8 feature active. Assert no network listener and no distributed-only
crate in
cargo tree. - Regression: re-run P0–P7 gates.
Command: cargo test -p operability --test it_p8_composition && cargo test --manifest-path smoke-crate/Cargo.toml
False pass:
- Running each P8 feature's suite with the others off. Every one is green that way; the interactions — reduction versus grading, metering versus purpose attribution, admission versus the join stage — are the reason this gate exists.
- Step 2 asserting the reduced run produced a score. Assert the re-fold too: a mapping stored outside the log passes the grading half and destroys drop-and-re-fold.
- Step 4 comparing metered totals to themselves. The provider's counter is the independent witness.
- Step 8 run with a non-default feature enabled to make it build. That is the leak, papered over.
Traps
- Treating this as a formality once P0–P7 are green. Every property here spans tasks that were each verified in isolation, and P8's features touch all of them.
Background (not required to do this task): rust-agentic-sys.md §8.6, §13, §14, §15 · rust-agentic-task.md