Files
poimen/tasks/T8.7-p8-composition-gate.md
T

5.4 KiB
Raw Blame History

T8.7 — P8 composition gate

Field Value
Phase P8 — Operability
Size L — over 3 days
Status Not started
Flags gate
Spec inlined below
Blocks — release gate

Goal

Prove the operability layer composes with everything before it — and that turning all of P8 on does not break the zero-infrastructure embedded product.

Phase gate criterion: metering, metrics, sandbox enforcement and capacity admission all active, with the embedded smoke crate still green.

Facts (inlined — no spec read needed)

  • P8's features all sit across the rest of the system rather than beside it, so each one's own test is necessarily narrow:
    • metering attributes calls the executor and grader make;
    • metrics observe every stage;
    • keyed capability constrains every tool call;
    • reduction rewrites what grading and verification later read;
    • admission control gates work the scheduler wants to run.
  • The properties no single P8 task owns:
    • a reduced episode is still gradeable — reduction and grading are built by different tasks and read the same blobs;
    • budget and capacity refusals are distinguishable end to end, since they look identical in a queue-depth graph and have opposite fixes;
    • metering totals reconcile with the model provider's own call count — attribution that misses a call path understates spend silently;
    • the embedded smoke crate still passes with every P8 feature enabled. If it does not, an operability feature has leaked a service dependency into the default path.
  • The retention ceiling is derived from hardware, not chosen: where max_context_tokens / 2 is tighter than the configured ceiling, the bound wins. That interaction spans T5.2 and T8.4 and belongs to neither.

Steps

  1. Enable every P8 feature and run the full behavioural suite from T7.8.
  2. Run the reduce-then-grade path: reduce a run, then grade the reduced episode.
  3. Reconcile metering totals against the provider's own counter.
  4. Run the embedded smoke crate with all features on.
  5. Drive both refusal reasons and assert they are separable everywhere.
  6. Make this the release gate.

Acceptance

  • The full behavioural suite passes with all P8 features enabled, in both deployment modes.
  • A reduced episode is still gradeable and still produces a valid Score.
  • Metering totals reconcile exactly with the provider's call count.
  • The embedded smoke crate passes with every feature on.
  • Capacity and budget refusals are distinguishable end to end.

Verify

Harness: T7.8's mode-parameterized suite; the provider call counter; the external side-effect ledger; the out-of-workspace smoke crate; a scrapeable metrics recorder.

Integration testtests/it_p8_composition.rs:

  1. Everything on: run the T7.8 behavioural suite with metering, metrics, keyed capability, reduction and admission control all enabled, in both modes.
  2. Reduce then grade: reduce a run past the ceiling, then grade it. Assert a valid Score, and assert the grading path handled getNone on reduced originals by following the Reduced event. Then drop state, re-fold, and assert byte-identical.
  3. Derived ceiling interaction: configure a 200k retention ceiling with a hardware-derived bound of 80k. Assert reduction targets 80k and that the judge's context request is admitted.
  4. Metering reconciliation: assert sum(metered calls) equals the provider's own counter, and that every call carries a non-default Purpose. A mismatch means a call path is unattributed.
  5. Refusal separability: drive one capacity refusal and one budget refusal. Assert two distinct metric series moved, that each error names its limit, and that the budget-refused run reaches terminal Ungraded while the capacity-refused work was never queued.
  6. Keyed capability under the real interpreter: assert every tool call in the suite carried a kernel-supplied key or was denied and audited — count against the intent records, not against the tool's own reporting.
  7. Zero swaps across the whole gate.
  8. Smoke crate: run it, outside the workspace, default features only, with every P8 feature active. Assert no network listener and no distributed-only crate in cargo tree.
  9. Regression: re-run P0P7 gates.

Command: cargo test -p operability --test it_p8_composition && cargo test --manifest-path smoke-crate/Cargo.toml

False pass:

  • Running each P8 feature's suite with the others off. Every one is green that way; the interactions — reduction versus grading, metering versus purpose attribution, admission versus the join stage — are the reason this gate exists.
  • Step 2 asserting the reduced run produced a score. Assert the re-fold too: a mapping stored outside the log passes the grading half and destroys drop-and-re-fold.
  • Step 4 comparing metered totals to themselves. The provider's counter is the independent witness.
  • Step 8 run with a non-default feature enabled to make it build. That is the leak, papered over.

Traps

  • Treating this as a formality once P0P7 are green. Every property here spans tasks that were each verified in isolation, and P8's features touch all of them.

Background (not required to do this task): rust-agentic-sys.md §8.6, §13, §14, §15 · rust-agentic-task.md