poimen/tasks/T7.8-p7-composition-gate.md

T7.8 — P7 composition gate

Field	Value
Phase	P7 — Distribution
Size	L — over 3 days
Status	Not started
Flags	gate
Spec	inlined below
Blocks	P8

Goal

Prove deployment mode is interchangeable: the same behavioural suite passes in embedded mode and distributed mode, and two nodes survive a partition with no double-execution.

Phase gate criterion: two nodes surviving a partition with no double-execution.

Facts (inlined — no spec read needed)

• Two deployment modes, one set of ports.

Mode	Log + state	Blobs	Coordination
Embedded — single binary, no services	redb	redb table	in-process
Distributed — multi-node, multi-tenant	Postgres	S3-compatible	Postgres advisory locks, or Redis if leases dominate

• Embedded mode is a first-class product, not a test harness. That constraint is what keeps the ports honest — and it is only enforced if the same suite runs in both modes.
• The properties no single P7 task owns:
  • a run that survives a partition must also survive it while grading is in flight — leases, the join stage and the outbox interact;
  • the outbox relay must ship correctly across a reclaim: entries committed by node A, relayed after node B takes over;
  • partition keys and per-run ordering must hold across a node handover — the same run's events must not split partitions because a different node published them;
  • exactly-once is achieved at the fold, not in transport, so redelivery during a partition must be a no-op.
• Per-run total order, nothing promised across runs. Everything up to verification is embarrassingly parallel; grading is a join; decide is single-writer.

Steps

1. Extract the behavioural suite — P1's composition matrix plus P4's and P5's gates — into a mode-parameterized harness.
2. Run it in embedded mode and in distributed mode. Both must pass unmodified.
3. Build the compound distributed scenario: partition during grading, with the broker down, across a lease handover.
4. Assert exactly-once effects and no lost runs across every scenario.
5. Make this the required CI job gating P8.

Acceptance

• The behavioural suite passes unmodified in both deployment modes.
• Two nodes survive a partition with no double-execution and no lost run.
• The outbox ships every event in order across a lease handover, exactly once at the fold.

Verify

Harness: turmoil two-node topology; real Postgres and MinIO; the external side-effect ledger as the double-execution witness; a consumer that folds on (BranchKey, Lsn).

Integration test — tests/it_p7_composition.rs:

1. Mode interchange: parameterize the behavioural suite over [embedded, distributed]. Both pass with zero suite edits. Any if mode == ... branch inside the suite is a finding about the ports.
2. Compound scenario: start a run, begin grading, take the broker down, partition node A mid-step. Assert node B reclaims, the run and its grading complete, and the ledger shows exactly one effect per intended effect.
3. Relay across handover: assert outbox entries committed by A are shipped after B takes over — in (BranchKey, Lsn) order, exactly once at the fold.
4. Partition-key stability across nodes: assert every event of the handed-over run landed on one broker partition, despite two different nodes publishing.
5. Fencing under load: heal the partition while B is executing. Assert A's writes are rejected by fence and A does not resume.
6. No lost runs: across all scenarios, count runs spawned versus runs reaching a terminal state. Assert equality.
7. Grading yields: during the scenario, assert agent-call latency stays within bound of baseline while the join stage is saturated.
8. Regression: re-run P0–P6 gates in embedded mode in the same job.

Command: cargo test -p distribution --test it_p7_composition -- --test-threads=1 --nocapture

False pass:

• A mode-parameterized suite containing mode-specific branches. At that point the two modes are not proven equivalent, which is the entire acceptance criterion — and embedded mode is a shipped product, not a fixture.
• Asserting runs completed. Completing twice also completes; only the external ledger distinguishes them.
• Step 3 tested without a handover, where the relay never changes owner — that is T7.4's test, already green.
• Step 6 omitted: a run silently dropped during a partition produces no error and passes every other assertion here.

Traps

• Skipping embedded mode in CI for speed. It is the mode that keeps the ports honest, and it is the cheap one to run.

---

Background (not required to do this task): rust-agentic-sys.md §7, §9 · rust-agentic-task.md