# T7.4 — Outbox relay

| Field | Value |
|---|---|
| Phase | P7 — Distribution |
| Size | M — 1 to 3 days |
| Status | Not started |
| Flags | — |
| Spec | inlined below |
| Blocks | — |

## Goal

Ship committed export intents out of process, with its own retry and its own
failure domain. The execution path never calls a broker.

## Facts (inlined — no spec read needed)

- **The framework never calls a broker from the execution path.** Export intent
  is written in the same transaction as the state (T0.6); a separate relay reads
  committed intents and ships them.
- This makes export restartable, keeps a broker outage from stalling a run, and
  is **the only pattern that survives a crash between "state committed" and
  "event published"**.
- **A user's broker being down is not an agent outage.**
- The relay reads through `drain_outbox` / `ack_outbox`. Ordering is per
  `BranchKey`, ascending `Lsn` — that is the only ordering the system promises,
  and it falls out of the outbox key shape.
- **Delivery is at-least-once. Exactly-once is achieved at the fold, not in
  transport**: `(BranchKey, Lsn)` is the natural idempotency key, so a redelivered
  record is a no-op insert.
- Only the live branch is exported (T2.3).

## Steps

1. Run the relay as its own process or task with its own supervision — separate
   failure domain is the point, so do not co-locate it with the executor's
   lifecycle.
2. Loop: `drain_outbox(tenant, limit)` → publish in `(BranchKey, Lsn)` order →
   `ack_outbox(shipped)`. Ack only what the broker accepted.
3. Retry with backoff inside the relay. Never propagate a broker error into run
   execution — there is no path back by construction, so verify none is added.
4. Set the broker partition key from `(TenantId, RunId)` (T7.5).
5. Make the consumer-side fold idempotent on `(BranchKey, Lsn)` so redelivery is
   a no-op insert.
6. Emit relay lag as a metric — outbox backlog is one of the two that grow
   silently.
7. Test with the broker down for an entire run.

## Acceptance

- Broker down for the whole run; **the run completes**; every event ships after
  the broker returns, **in order, exactly once at the fold**.

## Verify

**Harness:** a broker that can be held down for the whole test, plus a consumer
that folds on `(BranchKey, Lsn)` so redelivery is observable.

**Integration test** — `tests/it_outbox_relay.rs`:
1. Take the broker **down**. Run a full run to completion.
2. Assert the run **completed normally** — no stall, no error, and its latency is
   comparable to a broker-up baseline. A user's broker being down is not an agent
   outage.
3. Bring the broker up. Assert every event ships.
4. Assert ordering **per `BranchKey`, ascending `Lsn`** at the consumer.
5. Force redelivery (ack loss): assert the consumer's folded state is identical
   to the single-delivery case — exactly-once **at the fold**, not in transport.
6. Kill the relay mid-batch; restart; assert no event is lost and none is
   duplicated at the fold.
7. Assert the execution path never calls the broker: instrument the broker client
   and assert zero calls originate from executor threads.
8. Assert only the **live branch** is exported after a rewind.
9. Assert relay lag is emitted as a metric.

**Command:** `cargo test -p distribution outbox -- --test-threads=1`

**False pass:**
- Step 2 asserting only that the run completed. If the executor retries the
  broker inline, the run completes — slowly. The latency comparison is what
  catches it; step 7 is what proves it.
- Acking before the broker confirms: steps 3 and 4 still pass in a happy-path
  test, and events vanish silently under failure. Step 6 is the guard.
- Testing ordering within one branch only, where any implementation is ordered.
  Use several branches and several runs interleaved.

## Traps

- Publishing from the execution path "just for low latency". A broker outage then
  stalls agent work.
- Acking before the broker confirms. Events are lost with no trace.
- Attempting transport-level exactly-once. The fold is where it is achievable.

---

Background (not required to do this task):
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §8.3, §9.2, §9.3, §13.3, §15 ·
[rust-agentic-task.md](../../../rust-agentic-task.md)