79 lines
2.6 KiB
Markdown
79 lines
2.6 KiB
Markdown
|
# T10.3 — Idempotent run admission
|
||
|
|
|
||
|
|
| Field | Value |
|
||
|
|
|---|---|
|
||
|
|
| Phase | P10 — Orchestration |
|
||
|
|
| Size | S — under 1 day |
|
||
|
|
| Status | Not started |
|
||
|
|
| Flags | — |
|
||
|
|
| Spec | inlined below |
|
||
|
|
| Blocks | T10.4, T10.7 |
|
||
|
|
|
||
|
|
## Goal
|
||
|
|
|
||
|
|
`dispatch_key` on spawn requests: a redelivered spawn request (from Kafka
|
||
|
|
at-least-once delivery, T10.7) becomes a no-op insert instead of a duplicate
|
||
|
|
run.
|
||
|
|
|
||
|
|
## Facts (inlined — no spec read needed)
|
||
|
|
|
||
|
|
- Same principle as the outbox's `(BranchKey, Lsn)` fold-key idempotency
|
||
|
|
(T7.4) — exactly-once is achieved at the admission boundary, not in
|
||
|
|
transport.
|
||
|
|
- `dispatch_key` is caller-supplied, opaque, unique per intended spawn — e.g.
|
||
|
|
a `Reconciler` decision id.
|
||
|
|
- Admission with no `dispatch_key` behaves exactly as today. This is
|
||
|
|
additive, not a required field.
|
||
|
|
|
||
|
|
## Steps
|
||
|
|
|
||
|
|
1. Add optional `dispatch_key: Option<String>` to spawn-run input.
|
||
|
|
2. Add a unique index on `(TenantId, dispatch_key)` where `dispatch_key` is
|
||
|
|
set.
|
||
|
|
3. On spawn: if `dispatch_key` is set and already seen, return the existing
|
||
|
|
`RunId` instead of creating a new run — no error, no duplicate.
|
||
|
|
4. Test redelivery: same `dispatch_key` submitted twice, concurrently and
|
||
|
|
sequentially.
|
||
|
|
|
||
|
|
## Acceptance
|
||
|
|
|
||
|
|
- Two spawn calls with the same `dispatch_key` produce exactly one run; the
|
||
|
|
second call returns the first run's `RunId`.
|
||
|
|
- Spawn calls with no `dispatch_key` are unaffected.
|
||
|
|
|
||
|
|
## Verify
|
||
|
|
|
||
|
|
**Harness:** spawn endpoint (T10.4) or direct kernel call, two concurrent
|
||
|
|
callers.
|
||
|
|
|
||
|
|
**Integration test** — `tests/it_idempotent_admission.rs`:
|
||
|
|
1. Spawn with `dispatch_key=K`; assert one run created, `RunId=R`.
|
||
|
|
2. Spawn again with `dispatch_key=K`; assert no new run created, returned
|
||
|
|
`RunId == R`.
|
||
|
|
3. Fire 20 concurrent spawn calls with the same `dispatch_key`; assert
|
||
|
|
exactly one run exists after all resolve.
|
||
|
|
4. Spawn with no `dispatch_key` twice; assert two distinct runs (unaffected
|
||
|
|
baseline).
|
||
|
|
|
||
|
|
**Command:** `cargo test -p kernel idempotent_admission -- --test-threads=1`
|
||
|
|
|
||
|
|
**False pass:**
|
||
|
|
- Testing only sequential redelivery. Concurrent redelivery is the actual
|
||
|
|
failure mode under Kafka rebalance/retry and needs the unique-index race
|
||
|
|
caught, not a check-then-insert race.
|
||
|
|
|
||
|
|
## Traps
|
||
|
|
|
||
|
|
- Making `dispatch_key` required. Breaks every direct/manual spawn call that
|
||
|
|
isn't part of a goal chain.
|
||
|
|
- Deduping by a hash of `(workflow_ref, input)` instead of the caller-supplied
|
||
|
|
key. Two legitimately identical requests would then silently collapse into
|
||
|
|
one run.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
Background (not required to do this task):
|
||
|
|
[rust-agentic-sys.md](../rust-agentic-sys.md) §8.3, §9.3 ·
|
||
|
|
[T7.4-outbox-relay.md](T7.4-outbox-relay.md) ·
|
||
|
|
[T3.6-version-pinning-at-spawn.md](T3.6-version-pinning-at-spawn.md)
|