4.5 KiB
4.5 KiB
T6.1 — Challenger allocation
| Field | Value |
|---|---|
| Phase | P6 — Learning loop |
| Size | M — 1 to 3 days |
| Status | Not started |
| Flags | — |
| Spec | inlined below |
| Blocks | T6.3 |
Goal
One current version, at most one challenger, 95/5 traffic split, guarded by a generation counter with compare-and-swap.
Facts (inlined — no spec read needed)
- One current version, at most one challenger. The loop converges toward a single state rather than maintaining a population. That is the change that makes everything else affordable: no N-way traffic split, no per-variant aggregation across groups, no allocation state to contend over, one comparison per episode instead of a bracket.
- The cost is exploration. A single challenger at a time is hill-climbing — it finds improvements more slowly than a population would, and it can sit in a local optimum indefinitely with nothing in the loop able to say so. The held-out report (T6.6) is the only instrument that will notice, which makes it more important here, not less.
- The decide stage is single-writer per
(TenantId, WorkflowId). Two schedulers adjusting traffic allocation concurrently produce an allocation neither holds. A compare-and-swap on a generation counter is sufficient; no lock service is needed at this size. - The slow generate loop fires when the fast loop rejects a challenger without finding a replacement — a trigger, not a timer.
- Multi-variant selection stays available for deployments that enable the tournament and allow N challengers; the machinery is the same.
Steps
- Define the allocation record:
(TenantId, WorkflowId) → { current: WorkflowVersion, challenger: Option<WorkflowVersion>, split, generation: u64 }. - Registration of a challenger is rejected when one is already live — the error names the incumbent challenger, so the operator knows what to retire.
- Every update is a CAS on
generation. The loser retries against the winner's generation rather than overwriting. - Route spawns by the split: 95% current, 5% challenger. Record which version a run was allocated to on the spawn event.
- Emit the "rejected without replacement" trigger as an event so the generate loop can subscribe. Do not add a timer.
- Test two concurrent schedulers committing allocations.
Acceptance
- Registering a second challenger while one is live is rejected, naming the incumbent challenger.
- Two concurrent schedulers cannot both commit an allocation; the loser retries and observes the winner's generation.
Verify
Harness: two scheduler instances against one backend — real concurrency, not a simulated race.
Integration test — tests/it_challenger_allocation.rs:
- Register challenger A. Register challenger B while A is live; assert rejected, and that the error names A.
- Concurrent CAS: two schedulers commit different allocations simultaneously, in a loop of 100 rounds. Assert exactly one wins per round, the loser retries, and the final generation equals the number of successful commits — no lost updates.
- Traffic split: spawn 1000 runs; assert the challenger share is within tolerance of 5% and that each run's spawn event records which version it was allocated to.
- Assert control appears in every allocation, so T6.4's anchor exists.
- Trigger: reject a challenger with no replacement; assert the generate-loop trigger event fires once. Assert no timer path exists that would fire it otherwise — run 60s of simulated time with no rejection and assert zero triggers.
- Restart mid-decision; assert the allocation and generation survive.
Command: cargo test -p loop allocation -- --test-threads=1
False pass:
- Step 2 simulated by calling the update function twice sequentially. Last-write- wins passes that and loses an allocation under real concurrency. Use two live schedulers.
- Step 3 asserting the split ratio without asserting the per-run recorded version. Runs allocated correctly but recorded wrongly are unattributable, and the ratio still looks right.
- Step 5's negative half omitted, so a timer-driven generate loop passes.
Traps
- Last-write-wins on the allocation record. The result is a split neither scheduler intended and no error anywhere.
- A timer-driven generate loop. It proposes challengers when nothing has been learned.
Background (not required to do this task): rust-agentic-sys.md §9.1, §12.1 · rust-agentic-task.md