poimen/tasks/T8.6-capacity-admission-control.md

# T8.6 — Capacity admission control

| Field | Value |
|---|---|
| Phase | P8 — Operability |
| Size | M — 1 to 3 days |
| Status | Not started |
| Flags | — |
| Spec | inlined below |
| Blocks | — |

## Goal

Enforce the residency invariant at runtime, not only at load. Grading yields to
agent inference.

## Facts (inlined — no spec read needed)

- **Admission control, not backpressure.** Work whose `ResourceProfile` does not
  fit the current residency is **refused at admission with the limit named**.
  Queuing it would stall behind an eviction that the first principle does not
  permit.
- **Grading yields to agent work.** When both contend for the same resident
  model, agent inference wins and grading queues. **A framework that lets a judge
  call delay the work it is judging has inverted its own first principle.**
- **One resident model is the default configuration.** The agent's model is
  pinned and never evictable; the judge and the proposer run on that same model.
- A second model is legal only if the invariant holds with **both resident** —
  never by swapping between them per call, which is the failure mode this exists
  to prevent.
- **A swap is a test failure, not a slow path.** A nonzero swap rate on a
  single-model deployment means something is requesting a non-resident model, and
  the load cost — tens of seconds — will dominate everything else in the trace.
- Admission refusals are metered by reason, separating **capacity** from
  **budget**: they look identical in a queue-depth graph and have opposite fixes.
- Residency is per device. A model resident on node A does not make node B's runs
  admissible.

## Steps

1. Track current residency as live state: which models are resident on which
   devices, and current concurrent KV usage.
2. At admission, evaluate T5.2's invariant against the requesting work's
   `ResourceProfile` plus current residency.
3. Refuse when it does not fit, with the **named limit** in the error — not a
   generic capacity error, and not a queue.
4. Implement priority: agent inference preempts queued grading for the same
   resident model. Grading waits; agent work does not.
5. Assert on **agent-call latency under grading load**, not on queue ordering —
   a scheduler that merely queues correctly still fails the principle if grading
   wins the race.
6. Count model swaps. Emit as a metric and assert zero in the test.
7. Tag admission refusals `capacity` or `budget` at the refusal site (T8.1 owns
   the budget half).

## Acceptance

- With one resident model, a burst of grading work **never delays agent inference
  past a configured bound** — asserted on agent-call latency under grading load.
- **Zero model swaps** occur across the whole run; a swap is a test failure, not
  a slow path.

## Verify

**Harness:** a model runtime stub that reports residency and **counts swaps**,
plus a latency histogram on agent calls. No GPU needed — the stub enforces the
declared limits.

**Integration test** — `tests/it_admission_control.rs`:
1. Baseline: measure agent-call latency (p50 and p99) with **no** grading load.
2. Burst 100 grading tasks against one resident model while agent work continues.
3. **Assert agent-call p99 stays within the configured bound of baseline.** This
   is the acceptance criterion and it is a latency assertion, not an ordering
   one — a scheduler that merely queues correctly still fails the principle if
   grading wins the race.
4. **Assert zero model swaps** across the whole run. A swap is a test failure,
   not a slow path — assert `swap_count == 0`, not a threshold.
5. Submit work whose `ResourceProfile` does not fit current residency. Assert it
   is **refused at admission**, not queued — check the queue depth stayed at
   zero — and that the error **names the limit**.
6. Assert the refusal is tagged `capacity`, distinct from T8.1's `budget`
   refusals, and that both series exist.
7. Multi-node: assert a model resident on node A does not make node B's work
   admissible.
8. Assert a pinned model is never selected for eviction, even when the invariant
   would otherwise be satisfiable by evicting it.

**Command:** `cargo test -p operability admission -- --test-threads=1 --nocapture`

**False pass:**
- Step 3 asserting that grading was queued behind agent work. Correct ordering
  with a shared lock still adds latency to agent calls; the histogram is what
  detects it.
- Step 5 asserting an eventual error after queuing. Queuing over-capacity work
  stalls it behind an eviction that must not happen — assert the queue never grew.
- Step 4 as `swap_count < 5`. Any nonzero swap rate means something is requesting
  a non-resident model, and the tens-of-seconds load cost will dominate the
  trace.
- Measuring only mean latency, where a p99 stall from one swap disappears.

## Traps

- Queuing over-capacity work instead of refusing it. It stalls behind an eviction
  that must not happen.
- A fair scheduler between grading and agent work. Fair is the wrong policy here;
  agent work wins.
- Measuring only queue depth. Both refusal reasons look identical there.

---

Background (not required to do this task):
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §1, §14.2, §15 ·
[rust-agentic-task.md](../../../rust-agentic-task.md)