Admin Bot
fc5c0e967d
feat(cag): memory standing-context adapter, warmup semaphore, cache-aware routing
Issue #44: code-register memory adapter with standing-context resource; fix
dispatcher method-level Auth.Capability override (was always using verb-derived
capability, ignoring per-method Auth field).
Issue #45: per-backend bounded in-memory queue (VLLM_MAX_QUEUE_DEPTH=20) +
concurrency semaphore (VLLM_MAX_NUM_SEQS=4); warmup requests (X-Poimen-Warmup:
true) sleep WARMUP_DELAY_MS before queue entry; /metrics endpoint wired.
Issue #46: rendezvous-hash backend selection on X-Poimen-Cache-Key; round-robin
fallback; MarkFailed excludes unhealthy backends 30s; ModifyResponse captures
vllm-num-cached-tokens + emits structured log + increments counter.
Closes #44 Closes #45 Closes #46
2026-09-27 18:21:41 +08:00
..
2026-09-06 13:45:04 +00:00
2026-09-27 18:21:41 +08:00
2026-09-08 23:08:39 +00:00
2026-09-13 22:50:12 +00:00
2026-09-27 18:21:41 +08:00
2026-09-15 08:53:58 +00:00
2026-09-27 18:21:41 +08:00
2026-08-26 13:47:36 -07:00
2026-09-27 18:21:41 +08:00
2026-08-26 13:47:36 -07:00
2026-09-27 18:21:41 +08:00
2026-09-27 18:21:41 +08:00
2026-09-26 18:50:49 +08:00
2026-08-21 20:46:45 -07:00
2026-08-31 15:01:56 -07:00
2026-09-16 07:49:15 +09:00
2026-08-21 16:49:44 -07:00