feat(cag): memory standing-context adapter, warmup semaphore, cache-aware routing

Issue #44: code-register memory adapter with standing-context resource; fix
dispatcher method-level Auth.Capability override (was always using verb-derived
capability, ignoring per-method Auth field).

Issue #45: per-backend bounded in-memory queue (VLLM_MAX_QUEUE_DEPTH=20) +
concurrency semaphore (VLLM_MAX_NUM_SEQS=4); warmup requests (X-Poimen-Warmup:
true) sleep WARMUP_DELAY_MS before queue entry; /metrics endpoint wired.

Issue #46: rendezvous-hash backend selection on X-Poimen-Cache-Key; round-robin
fallback; MarkFailed excludes unhealthy backends 30s; ModifyResponse captures
vllm-num-cached-tokens + emits structured log + increments counter.

Closes #44 Closes #45 Closes #46
This commit is contained in:
Admin Bot
2026-09-27 18:21:41 +08:00
parent ea7bd8336b
commit fc5c0e967d
16 changed files with 861 additions and 17 deletions
+8
View File
@@ -198,3 +198,11 @@ func FromContext(ctx context.Context) *Logger {
func WithContext(ctx context.Context, l *Logger) context.Context {
return context.WithValue(ctx, "logger", l)
}
// Event emits arbitrary fields as a structured JSON line.
// Use when field values include non-string types (numerics, floats).
func Event(fields map[string]interface{}) {
ensure()
data, _ := json.Marshal(fields)
_, _ = global.out.Write(append(data, '\n'))
}