 Admin Bot
|
d703ace2c6
|
fix(cag): extract applyWarmup/acquireScheduledBackend helpers, add race-free tests
- proxy.go: extract applyWarmup + acquireScheduledBackend to reduce ServeHTTP complexity
- proxy_test.go: TestProxyQueueFull, TestProxyCacheHitLogging (AC5/AC6/AC7 HTTP-level coverage)
- scheduler_test.go: TestBackendAcquireCancelledContext + runtime.Gosched spin-wait (no time.Sleep)
- router_test.go: TestAuthenticate_MethodCapabilityOverrideDirect with real RS256 JWT + JWKS server
|
2026-09-27 18:57:04 +08:00 |
|
|
Admin Bot
|
a731185588
|
fix(cag): resolve data races, add queue observability, fix test coverage
Cycle 2 fixes from reviewer-sre + reviewer-quality:
- scheduler.go: remove delete() under RLock (data race); expired entries
skipped on read, cleaned lazily by next MarkFailed write
- proxy.go: pre-populate transports for scheduler backends at startup to
prevent concurrent map writes from ServeHTTP goroutines
- proxy.go: log + IncrQueueFull() on queue-full 503 path (was silent)
- proxy.go: remove stub total_tokens/hit_rate from cache-hit log event
- serviceadapter/router.go: Warnf when Auth.Required but jwtValidator nil
- observability: add queue_full_requests_total counter + Prometheus export
- scheduler_test.go: TestBackendAcquireCancelledContext (ctx cancel branch)
- proxy_test.go: implement TestWarmupHeaderHandling (remove t.Skip)
- router_test.go: TestAuthenticate_MethodCapabilityOverrideDirect (auth coverage)
|
2026-09-27 18:39:45 +08:00 |
|
|
Admin Bot
|
fc5c0e967d
|
feat(cag): memory standing-context adapter, warmup semaphore, cache-aware routing
Issue #44: code-register memory adapter with standing-context resource; fix
dispatcher method-level Auth.Capability override (was always using verb-derived
capability, ignoring per-method Auth field).
Issue #45: per-backend bounded in-memory queue (VLLM_MAX_QUEUE_DEPTH=20) +
concurrency semaphore (VLLM_MAX_NUM_SEQS=4); warmup requests (X-Poimen-Warmup:
true) sleep WARMUP_DELAY_MS before queue entry; /metrics endpoint wired.
Issue #46: rendezvous-hash backend selection on X-Poimen-Cache-Key; round-robin
fallback; MarkFailed excludes unhealthy backends 30s; ModifyResponse captures
vllm-num-cached-tokens + emits structured log + increments counter.
Closes #44 Closes #45 Closes #46
|
2026-09-27 18:21:41 +08:00 |
|
  poimenandrock
|
74ecfe7107
|
feat(serviceadapter): enforce JWT auth on X-Service dispatch (#16)
SQS dispatcher hardcoded a JWT validator pointing at authentik.riotpiao.com/application/o/sqs/jwks/ — provider doesn't exist. Every SQS request got 403 regardless of token.
Co-authored-by: poimen <[email protected]>
|
2026-09-08 23:20:31 +00:00 |
|