riotpiao-poimen/homelab-frontend
2
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(cag): CAG integration — memory adapter, warmup semaphore, cache-aware routing #47

Merged
rock merged 6 commits from harness/44-45-46-cag-integration into main 2026-09-27 11:16:41 +00:00
Conversation 0 Commits 6 Files Changed 16
+1597 -162
6 Commits
Author SHA1 Message Date
Admin Bot 78a5d6baa2 fix(proxy): mutex on transports, ctx-aware warmup delay, Retry-After on 503
CI / CI (pull_request) Successful in 2m11s
Details 
- transportsMu (sync.RWMutex): guard getOrCreateTransport reads/writes;
  double-checked locking pattern; safe for dynamic backends
- applyWarmup: replace time.Sleep with select{time.After, ctx.Done}
  so cancelled warmup requests release the goroutine immediately
- Retry-After: 5 header on queue-full 503 (client backoff hint)
- OTel trace_id skipped: collector in tracing ns, not monitoring ns
 2026-09-27 19:13:38 +08:00
Admin Bot 6257b8c201 fix(proxy): authenticateV1 Warnf + JWT test coverage (CRAP 42→9)
CI / CI (pull_request) Successful in 2m36s
Details 
- proxy.go: log Warnf when jwtValidator==nil so auth bypass is observable
- proxy_test.go: TestAuthenticateV1ProxyLevel covers 401/403/pass branches
  (authenticateV1 cov: 10.5% → 77.3%, CRAP: 42 → 8.5)
 2026-09-27 19:09:10 +08:00
Admin Bot a9a151a8b4 refactor(proxy): decompose ServeHTTP into 6 helpers (comp 38→9, CRAP<30) 
Extract handleAuthEndpoint, routeOrError, writeModelValidationError, authenticateV1,
enforceBodyLimit, setupCAGProxy. No behavior change. All tests pass with -race.
 2026-09-27 19:02:26 +08:00
Admin Bot d703ace2c6 fix(cag): extract applyWarmup/acquireScheduledBackend helpers, add race-free tests 
- proxy.go: extract applyWarmup + acquireScheduledBackend to reduce ServeHTTP complexity
- proxy_test.go: TestProxyQueueFull, TestProxyCacheHitLogging (AC5/AC6/AC7 HTTP-level coverage)
- scheduler_test.go: TestBackendAcquireCancelledContext + runtime.Gosched spin-wait (no time.Sleep)
- router_test.go: TestAuthenticate_MethodCapabilityOverrideDirect with real RS256 JWT + JWKS server
 2026-09-27 18:57:04 +08:00
Admin Bot a731185588 fix(cag): resolve data races, add queue observability, fix test coverage 
Cycle 2 fixes from reviewer-sre + reviewer-quality:

- scheduler.go: remove delete() under RLock (data race); expired entries
  skipped on read, cleaned lazily by next MarkFailed write
- proxy.go: pre-populate transports for scheduler backends at startup to
  prevent concurrent map writes from ServeHTTP goroutines
- proxy.go: log + IncrQueueFull() on queue-full 503 path (was silent)
- proxy.go: remove stub total_tokens/hit_rate from cache-hit log event
- serviceadapter/router.go: Warnf when Auth.Required but jwtValidator nil
- observability: add queue_full_requests_total counter + Prometheus export
- scheduler_test.go: TestBackendAcquireCancelledContext (ctx cancel branch)
- proxy_test.go: implement TestWarmupHeaderHandling (remove t.Skip)
- router_test.go: TestAuthenticate_MethodCapabilityOverrideDirect (auth coverage)
 2026-09-27 18:39:45 +08:00
Admin Bot fc5c0e967d feat(cag): memory standing-context adapter, warmup semaphore, cache-aware routing 
Issue #44: code-register memory adapter with standing-context resource; fix
dispatcher method-level Auth.Capability override (was always using verb-derived
capability, ignoring per-method Auth field).

Issue #45: per-backend bounded in-memory queue (VLLM_MAX_QUEUE_DEPTH=20) +
concurrency semaphore (VLLM_MAX_NUM_SEQS=4); warmup requests (X-Poimen-Warmup:
true) sleep WARMUP_DELAY_MS before queue entry; /metrics endpoint wired.

Issue #46: rendezvous-hash backend selection on X-Poimen-Cache-Key; round-robin
fallback; MarkFailed excludes unhealthy backends 30s; ModifyResponse captures
vllm-num-cached-tokens + emits structured log + increments counter.

Closes #44 Closes #45 Closes #46
 2026-09-27 18:21:41 +08:00