Commit Graph
4 Commits
Author SHA1 Message Date
rock 95c67f9437 P3.8: Restrict llm-serving ingress to api-gateway only (#15)
## Summary

Replaces hand-applied `llm-serving-default-deny` NetworkPolicy with a git-managed, namespace-scoped policy that only allows traffic from the api-gateway.

**Issue:** #13

Co-authored-by: rock <[email protected]>
2026-09-08 16:56:36 +00:00
Story Crater Bot 5d7b9acd43 feat(llm-serving): scale ornith to 2 replicas instead of a dedicated grm GPU
reasoning keeps its 2 GPUs untouched. verifier's freed GPU goes to a second ornith replica instead of a standalone qwen-only pod -- both replicas load ornith:35b + qwen2.5:3b-instruct, k8s Service load-balances across them, so 2 concurrent implementer-style calls get independent instances.
2026-08-18 18:25:12 -07:00
Story Crater Bot 50d00ae350 feat(llm-serving): retire verifier-predictor, add grm (qwen2.5:3b)
Frees verifier's GPU from an underused vLLM PRM deployment. qwen2.5:3b-instruct moves off ornith-predictor's shared pod onto its own dedicated GPU (grm.yaml), so verification/judge traffic stops contending with ornith:35b's agent traffic. /v1/qwen/chat/completions now points at grm-predictor; path unchanged.
2026-08-18 18:18:31 -07:00
Story Crater Bot afb9b35292 fix(agent-pod): remote tui session for multi-agent 2026-08-18 15:08:04 -07:00