Story Crater Bot 7bec3a8c49 feat(llm-serving): offload reasoning's KV cache to CPU DRAM ...

vLLM 0.11.0's native OffloadingConnector -- spills KV blocks to CPU RAM on
preemption instead of discarding them, avoiding recompute. Built into vLLM
core, no extra dependency. Bumped memory request/limit (+4Gi/replica) to
give the CPU block pool real room; worker-1 had ~18Gi of request headroom
across both replicas.

2026-08-19 15:02:28 -07:00

..

embeddings.yaml

fix(agent-pod): remote tui session for multi-agent

2026-08-18 15:08:04 -07:00

kustomization.yaml

feat(llm-serving): scale ornith to 2 replicas instead of a dedicated grm GPU

2026-08-18 18:25:12 -07:00

ornith.yaml

feat(llm-serving): scale ornith to 2 replicas instead of a dedicated grm GPU

2026-08-18 18:25:12 -07:00

reasoning.yaml

feat(llm-serving): offload reasoning's KV cache to CPU DRAM

2026-08-19 15:02:28 -07:00

reranker.yaml

fix(agent-pod): remote tui session for multi-agent

2026-08-18 15:08:04 -07:00