feat(llm-serving): poimen ingress, 256K coder, and a stable name for the candle-vllm handoff (#90)

Co-authored-by: rock <[email protected]>
2026-11-03 21:47:47 +00:00
parent 53e29d4476
commit c1477afc92
5 changed files with 221 additions and 1 deletions
@@ -14,7 +14,14 @@ resources:
  # v0.5.22+, published only on CUDA 13, and CUDA 13 dropped Volta kernels. The model
  # instead runs the implementation it ships (joint_schema_model.py) in fp16 on one V100.
  - clef-serve.yaml
+  # coder = Qwen3-Coder-30B-A3B-Instruct, 4-bit GPTQ, TP=2 at 262,144 tokens. It serves
+  # both `coder` and `reasoning`, so poimen-memory needs a host change only, never a
+  # model-name change. reasoning.yaml below can be dropped once that host change lands.
+  - coder.yaml
  - reasoning.yaml
+  # Stable client-facing name for the chat model. poimen-memory points at this Service,
+  # so the vLLM -> candle-vllm handoff is a selector edit here and no client change.
+  - reasoning-long-service.yaml
  # ornith is retired: its InferenceService still reports Ready=True while running zero
  # replicas, because desired equals available at 0. Its manifest was never committed,
  # so it is left unmanaged and is not listed here. Delete the CR when convenient.