feat(llm-serving): serve clef-flash on candle-vllm

Adds clef-candle.yaml: clef-flash (hybrid Qwen3_5, 8 attention + 24 GDN layers) on one V100 via candle-vllm, and repoints the clef-serve Service selector at it. The Service name and ClusterIP do not move, so no client changes. Runs on one card because the GDN recurrent state is per-sequence and cannot be sharded the way attention KV can. --kv-fraction 0.30 gives about 66.8K KV tokens, which is ample for a single-shot classifier-style endpoint. The image is candle-vllm:sm70, built by .gitea/workflows/candle-vllm-sm70.yaml on every merge to main. It must be a build containing the GDN conv fix: older images zero the key channel band in causal_conv1d_fwd, starve the delta rule and make every linear-attention layer return zero. That failure is silent - the response is well formed, just wrong. Verified against the reference implementation: token ids 420/420 identical, beta 8647.021 vs 8646.982, delta-rule and gated-norm absmax equal to six decimals, all three head answers agree.
2026-11-05 09:35:47 +08:00
parent 7bdc8b2de2
commit 3ab127e581
3 changed files with 88 additions and 1 deletions
@@ -21,6 +21,10 @@ resources:
  # streams against 71.5 ms and 215 tok/s for the 4-bit MoE. The V100 has FP16
  # tensor cores and no INT4 ones, so 4-bit pays dequantisation on the SM while
  # the memory controller idles at 19%.
+  # clef-candle.yaml serves clef-flash on candle-vllm (hybrid Qwen3_5, one GPU)
+  # and owns the clef-candle label that clef-serve.yaml's Service now selects.
+  # It must not sync against an image built before the GDN conv fix.
+  - clef-candle.yaml
  - coder-14b-fp16.yaml
  - clef-serve.yaml
  # Stable client-facing name for the chat model. poimen-memory points at this Service,