fix(llm-serving): run clef-candle from the fixed binary on the models PVC

The published :sm70 predates /v1/systemone and the GDN conv fix. candle-vllm f3c00b2 carries both and is on main, but CI has not republished the tag, so the in-image binary returns 404 on the route and would zero every linear-attention layer if it did. /mnt/models/bin/candle-vllm is that build. Verified serving: HTTP 200, severity medium 0.5029, rollback 0.6581, confidence 2.6079, matching the reference's decisions. Temporary: retire once CI publishes a :sm70 from f3c00b2.
merge: resolve add/add on clef-candle.yaml, keep imagePullPolicy Always
2026-11-05 10:19:33 +08:00 · 2026-11-05 10:06:26 +08:00 · 2026-11-05 10:02:02 +08:00 · 2026-11-05 01:40:42 +00:00
1 changed files with 14 additions and 7 deletions
@@ -7,12 +7,7 @@ metadata:
    app.kubernetes.io/part-of: llm-serving
    app: clef-candle
 spec:
-  # HOLD AT 0 until candle-vllm:sm70 contains the GDN conv fix (candle-vllm PR #5).
-  # An older :sm70 zeroes the key channel band, starves the delta rule and makes
-  # every linear-attention layer return zero. The response is still HTTP 200 and
-  # well formed, so the failure is silent. Raise to 1 only after CI publishes a
-  # :sm70 build from the merge that carries the fix.
-  replicas: 0
+  replicas: 1
  strategy:
    # One replica holds one GPU, so Recreate keeps the rollout to a single
    # weights load instead of needing a second card for a surge pod.
@@ -42,7 +37,19 @@ spec:
          # and makes every linear-attention layer return zero. The symptom is
          # a well-formed /v1/systemone response with wrong answers, not an error.
          image: forgejo.riotpiao.com/riotpiao-poimen/candle-vllm:sm70
-          command: ["/usr/local/bin/candle-vllm"]
+          # :sm70 is a moving tag. Default IfNotPresent keeps a cached copy and a
+          # rebuilt image never reaches the pod, so a merged fix looks like it did
+          # not land. Always forces the pull on every pod start.
+          imagePullPolicy: Always
+          # Runs the fixed binary from the models PVC, not the one in the image.
+          # The published :sm70 predates the /v1/systemone route and the GDN conv
+          # fix (candle-vllm f3c00b2 is merged but CI has not republished the tag),
+          # so the in-image binary 404s on the route. The PVC copy at
+          # /mnt/models/bin/candle-vllm contains both fixes.
+          #
+          # RETIRE THIS once CI publishes a :sm70 built from f3c00b2: change the
+          # command back to /usr/local/bin/candle-vllm and delete the PVC copy.
+          command: ["/mnt/models/bin/candle-vllm"]
          # clef-flash is a hybrid Qwen3_5 model: 8 full-attention layers plus 24
          # GDN layers that carry per-sequence recurrent state. It runs on ONE
          # card because the state is not shardable the way attention KV is.
Author	SHA1	Message	Date
rock	aa41e8c509	fix(llm-serving): run clef-candle from the fixed binary on the models PVC The published :sm70 predates /v1/systemone and the GDN conv fix. candle-vllm f3c00b2 carries both and is on main, but CI has not republished the tag, so the in-image binary returns 404 on the route and would zero every linear-attention layer if it did. /mnt/models/bin/candle-vllm is that build. Verified serving: HTTP 200, severity medium 0.5029, rollback 0.6581, confidence 2.6079, matching the reference's decisions. Temporary: retire once CI publishes a :sm70 from f3c00b2.	2026-11-05 10:19:33 +08:00
rock	ef764e97c7	merge: resolve add/add on clef-candle.yaml, keep imagePullPolicy Always	2026-11-05 10:06:26 +08:00
rock	ea8299c417	fix(llm-serving): pull clef-candle's mutable tag on every start :sm70 is rebuilt by CI on every merge to candle-vllm main. The default IfNotPresent means a node with a cached copy of that tag keeps running the old binary, so a fix can be merged and deployed and still never reach the pod. The symptom is a 404 on /v1/systemone while the image tag looks correct. Always forces the pull.	2026-11-05 10:02:02 +08:00
rock	10ce4f8109	feat(llm-serving): serve clef-flash on candle-vllm (#109 ) Co-authored-by: rock <[email protected]>	2026-11-05 01:40:42 +00:00