fix(llm-serving): run clef-candle from the fixed binary on the models PVC

The published :sm70 predates /v1/systemone and the GDN conv fix. candle-vllm
f3c00b2 carries both and is on main, but CI has not republished the tag, so the
in-image binary returns 404 on the route and would zero every linear-attention
layer if it did.

/mnt/models/bin/candle-vllm is that build. Verified serving: HTTP 200,
severity medium 0.5029, rollback 0.6581, confidence 2.6079, matching the
reference's decisions.

Temporary: retire once CI publishes a :sm70 from f3c00b2.
This commit is contained in:
2026-11-05 10:19:33 +08:00
parent ef764e97c7
commit aa41e8c509
+9 -1
View File
@@ -41,7 +41,15 @@ spec:
# rebuilt image never reaches the pod, so a merged fix looks like it did
# not land. Always forces the pull on every pod start.
imagePullPolicy: Always
command: ["/usr/local/bin/candle-vllm"]
# Runs the fixed binary from the models PVC, not the one in the image.
# The published :sm70 predates the /v1/systemone route and the GDN conv
# fix (candle-vllm f3c00b2 is merged but CI has not republished the tag),
# so the in-image binary 404s on the route. The PVC copy at
# /mnt/models/bin/candle-vllm contains both fixes.
#
# RETIRE THIS once CI publishes a :sm70 built from f3c00b2: change the
# command back to /usr/local/bin/candle-vllm and delete the PVC copy.
command: ["/mnt/models/bin/candle-vllm"]
# clef-flash is a hybrid Qwen3_5 model: 8 full-attention layers plus 24
# GDN layers that carry per-sequence recurrent state. It runs on ONE
# card because the state is not shardable the way attention KV is.