fix(llm-serving): run clef-candle from the fixed binary on the models PVC #112

Merged
rock merged 10 commits from fix/clef-run-pvc-binary into main 2026-11-05 02:20:01 +00:00
Showing only changes of commit ea8299c417 - Show all commits
+5
View File
@@ -42,6 +42,11 @@ spec:
# and makes every linear-attention layer return zero. The symptom is
# a well-formed /v1/systemone response with wrong answers, not an error.
image: forgejo.riotpiao.com/riotpiao-poimen/candle-vllm:sm70
# :sm70 is a moving tag. With the default IfNotPresent a rebuilt image
# is never pulled while any node holds a cached copy of that tag, so the
# cluster keeps serving the old binary and the fix looks like it did not
# land. Always forces the pull on every pod start.
imagePullPolicy: Always
command: ["/usr/local/bin/candle-vllm"]
# clef-flash is a hybrid Qwen3_5 model: 8 full-attention layers plus 24
# GDN layers that carry per-sequence recurrent state. It runs on ONE