fix(llm-serving): hold clef-candle at replicas 0 until the image carries the fix #110

Open
rock wants to merge 7 commits from feat/clef-candle-vllm into main
Showing only changes of commit 9617bbe814 - Show all commits
+6 -1
View File
@@ -7,7 +7,12 @@ metadata:
app.kubernetes.io/part-of: llm-serving
app: clef-candle
spec:
replicas: 1
# HOLD AT 0 until candle-vllm:sm70 contains the GDN conv fix (candle-vllm PR #5).
# An older :sm70 zeroes the key channel band, starves the delta rule and makes
# every linear-attention layer return zero. The response is still HTTP 200 and
# well formed, so the failure is silent. Raise to 1 only after CI publishes a
# :sm70 build from the merge that carries the fix.
replicas: 0
strategy:
# One replica holds one GPU, so Recreate keeps the rollout to a single
# weights load instead of needing a second card for a surge pod.