chore(comfyui): scale to 0 to free the 4th V100 for llm-serving

comfyui has served nothing for 42h while holding the 4th GPU on worker-1.
Its oauth2-proxy roll-out is stuck: the pod requests client-id,
client-secret and cookie-secret from Secret comfyui/comfyui-oauth2-proxy,
and that Secret carries only 1 key. It failed 11621 times. The old
ReplicaSet keeps running, so the Deployment reports 1/1 while every
request is refused at the auth step. ArgoCD reports the app Degraded.

The card sat at 0% utilisation the whole time.

llm-serving has no spare card. worker-1 has 4 and all 4 are allocated:
clef-serve, coder-predictor, reasoning-predictor, and this one. Freeing
it is the cheapest way to give an LLM workload headroom, because nothing
else on the node can yield a card without a service interruption.

This change is one field: replicas 1 -> 0. The Deployment, Service,
Ingress and the 50Gi comfyui-models PVC are untouched, so no data is at
risk and restoring is a one-line revert plus a fix to the Secret.
This commit is contained in:
2026-10-31 16:16:17 -07:00
parent 2d9e3abee8
commit 6e2b681b25
+12 -1
View File
@@ -8,7 +8,18 @@ metadata:
labels:
app: comfyui
spec:
replicas: 1
# Retired on purpose. This Deployment held the 4th V100 on worker-1 while
# serving nothing: its oauth2-proxy roll-out has been stuck for 42h and
# failing 11621 times, because the Secret comfyui/comfyui-oauth2-proxy carries
# only 1 key while the pod requests client-id, client-secret and
# cookie-secret. Every request was therefore refused, so the card sat at 0%
# utilisation and blocked a GPU that llm-serving can use.
#
# Scaling to 0 is the whole change. The Deployment, the Service, the Ingress
# and the 50Gi comfyui-models PVC all stay, so nothing is lost and no data is
# touched. To bring it back: set replicas to 1 here, then fix the Secret to
# carry all three keys, otherwise the same stuck roll-out returns.
replicas: 0
strategy:
type: Recreate
selector: