comfyui has served nothing for 42h while holding the 4th GPU on worker-1.
Its oauth2-proxy roll-out is stuck: the pod requests client-id,
client-secret and cookie-secret from Secret comfyui/comfyui-oauth2-proxy,
and that Secret carries only 1 key. It failed 11621 times. The old
ReplicaSet keeps running, so the Deployment reports 1/1 while every
request is refused at the auth step. ArgoCD reports the app Degraded.
The card sat at 0% utilisation the whole time.
llm-serving has no spare card. worker-1 has 4 and all 4 are allocated:
clef-serve, coder-predictor, reasoning-predictor, and this one. Freeing
it is the cheapest way to give an LLM workload headroom, because nothing
else on the node can yield a card without a service interruption.
This change is one field: replicas 1 -> 0. The Deployment, Service,
Ingress and the 50Gi comfyui-models PVC are untouched, so no data is at
risk and restoring is a one-line revert plus a fix to the Secret.
Root cause: k8s Service named 'comfyui' auto-injects COMFYUI_PORT=tcp://...
into pod env, clobbering ai-dock's own COMFYUI_PORT variable which expects
a port number. This broke caddy's proxy config, leaving port 8188 dead.
Fix: enableServiceLinks: false, revert to port 8188 (ai-dock default).
Also bump startup probe failureThreshold 60->120 (20min budget for model loading).
rock