fix(llm-serving): clef-flash asks for memory the node can reserve

worker-1 holds 64Gi and the other predictors request about 55Gi, so the 16Gi request left clef-flash Pending with '0/4 nodes are available: 1 Insufficient memory'. The weights live on the GPU; host memory covers SGLang's own bookkeeping, so 8Gi is the honest reservation. The 32Gi limit stays, so a burst still gets the room.
2026-10-27 23:59:28 -07:00
parent a0d2859c1d
commit f6e569c0e8
1 changed files with 6 additions and 1 deletions
@@ -104,7 +104,12 @@ spec:
          nvidia.com/gpu: '1'
        requests:
          cpu: '8'
-          memory: 16Gi
+          # 8Gi, not 16Gi. worker-1 holds 64Gi and the other predictors already
+          # request about 55Gi, so a 16Gi request left the pod Pending forever:
+          # "1 Insufficient memory". The weights live on the GPU, so host memory is
+          # for SGLang's own bookkeeping. The limit stays at 32Gi, so a burst that
+          # needs more is allowed; only the scheduler reservation is smaller.
+          memory: 8Gi
          nvidia.com/gpu: '1'
      startupProbe:
        # The image is large and the checkpoint is 18.8 GB on first pull.