riotpiao-poimen/homelab
2
0
Fork 0
Code Issues 23 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

fix(llm-serving): clef-flash asks for memory the node can reserve

Browse Source 
worker-1 holds 64Gi and the other predictors request about 55Gi, so the 16Gi request
left clef-flash Pending with '0/4 nodes are available: 1 Insufficient memory'. The
weights live on the GPU; host memory covers SGLang's own bookkeeping, so 8Gi is the
honest reservation. The 32Gi limit stays, so a burst still gets the room.
This commit is contained in:
rock
2026-10-27 23:59:28 -07:00
parent a0d2859c1d
commit f6e569c0e8
1 changed files with 6 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
k8s/apps/llm-serving/clef-flash.yaml
+6 -1
View File
@@ -104,7 +104,12 @@ spec:
nvidia.com/gpu: '1' nvidia.com/gpu: '1'
requests: requests:
cpu: '8' cpu: '8'
memory: 16Gi # 8Gi, not 16Gi. worker-1 holds 64Gi and the other predictors already
# request about 55Gi, so a 16Gi request left the pod Pending forever:
# "1 Insufficient memory". The weights live on the GPU, so host memory is
# for SGLang's own bookkeeping. The limit stays at 32Gi, so a burst that
# needs more is allowed; only the scheduler reservation is smaller.
memory: 8Gi
nvidia.com/gpu: '1' nvidia.com/gpu: '1'
startupProbe: startupProbe:
# The image is large and the checkpoint is 18.8 GB on first pull. # The image is large and the checkpoint is 18.8 GB on first pull.