riotpiao-poimen/homelab
2
0
Fork 0
Code Issues 23 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

fix(llm-serving): call the sglang CLI by its full path

Browse Source 
The container args began with a bare 'serve'. This image installs its CLI at
/opt/sglang/bin/sglang and puts nothing called serve on PATH, so every start ended
with 'exec: serve: not found' — 19 restarts, and the InferenceService never became
ready. Name the binary, then the subcommand.
This commit is contained in:
rock
2026-10-28 02:34:42 -07:00
parent 13343976e4
commit 569eb84737
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
k8s/apps/llm-serving/clef-flash.yaml
+4
View File
@@ -53,6 +53,10 @@ spec:
# layers use linear attention and hold a fixed 48 MiB state regardless # layers use linear attention and hold a fixed 48 MiB state regardless
# of length. Memory is therefore dominated by the BF16 weights (18.8 GB), # of length. Memory is therefore dominated by the BF16 weights (18.8 GB),
# not the cache, which is why one V100-32GB is enough. # not the cache, which is why one V100-32GB is enough.
# The image ships its CLI at /opt/sglang/bin/sglang and puts no `serve` on
# PATH, so a bare `serve` failed with "exec: serve: not found" on every start
# (19 restarts). Name the binary by its full path.
- /opt/sglang/bin/sglang
- serve - serve
- --model-path=Cloudflare/clef-flash - --model-path=Cloudflare/clef-flash
- --host=0.0.0.0 - --host=0.0.0.0