riotpiao-poimen/homelab
2
0
Fork 0
Code Issues 23 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

fix(llm-serving): stop shipping clef-flash, it cannot run on a V100

Browse Source 
Clef needs SGLang v0.5.22+, which upstream publishes only on CUDA 13. CUDA 13 dropped
Volta kernels and every GPU here is a Tesla V100 (sm_70):

  torch.AcceleratorError: CUDA error: no kernel image is available for execution on
  the device

SGLang also states sm75 as its floor: the README hardware table omits V100, sm75 support
was removed in sgl-project/sglang#9207, and FlashInfer requires sm75+. No build has both
Clef and Volta kernels, so the model is not runnable here at any dtype or flag.

Remove it from the kustomization so nothing crash-loops, and record the reason in both
files. The manifest stays in git for the day an Ampere-or-newer GPU arrives.
This commit is contained in:
rock
2026-10-28 10:29:24 -07:00
parent 695f03a0ab
commit ad2720a724
1 changed files with 16 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
k8s/apps/llm-serving/kustomization.yaml
+16 -1
View File
@@ -10,7 +10,22 @@ kind: Kustomization
# way after any edit here: a GPU predictor restart is a weights reload measured
# in tens of seconds, not a rolling update.
resources:
- clef-flash.yaml
# clef-flash.yaml is NOT listed: the model cannot run on this cluster.
#
# Clef needs SGLang v0.5.22 or newer, and SGLang publishes that only on CUDA 13
# (the last CUDA 12 tag is v0.5.19-cu129, which predates Clef support). CUDA 13
# dropped Volta kernels, and every GPU here is a Tesla V100 (sm_70):
#
# torch.AcceleratorError: CUDA error: no kernel image is available for
# execution on the device
#
# SGLang also states sm75 as its floor: the README hardware list omits V100,
# sm75 support was removed in sgl-project/sglang#9207, and FlashInfer needs
# sm75+. So no SGLang build exists that has both Clef and Volta kernels, and
# no flag or dtype change bridges that gap.
#
# Re-add it when the model gets an Ampere-or-newer GPU. The manifest itself is
# kept in git (k8s/apps/llm-serving/clef-flash.yaml) with the same reasoning.
# ornith and reasoning are live but unmanaged: their manifests were never committed
# to this directory, so listing them made `kustomize build` fail and left the whole
# Application in ComparisonError. Remove the entries rather than the models. The