Files
homelab/k8s/apps/llm-serving
rock ad2720a724 fix(llm-serving): stop shipping clef-flash, it cannot run on a V100
Clef needs SGLang v0.5.22+, which upstream publishes only on CUDA 13. CUDA 13 dropped
Volta kernels and every GPU here is a Tesla V100 (sm_70):

  torch.AcceleratorError: CUDA error: no kernel image is available for execution on
  the device

SGLang also states sm75 as its floor: the README hardware table omits V100, sm75 support
was removed in sgl-project/sglang#9207, and FlashInfer requires sm75+. No build has both
Clef and Volta kernels, so the model is not runnable here at any dtype or flag.

Remove it from the kustomization so nothing crash-loops, and record the reason in both
files. The manifest stays in git for the day an Ampere-or-newer GPU arrives.
2026-10-28 10:29:24 -07:00
..