fix(llm-serving): set runtimeClassName nvidia on the candle deployment

Without it the container runs under runc, libcuda.so is never injected and
candle-vllm panics in cudarc at startup. The pod reached Running but crashed
on every restart.

Verified live: pod Ready, coder-predictor endpoints populated, /v1/models
reports coder-14b-fp16 and a completion returns OK.
This commit is contained in:
2026-11-05 09:22:48 +08:00
parent f6fca4705b
commit 7bdc8b2de2
+4
View File
@@ -60,6 +60,10 @@ spec:
app.kubernetes.io/part-of: llm-serving
app: candle-vllm
spec:
# Required. Without this the container runs under runc, libcuda.so is never
# injected, and candle-vllm panics in cudarc at startup with
# "Unable to dynamically load the cuda shared library".
runtimeClassName: nvidia
containers:
- name: candle
image: forgejo.riotpiao.com/riotpiao-poimen/candle-vllm:sm70-gptq-slim