fix(llm-serving): set runtimeClassName nvidia on the candle deployment
Without it the container runs under runc, libcuda.so is never injected and candle-vllm panics in cudarc at startup. The pod reached Running but crashed on every restart. Verified live: pod Ready, coder-predictor endpoints populated, /v1/models reports coder-14b-fp16 and a completion returns OK.
This commit is contained in:
@@ -60,6 +60,10 @@ spec:
|
||||
app.kubernetes.io/part-of: llm-serving
|
||||
app: candle-vllm
|
||||
spec:
|
||||
# Required. Without this the container runs under runc, libcuda.so is never
|
||||
# injected, and candle-vllm panics in cudarc at startup with
|
||||
# "Unable to dynamically load the cuda shared library".
|
||||
runtimeClassName: nvidia
|
||||
containers:
|
||||
- name: candle
|
||||
image: forgejo.riotpiao.com/riotpiao-poimen/candle-vllm:sm70-gptq-slim
|
||||
|
||||
Reference in New Issue
Block a user