fix(gateway): stop routing clef-flash, the model cannot run here #73

2026-10-28T17:30:45Z

rock commented

2026-10-28 17:30:45 +00:00

Clef needs SGLang v0.5.22+, published only on CUDA 13, and CUDA 13 dropped Volta kernels. Every GPU is a Tesla V100 (sm_70), so SGLang aborts with "no kernel image is available for execution on the device". SGLang states sm75 as its floor regardless.

Advertising clef-flash meant a caller waited on a Service with no endpoints until its own timeout fired. This removes the entry and keeps the reasoning in its place.

Clef needs SGLang v0.5.22+, published only on CUDA 13, and CUDA 13 dropped Volta kernels. Every GPU is a Tesla V100 (sm_70), so SGLang aborts with "no kernel image is available for execution on the device". SGLang states sm75 as its floor regardless. Advertising clef-flash meant a caller waited on a Service with no endpoints until its own timeout fired. This removes the entry and keeps the reasoning in its place.

rock added 1 commit 2026-10-28 17:30:46 +00:00

fix(gateway): stop routing clef-flash, the model cannot run here

CI / CI (pull_request) Successful in 2m7s

Details

9ff41e3131

The model list still pointed clef-flash at clef-flash-predictor. That predictor is gone:
Clef needs SGLang v0.5.22+, published only on CUDA 13, and CUDA 13 dropped Volta
kernels. Every GPU is a Tesla V100 (sm_70), so SGLang aborts with 'no kernel image is
available'. SGLang's floor is sm75 regardless.

Advertising it meant a caller would wait for a Service with no endpoints until its own
timeout fired, instead of failing clearly. Remove the entry and keep the reasoning in
its place, so restoring it later is a deliberate act tied to new hardware.

rock merged commit b9f469a92f into main

2026-10-28 17:30:55 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: riotpiao-poimen/homelab-frontend#73