Commit Graph
1 Commits
Author SHA1 Message Date
rock 9aec1b3e30 feat(llm-serving): serve clef-flash
The gateway already dispatches /v1/systemone and its config already names
clef-flash-predictor, but this directory is an explicit allowlist and clef-flash.yaml
was not on it, so the model was silently dropped: no error, no drift, just no
predictor.

clef-flash is a decision model, not a chat model. It reads a state plus a schema of
typed questions and returns one probability per allowed option in a single prefill
pass, with no decode step. It needs SGLang rather than vLLM because it loads separate
joint_schema_head weights on top of the Qwen3.5-9B backbone and it implements the
/v1/systemone route.

worker-1 has four GPUs: reasoning holds two, ornith holds one, so one is free and this
fits without retiring ornith first.
2026-10-27 23:51:45 -07:00