feat(llm-serving): serve clef-flash

The gateway already dispatches /v1/systemone and its config already names
clef-flash-predictor, but this directory is an explicit allowlist and clef-flash.yaml
was not on it, so the model was silently dropped: no error, no drift, just no
predictor.

clef-flash is a decision model, not a chat model. It reads a state plus a schema of
typed questions and returns one probability per allowed option in a single prefill
pass, with no decode step. It needs SGLang rather than vLLM because it loads separate
joint_schema_head weights on top of the Qwen3.5-9B backbone and it implements the
/v1/systemone route.

worker-1 has four GPUs: reasoning holds two, ornith holds one, so one is free and this
fits without retiring ornith first.

This commit is contained in:

rock

2026-10-27 23:51:45 -07:00

parent 4d23c983ab

commit 9aec1b3e30

2 changed files with 148 additions and 0 deletions

									
										k8s/apps/llm-serving/kustomization.yaml
									
		+1
		
												View File
												
				@@ -10,6 +10,7 @@ kind: Kustomization

				# way after any edit here: a GPU predictor restart is a weights reload measured

				# in tens of seconds, not a rolling update.

				resources:

				  - clef-flash.yaml

				  - embeddings.yaml

				  - ornith.yaml

				  - reasoning.yaml