The published :sm70 predates /v1/systemone and the GDN conv fix. candle-vllm
f3c00b2 carries both and is on main, but CI has not republished the tag, so the
in-image binary returns 404 on the route and would zero every linear-attention
layer if it did.
/mnt/models/bin/candle-vllm is that build. Verified serving: HTTP 200,
severity medium 0.5029, rollback 0.6581, confidence 2.6079, matching the
reference's decisions.
Temporary: retire once CI publishes a :sm70 from f3c00b2.
:sm70 is rebuilt by CI on every merge to candle-vllm main. The default
IfNotPresent means a node with a cached copy of that tag keeps running the old
binary, so a fix can be merged and deployed and still never reach the pod. The
symptom is a 404 on /v1/systemone while the image tag looks correct.
Always forces the pull.
candle-vllm PR #5 (the conv fix) could not be merged, so CI has not published
a :sm70 image containing it. Deploying clef against an older :sm70 fails
silently: the delta rule returns zero on all 24 linear-attention layers and
/v1/systemone still answers 200 with well-formed but wrong classifications.
Raise to 1 once a :sm70 build from the fix merge exists.
Adds clef-candle.yaml: clef-flash (hybrid Qwen3_5, 8 attention + 24 GDN
layers) on one V100 via candle-vllm, and repoints the clef-serve Service
selector at it. The Service name and ClusterIP do not move, so no client
changes.
Runs on one card because the GDN recurrent state is per-sequence and cannot
be sharded the way attention KV can. --kv-fraction 0.30 gives about 66.8K KV
tokens, which is ample for a single-shot classifier-style endpoint.
The image is candle-vllm:sm70, built by .gitea/workflows/candle-vllm-sm70.yaml
on every merge to main. It must be a build containing the GDN conv fix: older
images zero the key channel band in causal_conv1d_fwd, starve the delta rule
and make every linear-attention layer return zero. That failure is silent -
the response is well formed, just wrong.
Verified against the reference implementation: token ids 420/420 identical,
beta 8647.021 vs 8646.982, delta-rule and gated-norm absmax equal to six
decimals, all three head answers agree.
rock