reasoning: fall back to official Qwen3-30B-A3B-Thinking-2507 AWQ-4bit

Kimi-K2.6-distilled Qwen3.6-35B-A3B crashed on boot -- model type
qwen3_5_moe unrecognized by transformers/vLLM 0.11.0, a genuinely
unsupported architecture, not a config issue. Using cpatonn's pre-quantized
AWQ-4bit build of the official Qwen3-30B-A3B-Thinking-2507 instead: native
vLLM support confirmed, no Kimi distillation but Qwen3's own tool-call
format is natively supported (the actual root problem being solved).
Restored max-num-seqs=4 since AWQ-4bit weight footprint leaves more KV
headroom than the bnb attempts did.

k8s/apps/llm-serving/reasoning.yaml

@@ -18,61 +18,50 @@ spec:
       # <think> block instead of emitting real tool_calls, regardless of
       # parser combo tried (deepseek_v3 400s outright, hermes parsed but the
       # model itself never called out to the real tool-call path). Root
-      # cause is upstream in the R1 distillation, not this config -- moving
-      # to a Qwen3-family model with a Kimi-K2.6 reasoning distillation
-      # instead, since Qwen3's own tool-call format is natively supported.
+      # cause is upstream in the R1 distillation, not this config -- Qwen3's
+      # own tool-call format is natively supported instead.
       #
-      # No pre-quantized AWQ/GPTQ/bnb checkpoint exists for this specific
-      # distilled model (only a GGUF, which is llama.cpp/Ollama-only and not
-      # usable here) -- pointing --quantization=bitsandbytes at the full
-      # bf16 checkpoint directly, which makes vLLM quantize on load instead
-      # of requiring a pre-quantized repo. This on-the-fly bnb path is
-      # well-trodden for dense models but less battle-tested for MoE
-      # (this model is 35B total / ~3B active) -- watch first boot closely;
-      # if it OOMs or errors on the MoE expert weights, that's the likely
-      # cause.
-      - --model=lordx64/Qwen3.6-35B-A3B-Kimi-K2.6-Reasoning-Distilled
+      # First attempt was a Kimi-K2.6-distilled Qwen3.6-35B-A3B checkpoint
+      # (lordx64/Qwen3.6-35B-A3B-Kimi-K2.6-Reasoning-Distilled) -- crashed on
+      # boot: `model type qwen3_5_moe` unrecognized by transformers/vLLM
+      # 0.11.0, not a config problem, a genuinely unsupported/obscure
+      # architecture. Falling back to the official Qwen3-30B-A3B-Thinking-2507
+      # (no Kimi distillation, but native vLLM support confirmed) using
+      # cpatonn's pre-quantized AWQ-4bit build -- avoids repeating the
+      # on-the-fly bnb gamble a second time.
+      - --model=cpatonn/Qwen3-30B-A3B-Thinking-2507-AWQ-4bit
       - --served-model-name=reasoning
-      - --quantization=bitsandbytes
-      - --trust-remote-code
-      - --dtype=bfloat16
+      - --quantization=awq_marlin
+      - --dtype=float16
       - --kv-cache-dtype=auto
       - --tensor-parallel-size=1
       - --max-model-len=16384
       - --gpu-memory-utilization=0.90
-      # Halved from 4 -- weights alone (35B total, MoE, all experts loaded
-      # regardless of ~3B active) at on-the-fly bnb 4bit are ~19-21GB on a
-      # 32GB card at 0.90 util (~29.5GB budget), leaving only ~8-10GB for KV
-      # cache + activations. Old DeepSeek-32B was in the same weight
-      # ballpark but MoE bnb overhead here is untested. Starting at 2 to
-      # confirm real headroom via nvidia-smi/vLLM logs before raising back.
-      - --max-num-seqs=2
+      # 30B total MoE at pre-quantized AWQ-4bit is ~15-16GB weights on a
+      # 32GB card at 0.90 util (~29.5GB budget) -- meaningfully more KV-cache
+      # headroom than the old DeepSeek-32B config had, so restoring the
+      # original max-num-seqs=4 rather than starting conservative again.
+      - --max-num-seqs=4
       - --enable-chunked-prefill
       - --enable-prefix-caching
-      # qwen3 parser handles this family's <think> reasoning blocks (best
-      # match for this architecture; unverified against this exact
-      # checkpoint -- if it 400s or fails to strip <think> tags, that's the
-      # first thing to check).
+      # qwen3 is vLLM's dedicated reasoning parser for this family's <think>
+      # blocks -- official pairing per Qwen3-Thinking's own deployment docs.
       - --reasoning-parser=qwen3
-      # hermes previously verified (on the old model) to work against a
-      # Qwen tokenizer's plain-text tool-call patterns without needing
-      # special tokens; Qwen3's native tool-call format is also
-      # hermes-style, so kept as-is.
+      # hermes is the documented tool-call parser for general Qwen3 models
+      # (qwen3_coder/qwen3_xml are Coder-variant-only).
       - --enable-auto-tool-choice
       - --tool-call-parser=hermes
       # vLLM 0.11.0's native OffloadingConnector -- spills KV cache blocks
       # to CPU DRAM instead of discarding them on preemption (max-num-seqs=4
       # + max-model-len=16384 means concurrent long sequences compete for
       # the same GPU KV space). No extra dependency, built into vLLM core.
-      # num_cpu_blocks=2000 hung the pod at startup ("Allocating 64 CPU
-      # tensors..." then nothing -- 64 is this model's layer count, one
-      # pinned host tensor per layer, each sized for every CPU block; 2000
-      # was oversized enough to stall pinning that much host memory, likely
-      # blowing well past the pod's memory limit). Dropped to a small,
-      # known-safe starting point -- confirm it actually comes up healthy,
-      # then watch real host memory usage and raise it deliberately rather
-      # than guessing a round number again. block_size=128 tokens matches
-      # vLLM's own example.
+      # num_cpu_blocks=2000 hung the pod at startup on the old model (one
+      # pinned host tensor per layer, sized per CPU block -- oversized enough
+      # to stall pinning that much host memory). 32 is a small, known-safe
+      # starting point independent of this model's own layer count --
+      # confirm it comes up healthy, then watch real host memory usage and
+      # raise it deliberately rather than guessing a round number again.
+      # block_size=128 tokens matches vLLM's own example.
       # Note: 0.11.0 ships the original (fragmented, small-transfer-block)
       # version of this connector -- 0.12.0 consolidates KV data into one
       # contiguous block per request and is reported an order of magnitude