Replaces the Qwen3-32B dense deployment, which needed two V100s for 131k context.
The MoE carries 96 KiB of KV per token against the dense model's 256 KiB, so the same
64k window fits on one card.
Measured on this cluster before the switch: 15.55 GiB of weights, 137k tokens of KV
cache, 2.09x concurrency at 65536 tokens per request, correct answer over a 5921-token
prompt, and clean reasoning_content separation through the gateway.
Three settings are load-bearing on sm_70 rather than tuning:
- VLLM_USE_FLASHINFER_SAMPLER=0 flashinfer's check_cuda_arch raises on CC 7.0
- VLLM_ATTENTION_BACKEND=TRITON_ATTN FlashInfer and FA2 both need sm75+
- deploymentStrategy Recreate a rolling update deadlocks once every card is allocated
The manifest is committed as well as applied: reasoning.yaml had never been in git.
rock
3eb6a91311