Files
homelab/k8s/apps
Story Crater Bot ff9f0e99e3 reasoning: fall back to official Qwen3-30B-A3B-Thinking-2507 AWQ-4bit
Kimi-K2.6-distilled Qwen3.6-35B-A3B crashed on boot -- model type
qwen3_5_moe unrecognized by transformers/vLLM 0.11.0, a genuinely
unsupported architecture, not a config issue. Using cpatonn's pre-quantized
AWQ-4bit build of the official Qwen3-30B-A3B-Thinking-2507 instead: native
vLLM support confirmed, no Kimi distillation but Qwen3's own tool-call
format is natively supported (the actual root problem being solved).
Restored max-num-seqs=4 since AWQ-4bit weight footprint leaves more KV
headroom than the bnb attempts did.
2026-08-19 18:20:34 -07:00
..