Files
homelab/k8s/apps/llm-serving
Story Crater Bot ea0c00f76e reasoning: swap to dense Qwen3-32B-bnb-4bit for reliable tool calling
DeepSeek-R1-distill's tool_choice=auto narration bug needed a real fix,
not a workaround -- Qwen3's native tool-call format (hermes-compatible
chat template) solves it at the source instead of parsing around it.
Dense Qwen3-32B avoids the MoE arch/quantization pitfalls hit by the two
prior swap attempts (Kimi-distilled Qwen3.6 MoE, AWQ Qwen3-30B-A3B) --
same bnb-4bit path already proven working on this sm70 (V100) node.
2026-08-19 18:40:15 -07:00
..