Files
homelab/k8s/apps/llm-serving
Story Crater Bot 5a8fc1885a fix(llm-serving): use hermes tool-call parser, not deepseek_v3
deepseek_v3 400s on this checkpoint: "could not locate tool call start/end tokens in the tokenizer". unsloth/DeepSeek-R1-Distill-Qwen-32B is a Qwen2.5 base distilled on R1 reasoning traces -- it kept R1's <think> format but never got DeepSeek-V3's own special tool-call tokens registered in its tokenizer. hermes parses from text patterns instead of special tokens, so it works against the underlying Qwen tokenizer.
2026-08-18 20:56:40 -07:00
..