rock/homelab
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

fix(llm-serving): enable tool calling on homelab-reasoning

Browse Source 
pi sends tool_choice="auto" for every session (Read/Bash/etc.) -- vLLM 400s on that without --enable-auto-tool-choice and a --tool-call-parser. Verified this deployed vLLM v0.11.0's registered parsers directly; deepseek_v3 matches, same family as the deepseek_r1 reasoning-parser already set (this Qwen-base distillation still emits DeepSeek's own tool-call format).
This commit is contained in:
Story Crater Bot
2026-08-18 20:50:21 -07:00
parent 1a4160b3de
commit 9774dea895
1 changed files with 10 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
k8s/apps/llm-serving/reasoning.yaml
+10
View File
@@ -24,6 +24,16 @@ spec:
- --enable-chunked-prefill - --enable-chunked-prefill
- --enable-prefix-caching - --enable-prefix-caching
- --reasoning-parser=deepseek_r1 - --reasoning-parser=deepseek_r1
# Without these, any client sending tool_choice="auto" (pi does, for
# Read/Bash/etc.) gets a 400: "auto" tool choice requires
# --enable-auto-tool-choice and --tool-call-parser to be set. Verified
# against this exact deployed vLLM v0.11.0's registered parsers
# (ToolParserManager.tool_parsers) -- deepseek_v3 matches, same family
# as the deepseek_r1 reasoning-parser already set above (this model
# emits DeepSeek's own special-token format, not Qwen's Hermes-style,
# despite being a Qwen-base distillation).
- --enable-auto-tool-choice
- --tool-call-parser=deepseek_v3
- --host=0.0.0.0 - --host=0.0.0.0
- --port=8080 - --port=8080
env: env: