homelab-frontend/tasks/2.11-anthropic-response-translation.md

2.11 — Anthropic non-streaming response translation (GREEN)

Phase: 2 — LLM surfaces Stage: GREEN Depends on: 2.10, 2.9

A non-streaming POST /llm/v1/messages gets an Anthropic Messages response, built from whatever the upstream returned. Upstreams speak the OpenAI chat-completion shape; the client on this surface must never see it.

• The response body is {"id","type":"message","role":"assistant","content":[...], "model","stop_reason","stop_sequence","usage":{"input_tokens","output_tokens"}} with type literally message and role literally assistant
• model echoes the model name the client sent, not an upstream-internal name
• Upstream finish_reason stop becomes stop_reason end_turn, and length becomes max_tokens
• A generation halted by a client-supplied stop sequence reports stop_reason stop_sequence and puts the matched string in stop_sequence; otherwise stop_sequence is null and present, not omitted
• Upstream prompt_tokens becomes usage.input_tokens and completion_tokens becomes usage.output_tokens; no other usage fields are invented
• reasoning_content, which vLLM returns as a field separate from content for reasoning, becomes a thinking content block that PRECEDES the text block
• When reasoning_content is absent or empty, no thinking block is emitted and content holds only the text block
• When content is empty but reasoning_content is not, the thinking block is still returned rather than an empty content array
• Content-Type is application/json, and no OpenAI field name such as choices, finish_reason or object appears anywhere in the body

reasoning runs DeepSeek-R1-Distill-Qwen-32B under vLLM with --reasoning-parser=deepseek_r1, which is why the reasoning text arrives as its own field and maps cleanly onto a thinking block.

Verify

curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
  -d '{"model":"reasoning","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}'
# expected: 200, type=message, role=assistant, content[0].type=thinking, content[1].type=text

curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
  -d '{"model":"reasoning","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}' \
  | grep -c -E '"choices"|"finish_reason"|"object"'
# expected: 0

curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
  -d '{"model":"qwen2.5:3b-instruct","max_tokens":4,"messages":[{"role":"user","content":"count to fifty"}]}'
# expected: 200, stop_reason=max_tokens, usage has input_tokens and output_tokens only