51 lines
2.8 KiB
Markdown
51 lines
2.8 KiB
Markdown
|
# 2.11 — Anthropic non-streaming response translation (GREEN)
|
||
|
|
|
||
|
|
Phase: 2 — LLM surfaces
|
||
|
|
Stage: GREEN
|
||
|
|
Depends on: [2.10](2.10-anthropic-request-translation.md), [2.9](2.9-canonical-request-model.md)
|
||
|
|
|
||
|
|
A non-streaming `POST /llm/v1/messages` gets an Anthropic Messages response, built
|
||
|
|
from whatever the upstream returned. Upstreams speak the OpenAI chat-completion shape;
|
||
|
|
the client on this surface must never see it.
|
||
|
|
|
||
|
|
- [ ] The response body is `{"id","type":"message","role":"assistant","content":[...],
|
||
|
|
"model","stop_reason","stop_sequence","usage":{"input_tokens","output_tokens"}}`
|
||
|
|
with `type` literally `message` and `role` literally `assistant`
|
||
|
|
- [ ] `model` echoes the model name the client sent, not an upstream-internal name
|
||
|
|
- [ ] Upstream `finish_reason` `stop` becomes `stop_reason` `end_turn`, and `length`
|
||
|
|
becomes `max_tokens`
|
||
|
|
- [ ] A generation halted by a client-supplied stop sequence reports `stop_reason`
|
||
|
|
`stop_sequence` and puts the matched string in `stop_sequence`; otherwise
|
||
|
|
`stop_sequence` is null and present, not omitted
|
||
|
|
- [ ] Upstream `prompt_tokens` becomes `usage.input_tokens` and `completion_tokens`
|
||
|
|
becomes `usage.output_tokens`; no other usage fields are invented
|
||
|
|
- [ ] `reasoning_content`, which vLLM returns as a field separate from `content` for
|
||
|
|
`reasoning`, becomes a `thinking` content block that PRECEDES the `text` block
|
||
|
|
- [ ] When `reasoning_content` is absent or empty, no `thinking` block is emitted and
|
||
|
|
`content` holds only the `text` block
|
||
|
|
- [ ] When `content` is empty but `reasoning_content` is not, the `thinking` block is
|
||
|
|
still returned rather than an empty `content` array
|
||
|
|
- [ ] `Content-Type` is `application/json`, and no OpenAI field name such as `choices`,
|
||
|
|
`finish_reason` or `object` appears anywhere in the body
|
||
|
|
|
||
|
|
`reasoning` runs DeepSeek-R1-Distill-Qwen-32B under vLLM with
|
||
|
|
`--reasoning-parser=deepseek_r1`, which is why the reasoning text arrives as its own
|
||
|
|
field and maps cleanly onto a thinking block.
|
||
|
|
|
||
|
|
## Verify
|
||
|
|
|
||
|
|
```bash
|
||
|
|
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||
|
|
-d '{"model":"reasoning","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}'
|
||
|
|
# expected: 200, type=message, role=assistant, content[0].type=thinking, content[1].type=text
|
||
|
|
|
||
|
|
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||
|
|
-d '{"model":"reasoning","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}' \
|
||
|
|
| grep -c -E '"choices"|"finish_reason"|"object"'
|
||
|
|
# expected: 0
|
||
|
|
|
||
|
|
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||
|
|
-d '{"model":"qwen2.5:3b-instruct","max_tokens":4,"messages":[{"role":"user","content":"count to fifty"}]}'
|
||
|
|
# expected: 200, stop_reason=max_tokens, usage has input_tokens and output_tokens only
|
||
|
|
```
|