54 lines
3.0 KiB
Markdown
54 lines
3.0 KiB
Markdown
|
# 2.10 — Anthropic request translation (GREEN)
|
||
|
|
|
||
|
|
Phase: 2 — LLM surfaces
|
||
|
|
Stage: GREEN
|
||
|
|
Depends on: [2.9](2.9-canonical-request-model.md), [2.1](2.1-model-registry.md)
|
||
|
|
|
||
|
|
`POST /llm/v1/messages` accepts an Anthropic Messages request body and turns it into
|
||
|
|
the dialect-neutral canonical request. The path carries `/v1/messages` because the
|
||
|
|
Anthropic base-URL convention appends that suffix; the gateway prefix is `/llm`.
|
||
|
|
|
||
|
|
- [ ] `POST /llm/v1/messages` is accepted and selects its upstream from the body's
|
||
|
|
`model` field, using the same registry as `/v1`: `reasoning` reaches
|
||
|
|
`reasoning-predictor.llm-serving:80`, `ornith:35b` and `qwen2.5:3b-instruct`
|
||
|
|
reach `ornith-predictor.llm-serving:80`
|
||
|
|
- [ ] The top-level `system` field becomes the canonical system instruction; it is a
|
||
|
|
distinct field in this dialect and is not a member of `messages`
|
||
|
|
- [ ] Each message `content` may be a plain string or an array of blocks; a string and
|
||
|
|
a single text block carrying the same characters translate identically
|
||
|
|
- [ ] `max_tokens` is REQUIRED on this surface, matching the Anthropic contract; a
|
||
|
|
request without it is rejected as a client error and no upstream is contacted
|
||
|
|
- [ ] `max_tokens` above what the upstream can serve is clamped rather than rejected,
|
||
|
|
and the clamp is logged; `reasoning` has `--max-model-len=16384`
|
||
|
|
- [ ] Roles are restricted to `user` and `assistant`; any other role, including
|
||
|
|
`system` inside `messages`, is a client error naming the offending role
|
||
|
|
- [ ] `stop_sequences` becomes the canonical stop sequences, and `stream` becomes the
|
||
|
|
canonical streaming flag
|
||
|
|
- [ ] `tools`, `tool_choice`, any `tool_use` or `tool_result` block, any `image` block,
|
||
|
|
any cache-control marker, and any user message with more than one content block
|
||
|
|
are rejected as unsupported, naming the feature; none is silently dropped
|
||
|
|
- [ ] Unknown top-level fields are rejected rather than ignored, so a client cannot
|
||
|
|
believe an unimplemented option took effect
|
||
|
|
- [ ] Reading the body respects the route's configured body size cap
|
||
|
|
|
||
|
|
`max_tokens` stays optional on `/v1/chat/completions`. The policy is per-surface: each
|
||
|
|
dialect keeps its own contract, and the canonical request records whichever value
|
||
|
|
resulted.
|
||
|
|
|
||
|
|
## Verify
|
||
|
|
|
||
|
|
```bash
|
||
|
|
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/llm/v1/messages \
|
||
|
|
-H 'content-type: application/json' \
|
||
|
|
-d '{"model":"reasoning","max_tokens":64,"system":"be terse","messages":[{"role":"user","content":"hi"}]}'
|
||
|
|
# expected: 200, reasoning stub hit, system text present in the upstream request
|
||
|
|
|
||
|
|
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||
|
|
-d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
|
||
|
|
# expected: 400, body names max_tokens as the missing required field, no upstream stub hit
|
||
|
|
|
||
|
|
curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
|
||
|
|
-d '{"model":"reasoning","max_tokens":8,"messages":[{"role":"system","content":"x"}]}'
|
||
|
|
# expected: 400, body names the rejected role, no upstream stub hit
|
||
|
|
```
|