Story Crater BotandClaude Opus 5
058f11cf2b
Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2.5 KiB
2.5 KiB
2.9 — Dialect-neutral canonical request (GREEN)
Phase: 2 — LLM surfaces Stage: GREEN Depends on: 2.1, 2.2
The gateway serves two permanent protocol dialects: /v1/* is OpenAI-compatible and
/llm/* is Anthropic Messages. Both translate into one internal shape before anything
else happens to them, and both translate back out of it on the way to the client.
- A single internal request representation exists that carries at minimum: the resolved upstream, the model name as the client sent it, the ordered turns, an optional system instruction, a maximum output token count, stop sequences, and a streaming flag
- The representation names no dialect: nothing in it is called openai or anthropic, and no field exists solely because one dialect happens to spell it that way
- Model dispatch, the shared slot controller, per-caller budgets, logging and metrics all read the canonical request and never the raw dialect body
- The surface that received a request is recorded as one field on the canonical request, used only to choose the response and error rendering, never to choose an upstream or a slot
- An identical prompt sent to
/v1/chat/completionsand to/llm/v1/messagesproduces the same resolved upstream, the same slot accounting and the same log fields apart from that one surface label - Adding a third dialect requires a new translator only; the slot controller, dispatch and budget code are untouched
- Translation failure is a client error at the surface boundary, and no partially populated canonical request ever reaches an upstream
reasoning maps to reasoning-predictor.llm-serving:80; ornith:35b and
qwen2.5:3b-instruct both map to ornith-predictor.llm-serving:80. Ports are 80.
Verify
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
# expected: 200, reasoning stub hit once
curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/llm/v1/messages \
-H 'content-type: application/json' \
-d '{"model":"reasoning","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}'
# expected: 200, reasoning stub hit once, same upstream and same slot counter as the /v1 call
grep -h 'upstream=' /tmp/gateway.log | tail -2
# expected: both lines show upstream=reasoning-predictor and model=reasoning, differing only in the surface label