chore: initial commit of Go API gateway

Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 20:54:34 -07:00
commit 058f11cf2b
109 changed files with 8992 additions and 0 deletions
@@ -0,0 +1,39 @@
+# 2.2 — Body-based model dispatch (GREEN)
+
+Phase: 2 — LLM surface
+Stage: GREEN
+Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md), [1.2](1.2-streaming-passthrough.md)
+
+- [ ] `POST /v1/chat/completions` selects its upstream from the `model` field of the JSON request body
+- [ ] The body reaching the upstream is byte-identical to the body received, `model` included — dispatch reads, it does not rewrite
+- [ ] The upstream sees the canonical path `/v1/chat/completions`
+- [ ] `"model":"reasoning"` reaches `reasoning-predictor.llm-serving:80`; `"model":"ornith:35b"` and `"model":"qwen2.5:3b-instruct"` both reach `ornith-predictor.llm-serving:80`
+- [ ] `"stream":true` streams unbuffered — chunks reach the client as the upstream emits them, and are not accumulated in order to inspect the body
+- [ ] A client disconnect mid-stream cancels the upstream request rather than orphaning it
+- [ ] `reasoning_content` is passed through untouched alongside `content`; the gateway does not merge, reorder or strip either
+- [ ] Reading the body to find `model` respects the route's body size cap and does not load an unbounded request into memory
+
+This is the single capability Kong OSS lacked — `ai-proxy-advanced` is Enterprise-only —
+and the entire reason this gateway exists. Everything else in this phase is a
+consequence of it. Ports are 80, not 8080.
+
+An orphaned generation holds one of only eight vLLM sequence slots in the cluster,
+which is why disconnect cancellation belongs in the acceptance criteria of this task
+and not only in the proxy layer.
+
+## Verify
+
+```bash
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, stub for reasoning-predictor recorded the hit with path /v1/chat/completions and an unmodified body
+
+curl -s localhost:8080/v1/chat/completions -H 'content-type: application/json' \
+  -d '{"model":"qwen2.5:3b-instruct","messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, ornith-predictor stub hit, reasoning-predictor stub not hit
+
+curl -N -s localhost:8080/v1/chat/completions -H 'content-type: application/json' \
+  -d '{"model":"reasoning","stream":true,"messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, content-type text/event-stream, first data: chunk arrives before the stub finishes emitting
+```