chore: initial commit of Go API gateway

Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
2026-08-19 20:54:34 -07:00
commit 058f11cf2b
109 changed files with 8992 additions and 0 deletions
@@ -0,0 +1,52 @@
+# 2.14 — Queue position event on `/llm/*` (GREEN)
+
+Phase: 2 — LLM surfaces
+Stage: GREEN
+Depends on: [2.12](2.12-anthropic-sse-state-machine.md), [4.1](4.1-gpu-slot-semaphore.md)
+
+The Anthropic event set has no way to say "you are queued". Its stream implicitly
+begins after a slot has been acquired, so a queued client sees nothing at all until
+generation starts. The `reasoning` upstream has only 8 sequence slots cluster-wide and
+the gateway caps below that, so waiting is normal and worth showing.
+
+- [ ] When a streaming `/llm/v1/messages` request waits for a slot, the gateway emits a
+      frame with `event: queue` before any `message_start`
+- [ ] The queue frame's data carries the caller's current position in the queue
+- [ ] Position updates are emitted as the queue drains, each as another `event: queue`
+      frame, until a slot is acquired
+- [ ] Once a slot is acquired the stream continues with the standard sequence beginning
+      at `message_start`, and no further queue frame is emitted for that request
+- [ ] A request that acquires a slot immediately emits no queue frame at all
+- [ ] Queue frames are flushed as they are produced, not buffered behind the first
+      upstream token
+- [ ] A client that disconnects while still queued is removed from the queue, never
+      reaches the upstream and never consumes a slot
+- [ ] Non-streaming requests emit no queue frames; they simply wait, then answer
+- [ ] The extension is documented in the surface's own docs as non-standard, alongside
+      the fact that a strict Anthropic client ignoring unknown events degrades to
+      showing nothing while queued rather than erroring
+
+This is a deliberate departure from the Anthropic contract. It is safe only because
+the sole client of `/llm/*` is the first-party riotpiao frontend. It must never be
+required for correctness: dropping every `event: queue` frame leaves a valid,
+complete Anthropic stream.
+
+## Verify
+
+```bash
+# Fill the slots first, with cap=2 configured against a stub that holds each request 3s.
+seq 4 | xargs -P4 -I{} curl -s -o /dev/null -X POST localhost:8080/llm/v1/messages \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' &
+
+sleep 1
+curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
+  | grep -m3 '^event:'
+# expected: event: queue arrives within a second, before any event: message_start
+
+curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"qwen2.5:3b-instruct","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
+  | head -1
+# expected: event: message_start, no queue frame on an uncontended upstream
+```