chore: initial commit of Go API gateway

Baseline for the Kong replacement on api.riotpiao.com. Brings the working
tree under version control for the first time: gateway source, the task
board that drives the agent runs, test fixtures, and K8s manifests.

Anchor the gateway ignore rule to the repo root. Unanchored, "gateway"
also matched the cmd/gateway/ source directory, so the program entrypoint
was excluded from every commit.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

tasks/0.1-module-and-entrypoint.md

@@ -0,0 +1,32 @@
+# 0.1 — Module and entrypoint (GREEN)
+
+Phase: 0 — Foundations
+Stage: GREEN
+
+- [x] A single Go module builds one static binary with no cgo
+- [x] The binary reads its configuration at startup and serves HTTP on a configurable listen address
+- [x] `SIGTERM` starts a drain: the listener stops accepting new connections, in-flight requests run to completion, then the process exits `0`
+- [x] A request already in flight when `SIGTERM` arrives receives its full, uncorrupted response body
+- [x] A request arriving after `SIGTERM` is not accepted on a new connection
+- [x] The drain has a bounded deadline; exceeding it forces exit with a non-zero code and a logged reason
+- [x] The process holds no Kubernetes credentials and makes no API-server calls
+
+The gateway sits behind ingress-nginx, which owns TLS. The gateway never terminates
+TLS and never listens on 443. Graceful drain matters because in-flight requests here
+are LLM generations that can legitimately run for many minutes -> killing them
+mid-stream loses work a caller cannot cheaply redo.
+
+## Verify
+
+```bash
+go test ./internal/server/... -run TestGracefulShutdown -race -v
+# expected: passes — a slow in-flight request completes with a full body after SIGTERM,
+# and a request issued post-SIGTERM is refused; process exit code is 0
+
+CGO_ENABLED=0 go build ./... && go vet ./...
+# expected: both succeed
+```
+
+`-race` is required, not optional. A server that starts a listener in one goroutine and
+exposes its address from another is the obvious shape here, and it is racy unless the
+shared state is guarded. A test that passes without `-race` proves nothing about it.

tasks/0.2-route-configuration.md

@@ -0,0 +1,29 @@
+# 0.2 — Declarative route configuration (RED)
+
+Phase: 0 — Foundations
+Stage: RED
+
+- [x] Routes and upstreams are declared in YAML loaded from a file path at startup
+- [x] Each upstream declares: address, path rewrite, connect timeout, read timeout, write timeout, maximum request body size, and an auth-required flag
+- [x] Every one of those fields is explicit — no silent defaults for timeouts, body caps or auth
+- [x] A config missing any required field fails startup with a non-zero exit and a message naming the offending route and field
+- [x] A config with a malformed duration, an unparseable address, or a duplicate route key fails startup the same way
+- [x] A valid config round-trips: every declared route is present in the loaded route table
+- [x] Loading is startup-only — no API-server watch, no CRD, no Kubernetes client
+
+Configuration lives in git and is mounted as a ConfigMap synced by Argo. It is
+deliberately not a CRD: a CRD would require the gateway to watch the API server,
+which needs RBAC and contradicts the invariant that the gateway holds no cluster
+credentials. It is also the exact indirection being retired with Kong, whose routing
+table was split across six `KongPlugin` CRs, seven Ingresses and a Helm values file.
+
+A gateway that starts with a silently dropped route is worse than one that refuses to
+start.
+
+## Verify
+
+```bash
+go test ./internal/config/... -v
+# expected: passes — valid fixtures load with all routes present; each invalid fixture
+# returns an error naming the offending route and field, and none of them load partially
+```

tasks/0.3-health-endpoints.md

@@ -0,0 +1,25 @@
+# 0.3 — Health endpoints (GREEN)
+
+Phase: 0 — Foundations
+Stage: GREEN
+Depends on: [0.2](0.2-route-configuration.md)
+
+- [x] `GET /healthz` returns `200` whenever the process is alive
+- [x] `GET /healthz` contacts no upstream and performs no network I/O
+- [x] `GET /readyz` returns `200` only when configuration is valid and, if auth is enabled, JWKS has been fetched at least once
+- [x] `GET /readyz` returns a non-`2xx` status while configuration is invalid or JWKS has never been fetched
+- [x] Neither endpoint requires authentication, even when the auth flag is on
+- [x] Neither path is proxied to any upstream, and neither can be shadowed by a configured route
+
+`/healthz` backs the liveness probe, so it must stay cheap and must not fail because
+an upstream is down — restarting the gateway does not fix a sick vLLM pod. `/readyz`
+backs the readiness probe and is allowed to fail, taking the pod out of the nginx
+endpoint pool until it can actually serve.
+
+## Verify
+
+```bash
+go test ./internal/server/... -run TestHealthEndpoints -v
+# expected: passes — /healthz is 200 with upstreams unreachable; /readyz is non-2xx
+# before first JWKS fetch and 200 after; both answer with no Authorization header
+```

tasks/0.4-local-dev-harness.md

@@ -0,0 +1,29 @@
+# 0.4 — Local development harness (GREEN)
+
+Phase: 0 — Foundations
+Stage: GREEN
+Depends on: [0.2](0.2-route-configuration.md)
+
+- [ ] The whole gateway runs from a checkout with no cluster, no kubeconfig and no credentials of any kind
+- [ ] A committed local config points every upstream at stub servers started by the harness
+- [ ] Stubs can serve a fixed JSON body, an SSE token stream, a chunked response, and a slow response
+- [ ] A test can assert on the real HTTP response: status, headers and body
+- [ ] A test can assert that streamed chunks arrive incrementally, before the upstream has finished
+- [ ] A test can disconnect the client mid-response and assert on what the stub upstream observed
+- [ ] One documented command runs the harness end to end and exits non-zero on failure
+- [ ] Running the harness never contacts `*.riotpiao.com` or any cluster address
+
+This is a hard requirement, not a convenience: it determines whether work can proceed
+unattended. Upstreams are configuration, so pointing them at local stubs is the entire
+mechanism. Every later phase's verification depends on this existing first.
+
+"It compiles" and "it starts" are not verification. Asserting on an actual HTTP
+response is.
+
+## Verify
+
+```bash
+env -u KUBECONFIG go test ./internal/testsupport/... ./internal/proxy/... -v
+# expected: passes with no kubeconfig and no network access beyond loopback —
+# includes an SSE test asserting incremental arrival and a mid-response disconnect test
+```

tasks/0.5-structured-logging.md

@@ -0,0 +1,26 @@
+# 0.5 — Structured logging (GREEN)
+
+Phase: 0 — Foundations
+Stage: GREEN
+Depends on: [0.1](0.1-module-and-entrypoint.md)
+
+- [ ] Logs are emitted as structured records with a consistent field set, one record per line
+- [ ] Every request log carries at least: route, upstream, method, path, status, duration
+- [ ] Every rejected request is logged with an explicit machine-readable reason field
+- [ ] Request bodies are never logged, in whole or in part
+- [ ] `Authorization` header values, bearer tokens, API keys and JWKS material are never logged, not even truncated or hashed-with-prefix
+- [ ] Log level is configurable, and no level unlocks body or token logging
+- [ ] A test asserts a rejected request produces exactly one record containing the reason and containing no token substring
+
+Rejections come from several layers — unknown model, body too large, auth failure,
+concurrency limit — and the reason field is what makes them countable later. The model
+API carries prompts that are user content and tokens that are credentials; neither
+belongs in a log line.
+
+## Verify
+
+```bash
+go test ./internal/logging/... ./internal/server/... -run 'TestLog' -v
+# expected: passes — captured log output for a rejected request contains the reason
+# field and does not contain the request body or the bearer token used
+```

tasks/0.6-ci-pipeline.md

@@ -0,0 +1,25 @@
+# 0.6 — CI pipeline (GREEN)
+
+Phase: 0 — Foundations
+Stage: GREEN
+Depends on: [0.4](0.4-local-dev-harness.md)
+
+- [ ] CI builds the binary on every push and pull request
+- [ ] CI runs `go vet` over all packages and fails on any finding
+- [ ] CI runs the full test suite, including the local harness tests, with the race detector on
+- [ ] CI runs `govulncheck` and fails the job on any HIGH or CRITICAL severity finding
+- [ ] CI needs no cluster, no kubeconfig and no credentials to pass
+- [ ] A deliberately broken commit — failing test, vet finding, or known-vulnerable dependency — fails CI rather than passing silently
+- [ ] Job status is visible on the commit or pull request
+
+CI is the outer loop for the same closed verification loop the harness gives locally.
+It must not depend on cluster access, or it stops running the moment the cluster is
+unavailable.
+
+## Verify
+
+```bash
+go vet ./... && go test -race ./... && govulncheck ./...
+# expected: all three exit 0 locally; pushing a branch with a failing test shows a
+# failed CI run on that commit
+```

tasks/1.1-reverse-proxy.md

@@ -0,0 +1,26 @@
+# 1.1 — Reverse proxy to configured upstreams (GREEN)
+
+Phase: 1 — Proxy core
+Stage: GREEN
+Depends on: [0.2](0.2-route-configuration.md), [0.4](0.4-local-dev-harness.md)
+
+- [ ] A request matching a configured route is proxied to that route's upstream address
+- [ ] The upstream's status code, response headers and body reach the client unmodified
+- [ ] The request method, query string and body reach the upstream unmodified
+- [ ] The route's configured path rewrite is applied to the upstream request path
+- [ ] Connections to upstreams are pooled and reused across requests — a second request to the same upstream does not open a new TCP connection
+- [ ] A request matching no configured route returns `404` and contacts no upstream
+- [ ] An unreachable upstream returns a `5xx` to the client and is logged with the upstream name
+
+Connection reuse is not a micro-optimisation here: the chat upstreams hold long-lived
+streaming responses, and churning connections under that pattern wastes handshakes and
+file descriptors on both ends.
+
+## Verify
+
+```bash
+go test ./internal/proxy/... -run 'TestProxy|TestConnectionReuse' -v
+# expected: passes — stub upstream sees the rewritten path and original body, client
+# sees the stub's exact status/headers/body, and the stub records one accepted
+# connection across two sequential requests
+```

tasks/1.2-streaming-passthrough.md

@@ -0,0 +1,26 @@
+# 1.2 — Streaming passthrough (RED)
+
+Phase: 1 — Proxy core
+Stage: RED
+Depends on: [1.1](1.1-reverse-proxy.md)
+
+- [ ] An SSE response from an upstream reaches the client unbuffered: each `data:` event is readable by the client before the upstream has sent the next one
+- [ ] A chunked response reaches the client chunk by chunk, not accumulated and flushed at completion
+- [ ] Response headers reach the client before the first body byte, not after
+- [ ] `Content-Type: text/event-stream` and the upstream's `Cache-Control` and `Connection` semantics survive the proxy
+- [ ] No response body is written to memory or disk in full before forwarding
+- [ ] The terminating `data: [DONE]` sentinel and the final zero-length chunk pass through
+- [ ] A test asserts wall-clock ordering: the Nth event is observed at the client before the upstream emits the N+1th
+
+The whole product is token streaming. If the gateway buffers, a caller waits minutes
+for a response that should have started in seconds, and the user-visible behaviour of
+the model API regresses versus Kong. nginx in front is already configured with
+`proxy-buffering: off`; the gateway must not reintroduce buffering behind it.
+
+## Verify
+
+```bash
+go test ./internal/proxy/... -run TestSSEUnbuffered -v
+# expected: passes — client observes each of 5 stub-emitted SSE events with the
+# upstream still open, and total observed inter-event gaps match the stub's emit delays
+```

tasks/1.3-disconnect-propagation.md

@@ -0,0 +1,25 @@
+# 1.3 — Client disconnect propagation (RED)
+
+Phase: 1 — Proxy core
+Stage: RED
+Depends on: [1.2](1.2-streaming-passthrough.md)
+
+- [ ] When a client closes the connection mid-response, the gateway cancels the upstream request immediately
+- [ ] The stub upstream observes its request context cancelled, not a request that runs to completion
+- [ ] Cancellation happens within a small bounded delay of the client close, not at the route's read timeout
+- [ ] The same holds for a non-streaming request abandoned before the upstream replies
+- [ ] The disconnect is logged with a reason distinguishing it from an upstream error
+- [ ] No goroutine or upstream connection is left alive after the disconnect — the test asserts this, not just the response
+
+This is load-bearing. The `reasoning` upstream runs 2 replicas at `--max-num-seqs=4`,
+which is 8 concurrent sequence slots cluster-wide. An orphaned generation holds one of
+those 8 until it finishes on its own, which for a 32B model on a Volta GPU can be
+minutes. A handful of abandoned browser tabs can starve the entire cluster.
+
+## Verify
+
+```bash
+go test ./internal/proxy/... -run TestClientDisconnectCancelsUpstream -race -v
+# expected: passes — stub upstream reports context cancellation within 1s of the client
+# closing mid-stream, and the post-test goroutine count returns to baseline
+```

tasks/1.4-per-route-timeouts.md

@@ -0,0 +1,29 @@
+# 1.4 — Per-route timeouts (GREEN)
+
+Phase: 1 — Proxy core
+Stage: GREEN
+Depends on: [0.2](0.2-route-configuration.md), [1.1](1.1-reverse-proxy.md)
+
+- [ ] Connect, read and write timeouts are taken per route from configuration, never from a global default
+- [ ] Chat routes use connect `10s`, read `1h`, write `1h`
+- [ ] Embeddings and rerank routes use connect `10s`, read `10m`, write `10m`
+- [ ] An upstream that never accepts a connection fails at the configured connect timeout, not later
+- [ ] An upstream that accepts then stalls fails at the configured read timeout with a `5xx` and a logged reason
+- [ ] A stream still emitting tokens is never cut by the read timeout — the timeout applies to inactivity, not total duration
+- [ ] No code path shortens a configured proxy timeout to enforce an application-level cap
+
+These are Kong's current values and they are deliberate. The 1-hour read timeout
+exists because a 32B model on a Volta GPU routinely exceeds 60 seconds per request.
+Any shorter application-level cap must be enforced by the gateway's own logic — a
+budget, a slot limit, an explicit max-generation-time — and never by shortening the
+proxy timeout, or long legitimate generations truncate mid-stream and callers see
+corrupted output rather than an error.
+
+## Verify
+
+```bash
+go test ./internal/proxy/... -run 'TestConnectTimeout|TestReadTimeout|TestLongStreamNotTruncated' -v
+# expected: passes — stalled upstream errors at the configured read timeout, a stub
+# emitting one event every 200ms for longer than the timeout window is not cut off,
+# and a blackholed address fails at ~10s
+```

tasks/1.5-header-hygiene.md

@@ -0,0 +1,27 @@
+# 1.5 — Header hygiene (GREEN)
+
+Phase: 1 — Proxy core
+Stage: GREEN
+Depends on: [1.1](1.1-reverse-proxy.md)
+
+- [ ] Hop-by-hop headers are stripped from both the upstream request and the client response
+- [ ] Headers named in a request's `Connection` header are also stripped, not just the fixed hop-by-hop list
+- [ ] `X-Forwarded-For` appends the immediate peer to the nginx-supplied value rather than replacing or fabricating it
+- [ ] `X-Forwarded-Proto` and `X-Forwarded-Host` are taken from the nginx-supplied values when present
+- [ ] Client-supplied `X-Forwarded-*` values are not trusted when the request did not arrive from the trusted ingress peer
+- [ ] End-to-end headers, including `Content-Type`, `Authorization` where the route requires it, and upstream response headers, pass through unchanged
+- [ ] A test asserts the exact header set the stub upstream receives
+
+ingress-nginx owns TLS and the edge, so it is the only source of truth for the
+original scheme, host and client address. The gateway fabricating these would make
+every upstream's view of the caller wrong, and would let a client spoof its own
+source address by sending the header itself.
+
+## Verify
+
+```bash
+go test ./internal/proxy/... -run TestHeaderHygiene -v
+# expected: passes — stub upstream sees no Connection/Keep-Alive/TE/Upgrade/
+# Proxy-Authorization headers, sees X-Forwarded-For ending in the nginx-supplied value
+# plus the peer, and a spoofed X-Forwarded-Proto from an untrusted peer is discarded
+```

tasks/1.6-websocket-upgrade.md

@@ -0,0 +1,27 @@
+# 1.6 — WebSocket upgrade (GREEN)
+
+Phase: 1 — Proxy core
+Stage: GREEN
+Depends on: [1.5](1.5-header-hygiene.md)
+
+- [ ] A route may be configured to allow protocol upgrade
+- [ ] An upgrade request on such a route reaches the upstream with its `Upgrade` and `Connection` headers intact, despite hop-by-hop stripping
+- [ ] The upstream's `101 Switching Protocols` response reaches the client, and bytes then flow bidirectionally
+- [ ] Frames pass in both directions with no buffering delay
+- [ ] Client close propagates to the upstream and upstream close propagates to the client
+- [ ] An upgrade attempt on a route that does not allow it is rejected, not silently downgraded to a plain proxied request
+- [ ] Idle upgraded connections are not cut by the route's read timeout while frames are still flowing
+
+`agent-pod/console` serves a WebSocket and is one of the seven Kong-class Ingresses
+being migrated. It is currently publicly routed and unauthenticated into a
+shell-capable container, so it must work through the gateway before it can be put
+behind gateway auth. Header hygiene and upgrade support interact directly: `Upgrade`
+and `Connection` are hop-by-hop, and a naive strip breaks the handshake.
+
+## Verify
+
+```bash
+go test ./internal/proxy/... -run TestWebSocketUpgrade -v
+# expected: passes — client receives 101 from the stub, an echoed frame round-trips in
+# both directions, and closing the client causes the stub to observe a close
+```

tasks/1.7-body-size-caps.md

@@ -0,0 +1,26 @@
+# 1.7 — Per-route body size caps (RED)
+
+Phase: 1 — Proxy core
+Stage: RED
+Depends on: [0.2](0.2-route-configuration.md), [1.1](1.1-reverse-proxy.md)
+
+- [ ] Each route enforces its own configured maximum request body size
+- [ ] A body over the cap is rejected with `413` and a body the upstream never sees
+- [ ] Rejection happens while reading, not after buffering the whole body into memory
+- [ ] A request with a lying or absent `Content-Length` is still capped by bytes actually read
+- [ ] A body at exactly the cap is accepted and proxied intact
+- [ ] The rejection is logged with a reason distinguishing it from other rejections
+- [ ] No global default cap silently applies to a route that failed to declare one — that is a config error, per 0.2
+
+nginx in front is configured with `proxy-body-size: 0`, meaning it enforces no limit
+at all, so the gateway is the only place a cap exists. Embedding and rerank callers
+can send large batches legitimately, which is why the cap is per route rather than
+one number for the whole surface.
+
+## Verify
+
+```bash
+go test ./internal/proxy/... -run TestBodySizeCap -v
+# expected: passes — a body one byte over the route cap returns 413 and the stub
+# upstream records zero requests; a body exactly at the cap returns the stub's 200
+```

tasks/2.1-model-registry.md

@@ -0,0 +1,35 @@
+# 2.1 — Model registry (GREEN)
+
+Phase: 2 — LLM surface
+Stage: GREEN
+Depends on: [0.2](0.2-route-configuration.md), [1.1](1.1-reverse-proxy.md)
+
+- [ ] A model name -> upstream map is loaded from configuration at startup, never compiled in
+- [ ] Each entry carries at minimum the model name clients send, the upstream address, and the upstream path to use
+- [ ] Two model names may point at the same upstream address, and both resolve independently
+- [ ] A duplicate model name, an empty model name, or an entry with no upstream address fails startup loudly with the offending entry named
+- [ ] The registry is queryable by exact model name; lookup is case-sensitive and does no fuzzy matching or defaulting
+- [ ] The set of known model names is enumerable, because `/v1/models` and unknown-model errors are both derived from it
+
+The five entries verified live on 2026-08-19. Ports are 80, not 8080.
+
+| model name clients send | upstream Service | engine |
+|---|---|---|
+| `reasoning` | `reasoning-predictor.llm-serving:80` | vLLM, DeepSeek-R1-Distill-Qwen-32B |
+| `ornith:35b` | `ornith-predictor.llm-serving:80` | Ollama |
+| `qwen2.5:3b-instruct` | `ornith-predictor.llm-serving:80` | Ollama, same pods |
+| `nomic-ai/nomic-embed-text-v2-moe` | `embeddings-predictor.llm-serving:80` | TEI |
+| `BAAI/bge-reranker-base` | `reranker-predictor.llm-serving:80` | TEI |
+
+`ornith:35b` and `qwen2.5:3b-instruct` share pods and both stay resident, so a
+registry that maps them to one address is correct, not a shortcut.
+
+## Verify
+
+```bash
+# with the five entries configured against local stubs
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/readyz   # expected: 200
+
+# duplicate model name in config must refuse to start
+./gateway --config testdata/duplicate-model.yaml; echo "exit=$?"  # expected: non-zero exit, stderr names the duplicated model
+```

tasks/2.10-anthropic-request-translation.md

@@ -0,0 +1,53 @@
+# 2.10 — Anthropic request translation (GREEN)
+
+Phase: 2 — LLM surfaces
+Stage: GREEN
+Depends on: [2.9](2.9-canonical-request-model.md), [2.1](2.1-model-registry.md)
+
+`POST /llm/v1/messages` accepts an Anthropic Messages request body and turns it into
+the dialect-neutral canonical request. The path carries `/v1/messages` because the
+Anthropic base-URL convention appends that suffix; the gateway prefix is `/llm`.
+
+- [ ] `POST /llm/v1/messages` is accepted and selects its upstream from the body's
+      `model` field, using the same registry as `/v1`: `reasoning` reaches
+      `reasoning-predictor.llm-serving:80`, `ornith:35b` and `qwen2.5:3b-instruct`
+      reach `ornith-predictor.llm-serving:80`
+- [ ] The top-level `system` field becomes the canonical system instruction; it is a
+      distinct field in this dialect and is not a member of `messages`
+- [ ] Each message `content` may be a plain string or an array of blocks; a string and
+      a single text block carrying the same characters translate identically
+- [ ] `max_tokens` is REQUIRED on this surface, matching the Anthropic contract; a
+      request without it is rejected as a client error and no upstream is contacted
+- [ ] `max_tokens` above what the upstream can serve is clamped rather than rejected,
+      and the clamp is logged; `reasoning` has `--max-model-len=16384`
+- [ ] Roles are restricted to `user` and `assistant`; any other role, including
+      `system` inside `messages`, is a client error naming the offending role
+- [ ] `stop_sequences` becomes the canonical stop sequences, and `stream` becomes the
+      canonical streaming flag
+- [ ] `tools`, `tool_choice`, any `tool_use` or `tool_result` block, any `image` block,
+      any cache-control marker, and any user message with more than one content block
+      are rejected as unsupported, naming the feature; none is silently dropped
+- [ ] Unknown top-level fields are rejected rather than ignored, so a client cannot
+      believe an unimplemented option took effect
+- [ ] Reading the body respects the route's configured body size cap
+
+`max_tokens` stays optional on `/v1/chat/completions`. The policy is per-surface: each
+dialect keeps its own contract, and the canonical request records whichever value
+resulted.
+
+## Verify
+
+```bash
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/llm/v1/messages \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":64,"system":"be terse","messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, reasoning stub hit, system text present in the upstream request
+
+curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
+# expected: 400, body names max_tokens as the missing required field, no upstream stub hit
+
+curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":8,"messages":[{"role":"system","content":"x"}]}'
+# expected: 400, body names the rejected role, no upstream stub hit
+```

tasks/2.11-anthropic-response-translation.md

@@ -0,0 +1,50 @@
+# 2.11 — Anthropic non-streaming response translation (GREEN)
+
+Phase: 2 — LLM surfaces
+Stage: GREEN
+Depends on: [2.10](2.10-anthropic-request-translation.md), [2.9](2.9-canonical-request-model.md)
+
+A non-streaming `POST /llm/v1/messages` gets an Anthropic Messages response, built
+from whatever the upstream returned. Upstreams speak the OpenAI chat-completion shape;
+the client on this surface must never see it.
+
+- [ ] The response body is `{"id","type":"message","role":"assistant","content":[...],
+      "model","stop_reason","stop_sequence","usage":{"input_tokens","output_tokens"}}`
+      with `type` literally `message` and `role` literally `assistant`
+- [ ] `model` echoes the model name the client sent, not an upstream-internal name
+- [ ] Upstream `finish_reason` `stop` becomes `stop_reason` `end_turn`, and `length`
+      becomes `max_tokens`
+- [ ] A generation halted by a client-supplied stop sequence reports `stop_reason`
+      `stop_sequence` and puts the matched string in `stop_sequence`; otherwise
+      `stop_sequence` is null and present, not omitted
+- [ ] Upstream `prompt_tokens` becomes `usage.input_tokens` and `completion_tokens`
+      becomes `usage.output_tokens`; no other usage fields are invented
+- [ ] `reasoning_content`, which vLLM returns as a field separate from `content` for
+      `reasoning`, becomes a `thinking` content block that PRECEDES the `text` block
+- [ ] When `reasoning_content` is absent or empty, no `thinking` block is emitted and
+      `content` holds only the `text` block
+- [ ] When `content` is empty but `reasoning_content` is not, the `thinking` block is
+      still returned rather than an empty `content` array
+- [ ] `Content-Type` is `application/json`, and no OpenAI field name such as `choices`,
+      `finish_reason` or `object` appears anywhere in the body
+
+`reasoning` runs DeepSeek-R1-Distill-Qwen-32B under vLLM with
+`--reasoning-parser=deepseek_r1`, which is why the reasoning text arrives as its own
+field and maps cleanly onto a thinking block.
+
+## Verify
+
+```bash
+curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, type=message, role=assistant, content[0].type=thinking, content[1].type=text
+
+curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}' \
+  | grep -c -E '"choices"|"finish_reason"|"object"'
+# expected: 0
+
+curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"qwen2.5:3b-instruct","max_tokens":4,"messages":[{"role":"user","content":"count to fifty"}]}'
+# expected: 200, stop_reason=max_tokens, usage has input_tokens and output_tokens only
+```

tasks/2.12-anthropic-sse-state-machine.md

@@ -0,0 +1,64 @@
+# 2.12 — Anthropic SSE state machine (RED)
+
+Phase: 2 — LLM surfaces
+Stage: RED
+Depends on: [2.11](2.11-anthropic-response-translation.md), [1.2](1.2-streaming-passthrough.md), [1.3](1.3-disconnect-propagation.md)
+
+`POST /llm/v1/messages` with `"stream":true` must emit Anthropic SSE. Anthropic uses
+NAMED events carrying content-block indices; the upstream emits flat OpenAI data-only
+chunks. Write the failing tests against the event sequence before writing a translator.
+
+- [ ] Every frame has both an `event:` line and a `data:` line; a bare `data:` frame is
+      a failure on this surface
+- [ ] `Content-Type` is `text/event-stream`
+- [ ] Event order for a full response is exactly: `message_start`, then for each block
+      `content_block_start`, one or more `content_block_delta`, `content_block_stop`,
+      then `message_delta`, then `message_stop`
+- [ ] `message_start` carries the message envelope with the client-sent model, `role`
+      `assistant`, empty `content`, and `usage.input_tokens`
+- [ ] The block carrying `reasoning_content` is index 0 with block type `thinking`, and
+      its deltas are `thinking_delta`
+- [ ] The block carrying `content` is index 1 with block type `text`, and its deltas
+      are `text_delta`
+- [ ] The end of reasoning is only knowable when `content` first arrives, so the
+      arrival of the first `content` token MUST emit `content_block_stop` for index 0
+      before `content_block_start` for index 1 — the two blocks never overlap
+- [ ] If a response has no `reasoning_content` at all, the text block is index 0 and no
+      thinking block is started; indices are assigned in emission order with no gaps
+- [ ] If a response has `reasoning_content` and never any `content`, the thinking block
+      is still closed before `message_delta`
+- [ ] `message_delta` carries `stop_reason` and `usage.output_tokens`; upstream
+      `finish_reason` `stop` becomes `end_turn` and `length` becomes `max_tokens`
+- [ ] `message_stop` is the final frame and is emitted exactly once per response
+- [ ] Translation is streaming and unbuffered: each upstream chunk is converted and
+      flushed as it arrives, and the response is never accumulated to be inspected
+- [ ] A client disconnect mid-stream cancels the upstream request immediately and
+      releases the slot, rather than orphaning the generation
+- [ ] An upstream failure after `message_start` terminates the stream with an error
+      frame rather than a truncated but apparently successful sequence
+- [ ] The OpenAI `data: [DONE]` sentinel is consumed by the translator and never
+      forwarded to a `/llm` client
+
+An orphaned generation holds one of only eight vLLM sequence slots in the cluster,
+which is why disconnect cancellation is an acceptance criterion here and not only in
+the proxy layer.
+
+## Verify
+
+```bash
+curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":64,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
+  | grep '^event:'
+# expected: message_start, content_block_start, content_block_delta..., content_block_stop,
+#           content_block_start, content_block_delta..., content_block_stop, message_delta, message_stop
+
+curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":64,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
+  | grep -n -E 'content_block_stop|"index":1' | head -3
+# expected: the index 0 content_block_stop line precedes the first line mentioning index 1
+
+timeout 1 curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":512,"stream":true,"messages":[{"role":"user","content":"long"}]}' >/dev/null
+grep -c 'cancelled' /tmp/stub-reasoning.log
+# expected: 1 within a second of the client going away
+```

tasks/2.13-anthropic-error-shape.md

@@ -0,0 +1,50 @@
+# 2.13 — Anthropic error shape on `/llm/*` (RED)
+
+Phase: 2 — LLM surfaces
+Stage: RED
+Depends on: [2.10](2.10-anthropic-request-translation.md), [4.3](4.3-problem-json-errors.md)
+
+`/v1/*` renders rejections as RFC 9457 `application/problem+json`. `/llm/*` must not.
+The same underlying rejection gets two renderings, chosen purely by which surface
+received the request. Write the failing tests against both renderings first.
+
+- [ ] Every error response from a `/llm/*` path has the body
+      `{"type":"error","error":{"type":"...","message":"..."}}` and
+      `Content-Type: application/json`
+- [ ] No `/llm/*` response ever carries `application/problem+json`, and no `/v1/*`
+      response ever carries the Anthropic error shape
+- [ ] An unknown or missing `model` returns `invalid_request_error` with a message
+      naming the rejected value and listing the configured model names, derived from
+      the registry rather than hardcoded
+- [ ] A request missing the required `max_tokens` returns `invalid_request_error`
+      naming `max_tokens`
+- [ ] A body that is not valid JSON returns `invalid_request_error` with a message
+      distinguishable from the unknown-model case
+- [ ] A request for an out-of-scope feature returns `invalid_request_error` naming the
+      feature, for example tools, images or prompt caching
+- [ ] Exhausting the shared slot queue returns HTTP 429 with error type
+      `rate_limit_error` and a `Retry-After` header
+- [ ] An upstream failure or timeout returns HTTP 5xx with error type `api_error`, and
+      the message leaks no upstream host, port or internal path
+- [ ] No rejection reaches an upstream, and none falls back to a default model
+- [ ] Each rejection is logged with its reason and its surface; the request body is
+      never logged
+- [ ] The same malformed request sent to `/v1/chat/completions` and `/llm/v1/messages`
+      yields the same HTTP status with the two different body shapes
+
+## Verify
+
+```bash
+curl -s -i localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"gpt-4","max_tokens":8,"messages":[]}'
+# expected: 4xx, content-type application/json, body {"type":"error","error":{"type":"invalid_request_error",...}}
+#           listing reasoning, ornith:35b, qwen2.5:3b-instruct
+
+curl -s -i localhost:8080/llm/v1/messages -H 'content-type: application/json' -d 'not json' \
+  | grep -i 'content-type'
+# expected: application/json, never application/problem+json
+
+curl -s -i localhost:8080/v1/chat/completions -H 'content-type: application/json' \
+  -d '{"model":"gpt-4","messages":[]}' | grep -i 'content-type'
+# expected: application/problem+json, confirming the two surfaces render differently
+```

tasks/2.14-queue-position-event.md

@@ -0,0 +1,52 @@
+# 2.14 — Queue position event on `/llm/*` (GREEN)
+
+Phase: 2 — LLM surfaces
+Stage: GREEN
+Depends on: [2.12](2.12-anthropic-sse-state-machine.md), [4.1](4.1-gpu-slot-semaphore.md)
+
+The Anthropic event set has no way to say "you are queued". Its stream implicitly
+begins after a slot has been acquired, so a queued client sees nothing at all until
+generation starts. The `reasoning` upstream has only 8 sequence slots cluster-wide and
+the gateway caps below that, so waiting is normal and worth showing.
+
+- [ ] When a streaming `/llm/v1/messages` request waits for a slot, the gateway emits a
+      frame with `event: queue` before any `message_start`
+- [ ] The queue frame's data carries the caller's current position in the queue
+- [ ] Position updates are emitted as the queue drains, each as another `event: queue`
+      frame, until a slot is acquired
+- [ ] Once a slot is acquired the stream continues with the standard sequence beginning
+      at `message_start`, and no further queue frame is emitted for that request
+- [ ] A request that acquires a slot immediately emits no queue frame at all
+- [ ] Queue frames are flushed as they are produced, not buffered behind the first
+      upstream token
+- [ ] A client that disconnects while still queued is removed from the queue, never
+      reaches the upstream and never consumes a slot
+- [ ] Non-streaming requests emit no queue frames; they simply wait, then answer
+- [ ] The extension is documented in the surface's own docs as non-standard, alongside
+      the fact that a strict Anthropic client ignoring unknown events degrades to
+      showing nothing while queued rather than erroring
+
+This is a deliberate departure from the Anthropic contract. It is safe only because
+the sole client of `/llm/*` is the first-party riotpiao frontend. It must never be
+required for correctness: dropping every `event: queue` frame leaves a valid,
+complete Anthropic stream.
+
+## Verify
+
+```bash
+# Fill the slots first, with cap=2 configured against a stub that holds each request 3s.
+seq 4 | xargs -P4 -I{} curl -s -o /dev/null -X POST localhost:8080/llm/v1/messages \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' &
+
+sleep 1
+curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
+  | grep -m3 '^event:'
+# expected: event: queue arrives within a second, before any event: message_start
+
+curl -N -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"qwen2.5:3b-instruct","max_tokens":16,"stream":true,"messages":[{"role":"user","content":"hi"}]}' \
+  | head -1
+# expected: event: message_start, no queue frame on an uncontended upstream
+```

tasks/2.15-dialect-scope-boundary.md

@@ -0,0 +1,51 @@
+# 2.15 — Anthropic dialect scope boundary (GREEN)
+
+Phase: 2 — LLM surfaces
+Stage: GREEN
+Depends on: [2.10](2.10-anthropic-request-translation.md), [2.13](2.13-anthropic-error-shape.md)
+
+The only client of `/llm/*` is the first-party riotpiao frontend. It is not Claude
+Code and not the Anthropic SDK, so the surface implements a deliberately narrow slice
+of the Messages API. The narrowness is the design; the risk is a half-built feature
+that appears to work.
+
+- [ ] The unsupported set is enumerated in one place and covers at least: `tools` and
+      `tool_choice`, `tool_use` blocks, `tool_result` turns, image content blocks,
+      prompt-caching controls and cache headers, the batch API, and user messages
+      carrying more than one content block
+- [ ] Each unsupported feature is detected during request translation, before any
+      upstream is contacted
+- [ ] Rejection uses the Anthropic error shape
+      `{"type":"error","error":{"type":"invalid_request_error","message":"..."}}` and
+      the message names the specific unsupported feature, not just "unsupported"
+- [ ] No unsupported feature is silently ignored, stripped, or partially honoured; a
+      request containing one never produces a 200
+- [ ] A request combining a supported and an unsupported field is rejected, not
+      serviced with the unsupported part dropped
+- [ ] The `/v1/*` OpenAI surface is unaffected: tool calling continues to work there
+      exactly as it does today
+- [ ] Every entry in the unsupported set has a test asserting the rejection, so
+      widening scope forces a deliberate test change rather than a quiet code change
+- [ ] The list is documented on the surface, so the frontend can see the boundary
+      without reading gateway code
+
+Enabling any of these later must be an explicit config or code change accompanied by
+its own translation work and tests. A silent partial implementation is the specific
+failure this task exists to prevent.
+
+## Verify
+
+```bash
+curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":8,"tools":[{"name":"x"}],"messages":[{"role":"user","content":"hi"}]}'
+# expected: 4xx, error.message names tools, no upstream stub hit
+
+curl -s localhost:8080/llm/v1/messages -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":8,"messages":[{"role":"user","content":[{"type":"image","source":{}}]}]}'
+# expected: 4xx, error.message names image content blocks, no upstream stub hit
+
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}],"tools":[{"type":"function","function":{"name":"x"}}]}'
+# expected: 200, tool calling still works on the OpenAI surface
+```

tasks/2.2-body-based-dispatch.md

@@ -0,0 +1,39 @@
+# 2.2 — Body-based model dispatch (GREEN)
+
+Phase: 2 — LLM surface
+Stage: GREEN
+Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md), [1.2](1.2-streaming-passthrough.md)
+
+- [ ] `POST /v1/chat/completions` selects its upstream from the `model` field of the JSON request body
+- [ ] The body reaching the upstream is byte-identical to the body received, `model` included — dispatch reads, it does not rewrite
+- [ ] The upstream sees the canonical path `/v1/chat/completions`
+- [ ] `"model":"reasoning"` reaches `reasoning-predictor.llm-serving:80`; `"model":"ornith:35b"` and `"model":"qwen2.5:3b-instruct"` both reach `ornith-predictor.llm-serving:80`
+- [ ] `"stream":true` streams unbuffered — chunks reach the client as the upstream emits them, and are not accumulated in order to inspect the body
+- [ ] A client disconnect mid-stream cancels the upstream request rather than orphaning it
+- [ ] `reasoning_content` is passed through untouched alongside `content`; the gateway does not merge, reorder or strip either
+- [ ] Reading the body to find `model` respects the route's body size cap and does not load an unbounded request into memory
+
+This is the single capability Kong OSS lacked — `ai-proxy-advanced` is Enterprise-only —
+and the entire reason this gateway exists. Everything else in this phase is a
+consequence of it. Ports are 80, not 8080.
+
+An orphaned generation holds one of only eight vLLM sequence slots in the cluster,
+which is why disconnect cancellation belongs in the acceptance criteria of this task
+and not only in the proxy layer.
+
+## Verify
+
+```bash
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, stub for reasoning-predictor recorded the hit with path /v1/chat/completions and an unmodified body
+
+curl -s localhost:8080/v1/chat/completions -H 'content-type: application/json' \
+  -d '{"model":"qwen2.5:3b-instruct","messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, ornith-predictor stub hit, reasoning-predictor stub not hit
+
+curl -N -s localhost:8080/v1/chat/completions -H 'content-type: application/json' \
+  -d '{"model":"reasoning","stream":true,"messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, content-type text/event-stream, first data: chunk arrives before the stub finishes emitting
+```

tasks/2.3-unknown-model-errors.md

@@ -0,0 +1,33 @@
+# 2.3 — Unknown and missing model errors (RED)
+
+Phase: 2 — LLM surface
+Stage: RED
+Depends on: [2.1](2.1-model-registry.md), [2.2](2.2-body-based-dispatch.md)
+
+- [ ] `POST /v1/chat/completions` with a `model` that is not in the configured registry returns a 4xx client error, never 5xx
+- [ ] A request with no `model` field, a null `model`, or an empty-string `model` is the same class of client error
+- [ ] A body that is not valid JSON is also a client error, distinguishable from an unknown model
+- [ ] The response is RFC 9457 with `Content-Type: application/problem+json`
+- [ ] The problem body names the rejected value and enumerates every currently configured model name, derived from the registry rather than written out by hand
+- [ ] No fallback to a default model happens under any of these conditions, and no upstream is contacted
+- [ ] The rejection is logged with its reason; the request body is not logged
+
+Write the failing tests first. A silent fallback to a default model is the specific
+failure mode this task exists to prevent: it turns a client typo into a bill against
+the wrong GPU and hides the mistake from the caller.
+
+## Verify
+
+```bash
+curl -s -i localhost:8080/v1/chat/completions -H 'content-type: application/json' \
+  -d '{"model":"gpt-4","messages":[]}'
+# expected: 400 (or 404), content-type: application/problem+json, body lists reasoning, ornith:35b, qwen2.5:3b-instruct, and the embedding/rerank models
+
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' -d '{"messages":[]}'
+# expected: 4xx, no upstream stub recorded any hit
+
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' -d 'not json'
+# expected: 400, problem+json, reason distinct from unknown-model
+```

tasks/2.4-legacy-path-aliases.md

@@ -0,0 +1,30 @@
+# 2.4 — Legacy path aliases (GREEN)
+
+Phase: 2 — LLM surface
+Stage: GREEN
+Depends on: [2.2](2.2-body-based-dispatch.md)
+
+- [ ] `POST /v1/reasoning/chat/completions` behaves as the canonical endpoint with `model` forced to `reasoning`
+- [ ] `POST /v1/ornith/chat/completions` behaves as the canonical endpoint with `model` forced to `ornith:35b`
+- [ ] `POST /v1/qwen/chat/completions` behaves as the canonical endpoint with `model` forced to `qwen2.5:3b-instruct`
+- [ ] The forced value overrides whatever `model` the body carries, including a conflicting one, and the upstream receives the forced value
+- [ ] The upstream sees the canonical path `/v1/chat/completions`, not the alias path
+- [ ] Streaming, disconnect cancellation and `reasoning_content` passthrough behave identically to the canonical endpoint
+- [ ] The alias set is configuration, so removal is a config change and needs no code change
+- [ ] Each alias is marked temporary where it is defined, with the condition for removal stated: all callers migrated
+
+These three paths exist only because Kong OSS could not dispatch on the request body.
+They keep the cutover reversible — pi is a live caller today. They are deleted once
+callers have moved to `POST /v1/chat/completions` with `model` in the body.
+
+## Verify
+
+```bash
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/reasoning/chat/completions \
+  -H 'content-type: application/json' -d '{"messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, reasoning-predictor stub hit at /v1/chat/completions with body model=reasoning
+
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/qwen/chat/completions \
+  -H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}'
+# expected: 200, ornith-predictor stub hit with body model=qwen2.5:3b-instruct — the body's "reasoning" is overridden
+```

tasks/2.5-models-endpoint.md

@@ -0,0 +1,33 @@
+# 2.5 — `GET /v1/models` (GREEN)
+
+Phase: 2 — LLM surface
+Stage: GREEN
+Depends on: [2.1](2.1-model-registry.md)
+
+- [ ] `GET /v1/models` returns 200 with `Content-Type: application/json`
+- [ ] The response shape is `{"object":"list","data":[{"id","object":"model","owned_by","created"}]}`
+- [ ] Every entry's `object` is the literal string `model`
+- [ ] The `id` values are exactly the model names the configured registry will accept for dispatch — no more, no fewer
+- [ ] Adding or removing a model in configuration changes this response with no code change
+- [ ] No model list is hardcoded anywhere; the list cannot disagree with what routing accepts
+- [ ] The endpoint contacts no upstream and stays cheap
+
+Kong served a static list via `request-termination`, and its own manifest flags that
+the list can drift from what the engines actually serve. Deriving from the registry
+makes that drift structurally impossible: the same source answers this endpoint and
+decides which `model` values dispatch.
+
+## Verify
+
+```bash
+curl -s localhost:8080/v1/models
+# expected: 200, {"object":"list","data":[...]} with ids reasoning, ornith:35b, qwen2.5:3b-instruct,
+#           nomic-ai/nomic-embed-text-v2-moe, BAAI/bge-reranker-base
+
+# every advertised id must dispatch; nothing advertised may 404
+for m in $(curl -s localhost:8080/v1/models | grep -o '"id":"[^"]*"' | cut -d'"' -f4); do
+  curl -s -o /dev/null -w "$m %{http_code}\n" localhost:8080/v1/chat/completions \
+    -H 'content-type: application/json' -d "{\"model\":\"$m\",\"messages\":[]}"
+done
+# expected: no unknown-model rejection for any advertised id
+```

tasks/2.6-embeddings-passthrough.md

@@ -0,0 +1,25 @@
+# 2.6 — `POST /v1/embeddings` passthrough (GREEN)
+
+Phase: 2 — LLM surface
+Stage: GREEN
+Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md)
+
+- [ ] `POST /v1/embeddings` reaches `embeddings-predictor.llm-serving:80` at the path `/v1/embeddings`, unrewritten
+- [ ] The request body is forwarded byte-identical, and the upstream response body is returned byte-identical
+- [ ] The route's connect timeout is 10s and its read and write timeouts are 10m, explicit in configuration rather than inherited
+- [ ] The route has an explicit body size cap
+- [ ] Upstream error statuses are surfaced as-is; the gateway invents no retries and no substitute response
+
+TEI already serves the canonical path, so this route rewrites nothing. The model
+served here is `nomic-ai/nomic-embed-text-v2-moe`. Port is 80, not 8080. Whether this
+route dispatches on the body's `model` or is pinned to the single embeddings upstream
+is a design decision for whoever works it; either way the path must not change and an
+unknown `model` must not silently reach the wrong upstream.
+
+## Verify
+
+```bash
+curl -s -i localhost:8080/v1/embeddings -H 'content-type: application/json' \
+  -d '{"model":"nomic-ai/nomic-embed-text-v2-moe","input":"hello"}'
+# expected: 200, embeddings stub hit at path /v1/embeddings, request body unmodified, response body byte-identical to the stub's
+```

tasks/2.7-rerank-rewrite.md

@@ -0,0 +1,25 @@
+# 2.7 — `POST /v1/rerank` path rewrite (GREEN)
+
+Phase: 2 — LLM surface
+Stage: GREEN
+Depends on: [2.1](2.1-model-registry.md), [1.1](1.1-reverse-proxy.md)
+
+- [ ] `POST /v1/rerank` reaches `reranker-predictor.llm-serving:80` at the path `/rerank`
+- [ ] The rewrite is expressed in the route configuration, not special-cased in dispatch code
+- [ ] The request body is forwarded byte-identical, and the upstream response body is returned byte-identical
+- [ ] The route's connect timeout is 10s and its read and write timeouts are 10m, explicit in configuration
+- [ ] The route has an explicit body size cap
+- [ ] A request to `/v1/rerank` never reaches the upstream as `/v1/rerank`
+
+TEI does not serve `/v1/rerank`: probing it returned 404, while `/rerank` returned 405
+for the wrong method — that is how the correct upstream path was established. The
+model served here is `BAAI/bge-reranker-base`. Port is 80, not 8080. This is the only
+route in the LLM surface that rewrites its path.
+
+## Verify
+
+```bash
+curl -s -i localhost:8080/v1/rerank -H 'content-type: application/json' \
+  -d '{"model":"BAAI/bge-reranker-base","query":"q","texts":["a","b"]}'
+# expected: 200, reranker stub recorded exactly one hit at path /rerank and zero at /v1/rerank
+```

tasks/2.8-kong-parity-test.md

@@ -0,0 +1,31 @@
+# 2.8 — Kong parity test (RED)
+
+Phase: 2 — LLM surface
+Stage: RED
+Depends on: [2.2](2.2-body-based-dispatch.md), [2.4](2.4-legacy-path-aliases.md), [2.5](2.5-models-endpoint.md), [2.6](2.6-embeddings-passthrough.md), [2.7](2.7-rerank-rewrite.md)
+
+- [ ] A repeatable check issues the same request to Kong and to the gateway and compares the real HTTP responses: status, meaningful headers, and body
+- [ ] It covers every migrated route: `GET /v1/models`, the three `/v1/{reasoning,ornith,qwen}/chat/completions` aliases, `POST /v1/embeddings`, `POST /v1/rerank`, and the new canonical `POST /v1/chat/completions`
+- [ ] It covers a streaming chat request and asserts equivalent chunk sequencing, not just an equal final concatenation
+- [ ] It covers a mid-stream client disconnect and asserts the upstream request is cancelled rather than left running
+- [ ] It asserts `reasoning_content` and `content` are both present and untouched for the `reasoning` model
+- [ ] Fields that legitimately differ per request — ids, timestamps, generated text — are normalised before comparison, and the normalisation is explicit rather than a blanket ignore
+- [ ] Any difference fails the check loudly and names the route and the field
+- [ ] `GET /v1/models` is expected to differ from Kong's static list only where the gateway's derived list is more accurate; that difference is recorded deliberately, not normalised away
+- [ ] The check passes before cutover ([6.5](6.5-cutover.md)) and blocks it if it does not
+
+Kong is serving live traffic while this runs. The check reads only; it changes no
+cluster state. Eleven ReplicaSets exist on the Kong Deployment, so re-run this
+immediately before cutover rather than trusting an old result.
+
+## Verify
+
+```bash
+./parity --kong https://api.riotpiao.com --gateway http://homelab-frontend.api.svc.cluster.local
+# expected: exit 0, one PASS line per route, explicit report of the /v1/models difference
+
+# spot-check by hand, same body to both
+curl -s -o /dev/null -w 'kong %{http_code}\n' https://api.riotpiao.com/v1/rerank \
+  -H 'content-type: application/json' -d '{"model":"BAAI/bge-reranker-base","query":"q","texts":["a"]}'
+# expected: identical status from both endpoints
+```

tasks/2.9-canonical-request-model.md

@@ -0,0 +1,48 @@
+# 2.9 — Dialect-neutral canonical request (GREEN)
+
+Phase: 2 — LLM surfaces
+Stage: GREEN
+Depends on: [2.1](2.1-model-registry.md), [2.2](2.2-body-based-dispatch.md)
+
+The gateway serves two permanent protocol dialects: `/v1/*` is OpenAI-compatible and
+`/llm/*` is Anthropic Messages. Both translate into one internal shape before anything
+else happens to them, and both translate back out of it on the way to the client.
+
+- [ ] A single internal request representation exists that carries at minimum: the
+      resolved upstream, the model name as the client sent it, the ordered turns, an
+      optional system instruction, a maximum output token count, stop sequences, and a
+      streaming flag
+- [ ] The representation names no dialect: nothing in it is called openai or anthropic,
+      and no field exists solely because one dialect happens to spell it that way
+- [ ] Model dispatch, the shared slot controller, per-caller budgets, logging and
+      metrics all read the canonical request and never the raw dialect body
+- [ ] The surface that received a request is recorded as one field on the canonical
+      request, used only to choose the response and error rendering, never to choose an
+      upstream or a slot
+- [ ] An identical prompt sent to `/v1/chat/completions` and to `/llm/v1/messages`
+      produces the same resolved upstream, the same slot accounting and the same log
+      fields apart from that one surface label
+- [ ] Adding a third dialect requires a new translator only; the slot controller,
+      dispatch and budget code are untouched
+- [ ] Translation failure is a client error at the surface boundary, and no partially
+      populated canonical request ever reaches an upstream
+
+`reasoning` maps to `reasoning-predictor.llm-serving:80`; `ornith:35b` and
+`qwen2.5:3b-instruct` both map to `ornith-predictor.llm-serving:80`. Ports are 80.
+
+## Verify
+
+```bash
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, reasoning stub hit once
+
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/llm/v1/messages \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}'
+# expected: 200, reasoning stub hit once, same upstream and same slot counter as the /v1 call
+
+grep -h 'upstream=' /tmp/gateway.log | tail -2
+# expected: both lines show upstream=reasoning-predictor and model=reasoning, differing only in the surface label
+```

tasks/3.1-jwks-fetch-and-rotation.md

@@ -0,0 +1,34 @@
+# 3.1 — JWKS fetch and rotation (GREEN)
+
+Phase: 3 — Authentication
+Stage: GREEN
+Depends on: [0.2](0.2-route-configuration.md), [0.3](0.3-health-endpoints.md)
+
+- [ ] The signing key set is fetched from Authentik at `https://authentik.riotpiao.com` at runtime; no public key is pinned in configuration, in an image, or in a manifest
+- [ ] The JWKS URL is configuration, so a local stub issuer can be pointed at instead — no cluster and no credentials needed to verify this task
+- [ ] Fetched keys are cached and reused; a token verification does not trigger a network call per request
+- [ ] A token whose key id is not in the cache triggers a refetch, and the refetch is rate-limited so an unknown-key flood cannot hammer Authentik
+- [ ] After a key rotates at the issuer, tokens signed by the new key verify without restarting, redeploying, or editing configuration
+- [ ] Tokens signed by a key that is no longer published stop verifying once the cache reflects that
+- [ ] `GET /readyz` fails while JWKS has never been fetched successfully, and `GET /healthz` is unaffected
+- [ ] A JWKS fetch failure while a valid cache exists does not take the gateway down; it is logged and retried
+- [ ] No key material, token, or fetched secret appears in logs
+
+This deletes the rotation runbook `AUTH-PLAN.md` was forced to propose. That runbook
+existed only because Kong OSS needed a pinned `rsa_public_key`; it must not be
+carried forward.
+
+## Verify
+
+```bash
+# stub issuer serving a JWKS, gateway pointed at it
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/readyz    # expected: 200 once JWKS has been fetched
+
+# stub returns 500 for JWKS from a cold start
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/readyz    # expected: 503
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/healthz   # expected: 200
+
+# rotate the stub's key, then present a token signed by the new key, no restart
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models -H "authorization: Bearer $NEW_TOKEN"
+# expected: 200 — refetch happened on the unknown key id
+```

tasks/3.2-bearer-validation.md

@@ -0,0 +1,37 @@
+# 3.2 — Bearer token validation (GREEN)
+
+Phase: 3 — Authentication
+Stage: GREEN
+Depends on: [3.1](3.1-jwks-fetch-and-rotation.md)
+
+- [ ] `Authorization: Bearer <jwt>` is the credential the gateway accepts on protected routes
+- [ ] The scheme match is case-insensitive, as the HTTP spec requires — `bearer`, `Bearer` and `BEARER` all work
+- [ ] A valid, unexpired token signed by a currently published Authentik key returns the upstream response
+- [ ] Missing header, wrong scheme, malformed token, bad signature, expired token, and wrong issuer or audience each return 401 with `WWW-Authenticate` set
+- [ ] Rejections are RFC 9457 `application/problem+json`, and the reason is logged without logging the token
+- [ ] The `Authorization` header is not forwarded to upstreams
+- [ ] The validated caller identity is available to later stages, since [3.5](3.5-capability-authorization.md) authorizes on it
+- [ ] Signature verification is real: a token with a valid-looking payload and a forged signature is rejected
+
+This is precisely what Kong OSS `key-auth` could not do. Verified live: a raw
+`apikey:` header succeeded with 200 while `Authorization: Bearer` failed with 401,
+which hard-blocked every OpenAI-compatible client and is why authentication is OFF
+on the model API today.
+
+Authentik is at `https://authentik.riotpiao.com`. A local stub issuer must be enough
+to work this task — no cluster, no credentials.
+
+## Verify
+
+```bash
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
+  -H "authorization: Bearer $VALID_TOKEN" -H 'content-type: application/json' \
+  -d '{"model":"reasoning","messages":[]}'
+# expected: 200
+
+curl -s -i localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{"model":"reasoning"}'
+# expected: 401, WWW-Authenticate present, content-type application/problem+json
+
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models -H "apikey: $OLD_KONG_KEY"
+# expected: 401 — the retired Kong credential form is not accepted
+```

tasks/3.3-authentik-service-account.md

@@ -0,0 +1,30 @@
+# 3.3 — Authentik service account and token provider (GREEN)
+
+Phase: 3 — Authentication
+Stage: GREEN
+Depends on: [3.2](3.2-bearer-validation.md)
+
+- [ ] A service account exists in Authentik at `https://authentik.riotpiao.com` for machine callers of the model API
+- [ ] An OAuth2 provider is configured so that account can obtain a token non-interactively, with no browser step
+- [ ] The token endpoint returns a signed JWT whose issuer and audience match what the gateway validates, and which the gateway accepts on a protected route
+- [ ] Token lifetime is set deliberately and recorded, not left at whatever the default is
+- [ ] The client secret is stored as a Kubernetes Secret referenced from git, never committed in plaintext
+- [ ] The Authentik configuration is captured in the repo as reproducible steps or declarative config, so it can be rebuilt rather than clicked together again
+- [ ] The token carries whatever claim [3.5](3.5-capability-authorization.md) authorizes on
+
+Open risk recorded in `AUTH-PLAN.md` and unresolved: Authentik 2026.x may require an
+app-password or JWT-assertion flow rather than a plain `client_secret` POST to the
+token endpoint. Verify which flow the running version accepts before wiring anything
+that depends on it, and record the answer here.
+
+## Verify
+
+```bash
+curl -s -i -X POST https://authentik.riotpiao.com/application/o/token/ \
+  -d grant_type=client_credentials -d "client_id=$CID" -d "client_secret=$CSEC"
+# expected: 200, JSON with access_token; if 400/401, the app-password / JWT-assertion risk above is real — record which
+
+curl -s -o /dev/null -w '%{http_code}\n' http://homelab-frontend.api.svc.cluster.local/v1/models \
+  -H "authorization: Bearer $ACCESS_TOKEN"
+# expected: 200 — the gateway accepts a real Authentik-issued token, not only a stub one
+```

tasks/3.4-flag-gated-rollout.md

@@ -0,0 +1,33 @@
+# 3.4 — Flag-gated auth rollout (GREEN)
+
+Phase: 3 — Authentication
+Stage: GREEN
+Depends on: [3.2](3.2-bearer-validation.md)
+
+- [ ] Authentication is controlled by an explicit flag in configuration, and its default is OFF
+- [ ] With the flag off, every route answers exactly as it did before Phase 3 existed — no 401s, no `WWW-Authenticate`, no behaviour change
+- [ ] With the flag on, protected routes require `Authorization: Bearer` and reject anything else with 401
+- [ ] `GET /healthz` and `GET /readyz` never require authentication in either state
+- [ ] Which routes are protected is per-route configuration, so auth can be turned on for one surface at a time
+- [ ] Turning the flag on is a git change synced by Argo; it is never toggled by hand against the cluster
+- [ ] The flag's current state is visible in logs at startup and in metrics, so nobody has to guess whether auth is on
+- [ ] Turning the flag off again fully restores unauthenticated access, making the rollout reversible
+
+The model API is unauthenticated today — verified live, a request with no credentials
+returns 200. Enabling this flag breaks every current caller until they hold a token,
+pi included. That is why it defaults off and is enabled deliberately, after
+[3.6](3.6-pi-client-migration.md).
+
+## Verify
+
+```bash
+# flag off
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/models              # expected: 200
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' # expected: 200
+
+# flag on, no credentials
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' # expected: 401
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/healthz                # expected: 200
+```

tasks/3.5-capability-authorization.md

@@ -0,0 +1,33 @@
+# 3.5 — Capability authorization (RED)
+
+Phase: 3 — Authentication
+Stage: RED
+Depends on: [3.2](3.2-bearer-validation.md), [3.4](3.4-flag-gated-rollout.md)
+
+- [ ] Beyond proving who the caller is, the gateway checks the token is permitted to invoke the capability being called
+- [ ] Each capability prefix — `/v1/*` for the model surface, and the future `/sqs/*`, `/workflow/*`, `/cluster/*`, `/db/*` — maps to a required grant, and the mapping is configuration
+- [ ] A token carrying a queue grant but no model grant is rejected on `POST /v1/chat/completions` with 403, not 401 — it authenticated fine, it is not permitted
+- [ ] A token with no recognised grant at all is rejected on every protected route
+- [ ] Rejections are RFC 9457 `application/problem+json` and state which capability was denied, without echoing the token or its claims verbatim
+- [ ] Denials are logged with the caller identity and the capability, and counted as a distinct rejection reason
+- [ ] A route with no required grant configured while auth is on fails startup rather than defaulting to allow-all
+
+Write the failing tests first: a queue token must not invoke a GPU. The default on an
+unconfigured route is deny, because a fail-open authorization layer is worse than none
+— it reads as protection while granting everything.
+
+## Verify
+
+```bash
+curl -s -i localhost:8080/v1/chat/completions -H "authorization: Bearer $QUEUE_ONLY_TOKEN" \
+  -H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}'
+# expected: 403, application/problem+json naming the denied capability, no upstream stub hit
+
+curl -s -o /dev/null -w '%{http_code}\n' localhost:8080/v1/chat/completions \
+  -H "authorization: Bearer $MODEL_TOKEN" -H 'content-type: application/json' \
+  -d '{"model":"reasoning","messages":[]}'
+# expected: 200
+
+./gateway --config testdata/auth-on-route-without-grant.yaml; echo "exit=$?"
+# expected: non-zero exit, stderr names the route missing a required grant
+```

tasks/3.6-pi-client-migration.md

@@ -0,0 +1,37 @@
+# 3.6 — Migrate pi to Bearer and one provider (REFACTOR)
+
+Phase: 3 — Authentication
+Stage: REFACTOR
+Depends on: [2.2](2.2-body-based-dispatch.md), [3.3](3.3-authentik-service-account.md), [3.4](3.4-flag-gated-rollout.md)
+
+- [ ] `~/.pi/agent/models.json` no longer contains a `customHeaders: {apikey: ...}` block anywhere
+- [ ] pi authenticates with `Authorization: Bearer` carrying an Authentik-issued token
+- [ ] The three provider entries `homelab-reasoning`, `homelab-ornith` and `homelab-qwen` collapse into ONE provider entry
+- [ ] That single provider's base URL is the canonical OpenAI base, and the three models are listed under it — body-based dispatch makes per-model base URLs unnecessary
+- [ ] pi reaches all three models through `POST /v1/chat/completions` with `model` set to `reasoning`, `ornith:35b`, and `qwen2.5:3b-instruct`
+- [ ] Streaming still works from pi, and interrupting a generation cancels it upstream rather than leaving it running
+- [ ] The old config is captured before editing, so a revert is one file restore
+- [ ] Verified with auth ON, because that is the state the migration exists to survive
+
+The `apikey` header exists only because Kong OSS `key-auth` rejected
+`Authorization: Bearer`. Once this lands, nothing depends on the legacy aliases in
+[2.4](2.4-legacy-path-aliases.md) and they can be removed.
+
+## Verify
+
+```bash
+grep -c apikey ~/.pi/agent/models.json     # expected: 0
+grep -c '"homelab' ~/.pi/agent/models.json # expected: 1 provider entry, not 3
+
+for m in reasoning ornith:35b qwen2.5:3b-instruct; do
+  curl -s -o /dev/null -w "$m %{http_code}\n" https://api.riotpiao.com/v1/chat/completions \
+    -H "authorization: Bearer $ACCESS_TOKEN" -H 'content-type: application/json' \
+    -d "{\"model\":\"$m\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}"
+done
+# expected: 200 for all three, with auth enabled
+
+curl -N -s https://api.riotpiao.com/v1/chat/completions -H "authorization: Bearer $ACCESS_TOKEN" \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","stream":true,"messages":[{"role":"user","content":"hi"}]}'
+# expected: text/event-stream chunks arriving incrementally; reasoning_content and content both present
+```

tasks/4.1-gpu-slot-semaphore.md

@@ -0,0 +1,59 @@
+# 4.1 — GPU slot semaphore for `reasoning` (GREEN)
+
+Phase: 4 — Limits and budgets
+Stage: GREEN
+Depends on: [4.3](4.3-problem-json-errors.md), [2.9](2.9-canonical-request-model.md)
+
+The `reasoning` upstream is 2 replicas x `--max-num-seqs=4` = 8 concurrent sequence
+slots cluster-wide. That 8 is a vLLM concurrency setting, not a GPU count — worker-1
+has 4 physical GPUs and the two numbers are unrelated. Exceeding 8 does not fail
+fast; it silently queues inside vLLM where the gateway has no visibility and no
+ability to cancel.
+
+The gateway serves two permanent dialects over the same predictors: `/v1/*` is
+OpenAI-compatible and `/llm/*` is Anthropic Messages. There is ONE controller, keyed by
+upstream, sitting below both.
+
+- [ ] Concurrent in-flight requests to `reasoning` are capped at a configured limit strictly below 8, leaving operator headroom
+- [ ] The cap, the queue depth, and the queue wait timeout are all explicit in configuration with no silent defaults
+- [ ] Requests over the cap wait in a bounded queue rather than being rejected immediately
+- [ ] Once the queue is full, further requests are rejected with a retryable status and a `Retry-After`, as an `application/problem+json` document
+- [ ] A slot is released when the response completes, when the upstream errors, and when the client disconnects mid-stream — no path leaks a slot
+- [ ] A client that disconnects while still queued never reaches the upstream and never consumes a slot
+- [ ] The cap applies only to `reasoning`; other upstreams are unaffected and are not blocked behind its queue
+- [ ] The controller is keyed by UPSTREAM, never by route, path prefix or dialect — `reasoning-predictor.llm-serving:80` has exactly one cap and one queue
+- [ ] A request arriving on `/v1/chat/completions` and one arriving on `/llm/v1/messages` contend for the same slots and the same queue, admitted in arrival order
+- [ ] Total in-flight requests against `reasoning` never exceed the configured cap regardless of which surface they arrived through, including when both surfaces are saturated at once
+- [ ] Per-dialect semaphores are explicitly wrong and must not exist: the 8 sequence slots are physical, so two independent gates would each believe they were within budget while together exceeding it
+- [ ] The controller reads the dialect-neutral canonical request and has no knowledge of which surface produced it; adding a third dialect requires no change here
+- [ ] Slot occupancy and queue depth are reported per upstream, and requests are attributable to a surface for logging only, never for admission
+- [ ] Rejection at a full queue renders per surface — `application/problem+json` on `/v1`, the Anthropic error shape on `/llm` — from one shared decision
+
+## Verify
+
+```bash
+# Fire 40 concurrent chat requests at a stub reasoning upstream that holds each for 2s,
+# with cap=6 and queue=8 configured.
+seq 40 | xargs -P40 -I{} curl -s -o /dev/null -w '%{http_code}\n' \
+  -X POST localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' | sort | uniq -c
+# expected: a mix of 200 and 503; zero 500s
+
+# expected: the stub logged at most 6 simultaneous in-flight requests, never 7
+grep -c 'max_concurrent=7' /tmp/stub-reasoning.log
+# expected: 0
+
+curl -s -D - -o /dev/null -X POST localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' | grep -i retry-after
+# expected: Retry-After present on the rejected request
+
+# Same cap=6 stub, but split the load across both dialects: 20 on /v1 and 20 on /llm.
+seq 20 | xargs -P20 -I{} curl -s -o /dev/null -X POST localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' -d '{"model":"reasoning","messages":[]}' &
+seq 20 | xargs -P20 -I{} curl -s -o /dev/null -X POST localhost:8080/llm/v1/messages \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}' &
+wait
+grep -c 'max_concurrent=7' /tmp/stub-reasoning.log
+# expected: 0 — the two surfaces share one cap, they do not get 6 each
+```

tasks/4.2-per-caller-budgets.md

@@ -0,0 +1,32 @@
+# 4.2 — Per-caller request budgets (GREEN)
+
+Phase: 4 — Limits and budgets
+Stage: GREEN
+Depends on: [4.3](4.3-problem-json-errors.md)
+
+No `rate-limiting` plugin exists anywhere in the cluster today. This is net-new work,
+not a migration — there is no prior behaviour to preserve.
+
+- [ ] An identified caller gets a bounded number of requests per configured time window
+- [ ] Caller identity comes from the authenticated token's subject when auth is on, and from a documented fallback attribute when auth is off
+- [ ] Budget size and window length are explicit in configuration, per caller class, with no silent defaults
+- [ ] Exceeding the budget is rejected with a retryable status, an `application/problem+json` body, and a `Retry-After` naming when the window resets
+- [ ] Remaining budget and reset time are observable to the caller on allowed requests, not only on rejections
+- [ ] Budgets are enforced independently of the `reasoning` concurrency cap — a caller under budget can still be queued or rejected for slot pressure, and vice versa
+- [ ] Two distinct callers do not consume each other's budget
+- [ ] Budget state is per-replica-safe: the documented behaviour with 2 replicas is stated, not accidental
+
+## Verify
+
+```bash
+# budget=5 per 60s window for the test caller
+for i in $(seq 1 7); do
+  curl -s -o /dev/null -w '%{http_code} ' -H 'authorization: Bearer test-caller-a' \
+    localhost:8080/v1/models
+done; echo
+# expected: 200 200 200 200 200 429 429
+
+curl -s -o /dev/null -w '%{http_code}\n' -H 'authorization: Bearer test-caller-b' \
+  localhost:8080/v1/models
+# expected: 200 — caller B has its own budget
+```

tasks/4.3-problem-json-errors.md

@@ -0,0 +1,31 @@
+# 4.3 — RFC 9457 problem+json rejections (RED)
+
+Phase: 4 — Limits and budgets
+Stage: RED
+
+Every rejection the gateway generates itself must be a machine-readable problem
+document. Upstream responses are passed through untouched — this covers only errors
+the gateway originates.
+
+- [ ] Every gateway-originated rejection responds with `Content-Type: application/problem+json`
+- [ ] The body carries at minimum `type`, `title`, `status`, and `detail`, and `status` equals the HTTP status line
+- [ ] `type` is a stable, distinct URI per rejection reason, so a client can branch on it without parsing prose
+- [ ] `detail` is human-useful and names the offending input where one exists
+- [ ] `Retry-After` is set whenever a retry time is knowable — queue full, budget exhausted, upstream saturated
+- [ ] `Retry-After` is absent when no retry will help — unknown model, malformed body, oversized body
+- [ ] No token, credential, header value, or request body content appears in any field
+- [ ] A rejection is emitted as a structured log line carrying the same reason identifier used in `type`
+
+Tests come first and must fail because the shape does not exist yet, not because a
+route is missing.
+
+## Verify
+
+```bash
+curl -s -D - -o /tmp/p.json -X POST localhost:8080/v1/chat/completions \
+  -H 'content-type: application/json' -d '{"model":"nope"}' | grep -i content-type
+# expected: application/problem+json
+
+python3 -c "import json;d=json.load(open('/tmp/p.json'));assert{'type','title','status','detail'}<=d.keys();assert d['status']==400;print('ok')"
+# expected: ok
+```

tasks/5.1-prometheus-parity.md

@@ -0,0 +1,33 @@
+# 5.1 — Prometheus parity with the retiring Kong plugin (RED)
+
+Phase: 5 — Observability
+Stage: RED
+
+Kong runs a cluster-wide `prometheus` plugin today. It is deleted at teardown. If the
+gateway does not carry equivalent signal, observability REGRESSES at cutover and
+nobody notices until an incident.
+
+- [ ] A Prometheus text-format endpoint is exposed and scrapeable without authentication from inside the cluster
+- [ ] Request rate is observable, labelled by route and by upstream
+- [ ] Request latency is observable as a distribution, not a mean, labelled by route and upstream
+- [ ] Response status is observable, labelled by route, upstream, and status class
+- [ ] Bandwidth in both directions is observable per route and upstream
+- [ ] Upstream health is observable — whether each configured upstream is currently reachable and answering
+- [ ] Label values are drawn from the configured route and upstream names, never from raw request paths or user input, so cardinality cannot be driven by a caller
+- [ ] Streaming responses record their full byte count and full duration, not the time to first byte
+- [ ] The metrics endpoint is not reachable through the public route surface
+
+Write the assertions against the metric families first; they must fail because the
+families are absent, not because the endpoint 404s.
+
+## Verify
+
+```bash
+curl -s localhost:8080/metrics | grep -E '^# TYPE' | awk '{print $3}' | sort -u
+# expected: includes counter and histogram families covering requests, duration, bytes, upstream health
+
+curl -s -X POST localhost:8080/v1/chat/completions -H 'content-type: application/json' \
+  -d '{"model":"reasoning","messages":[]}' >/dev/null
+curl -s localhost:8080/metrics | grep 'upstream="reasoning-predictor"' | head
+# expected: request, duration and byte samples all carry route and upstream labels
+```

tasks/5.2-gateway-metrics.md

@@ -0,0 +1,33 @@
+# 5.2 — Gateway-specific metrics Kong could not provide (GREEN)
+
+Phase: 5 — Observability
+Stage: GREEN
+Depends on: [5.1](5.1-prometheus-parity.md)
+
+These are the signals that justify replacing Kong. Without them the concurrency cap
+and budget layers are unfalsifiable — you cannot tell a saturated GPU from a broken
+gateway.
+
+- [ ] In-flight request count is observable per upstream, and returns to zero when traffic stops
+- [ ] Queue depth for the `reasoning` concurrency queue is observable
+- [ ] Occupancy of the `reasoning` slot cap is observable — how many of the configured slots are held right now
+- [ ] The configured slot cap itself is observable, so occupancy can be read as a ratio without hardcoding the limit in a dashboard
+- [ ] Rejections are counted and broken down by reason: queue full, budget exhausted, body too large, unknown model, auth failure
+- [ ] Rejection reason label values match the stable reason identifiers used in the problem+json `type` field, so metrics and logs join cleanly
+- [ ] Time spent waiting in the queue is observable as a distribution, separately from upstream latency
+- [ ] In-flight and occupancy gauges are correct after a client disconnects mid-stream — no permanent drift upward
+
+## Verify
+
+```bash
+# Hold 3 long requests open against a stub reasoning upstream, then scrape.
+curl -s localhost:8080/metrics | grep -E 'inflight|queue_depth|slots'
+# expected: in-flight for the reasoning upstream reads 3, queue depth reads 0, slot cap is exported
+
+# Kill the clients mid-stream, wait, scrape again.
+curl -s localhost:8080/metrics | grep -E 'inflight'
+# expected: back to 0 — disconnects released their slots
+
+curl -s localhost:8080/metrics | grep 'reason='
+# expected: rejection counters split by reason, matching the problem+json type identifiers
+```

tasks/5.3-servicemonitor.md

@@ -0,0 +1,30 @@
+# 5.3 — ServiceMonitor for gateway scraping (GREEN)
+
+Phase: 5 — Observability
+Stage: GREEN
+Depends on: [5.2](5.2-gateway-metrics.md)
+
+Exposing metrics is not the same as having them collected. Kong's plugin was picked
+up cluster-wide; the gateway must be explicitly registered or the dashboards go
+blank at cutover.
+
+- [ ] A ServiceMonitor selects the gateway Service and is committed to git, applied by Argo — never `kubectl apply`
+- [ ] It carries whatever label the cluster's Prometheus uses to select ServiceMonitors, verified against the running Prometheus rather than assumed
+- [ ] The scraped port is a named port on the gateway Service, matched by name not number
+- [ ] Scrape interval and timeout are explicit
+- [ ] The gateway appears as an `up == 1` target in Prometheus, in the expected namespace
+- [ ] Metric labels identify the gateway pod and namespace, so 2 replicas are distinguishable
+- [ ] Scraping works while the gateway is still unexposed to public traffic — this lands before cutover, not after
+- [ ] Note: `prometheus-operated.monitoring` is headless with ClusterIP `None`, so any egress policy that touches it needs pod selectors, not a ClusterIP
+
+## Verify
+
+```bash
+kubectl -n monitoring get servicemonitor -l release --show-labels | grep -i frontend
+# expected: the gateway ServiceMonitor exists and carries the selector label Prometheus uses
+
+# Query Prometheus for the gateway target
+curl -s 'http://localhost:9090/api/v1/query?query=up{job=~".*homelab-frontend.*"}' \
+  | python3 -c "import json,sys;r=json.load(sys.stdin)['data']['result'];print(len(r),[x['value'][1] for x in r])"
+# expected: 2 targets, both reporting "1"
+```

tasks/6.1-hardened-image.md

@@ -0,0 +1,33 @@
+# 6.1 — Hardened container image (GREEN)
+
+Phase: 6 — Deploy and cutover
+Stage: GREEN
+
+The gateway is the public edge process. It gets no shell, no package manager, no
+writable filesystem and no privileges it does not need.
+
+- [ ] The runtime image is distroless or scratch — no shell, no package manager, no busybox
+- [ ] The container runs as a non-root user, enforced by `runAsNonRoot` and an explicit non-zero UID
+- [ ] The root filesystem is read-only; any writable path the process genuinely needs is an explicitly mounted volume, not a relaxed rootfs
+- [ ] All Linux capabilities are dropped, and none are added back
+- [ ] `seccompProfile` is `RuntimeDefault`
+- [ ] Privilege escalation is disabled
+- [ ] Image tags are the commit SHA of the source that built them. Never `:latest`, never a moving tag — Argo `selfHeal` cannot reconcile a mutable tag reliably, and a rollback needs a tag that still means what it meant yesterday
+- [ ] The image is reproducible from a committed build definition; nothing is built by hand
+- [ ] The image contains no credentials, kubeconfig or service-account token baked in (G2)
+
+## Verify
+
+```bash
+docker run --rm --entrypoint sh <image>:<sha>
+# expected: fails — no shell in the image
+
+docker run --rm --read-only --user 65532 <image>:<sha> --version
+# expected: starts and exits cleanly under a read-only rootfs as a non-root user
+
+docker inspect <image>:<sha> --format '{{.Config.User}}'
+# expected: a non-zero numeric UID, not empty and not "root"
+
+grep -rn 'image:' k8s/ | grep -v '@sha256\|:[0-9a-f]\{7,\}'
+# expected: no matches — every image reference is pinned to a SHA
+```

tasks/6.2-kubernetes-manifests.md

@@ -0,0 +1,35 @@
+# 6.2 — Deployment, Service, NetworkPolicy (GREEN)
+
+Phase: 6 — Deploy and cutover
+Stage: GREEN
+Depends on: [6.1](6.1-hardened-image.md)
+
+Manifests only. Nothing here exposes the gateway publicly — that is the cutover, and
+it is a separate step.
+
+- [ ] A Deployment runs the gateway with at least 2 replicas, matching Kong's current replica count
+- [ ] The pod spec carries the hardening from 6.1: non-root, read-only rootfs, all capabilities dropped, `seccompProfile: RuntimeDefault`
+- [ ] Liveness probes hit `/healthz` and readiness probes hit `/readyz`; readiness fails while config is invalid or JWKS has never been fetched
+- [ ] Route configuration is mounted from a ConfigMap sourced from git — not a CRD, and not baked into the image
+- [ ] A config change rolls the pods; a stale ConfigMap cannot be silently served by a long-lived pod
+- [ ] Termination allows in-flight requests to drain, with a grace period long enough for the streaming timeouts in use
+- [ ] A Service exposes the gateway in-cluster with a named port, resolvable at `http://<svc>.api.svc.cluster.local`
+- [ ] No ServiceAccount with any RBAC is bound; the pod does not need or receive an API-server token (G2)
+- [ ] A NetworkPolicy allows ingress only from `ingress-nginx`
+- [ ] The same NetworkPolicy allows egress only to the proxied upstreams plus Authentik, plus DNS. Everything else is denied
+- [ ] Resource requests and limits are set explicitly
+- [ ] Every manifest is committed to git and applied by Argo. No `kubectl apply`, no `helm upgrade` (G7)
+
+## Verify
+
+```bash
+kubectl -n api get deploy homelab-frontend -o jsonpath='{.spec.replicas} {.spec.template.spec.securityContext}{"\n"}'
+# expected: 2, with runAsNonRoot true and seccompProfile RuntimeDefault
+
+kubectl -n api exec deploy/homelab-frontend -- true 2>&1
+# expected: fails — distroless image has no shell to exec into
+
+kubectl -n llm-serving run np-probe --rm -it --image=curlimages/curl --restart=Never -- \
+  curl -s -m 5 http://homelab-frontend.api.svc.cluster.local/healthz
+# expected: times out or is refused — ingress is restricted to ingress-nginx only
+```

tasks/6.3-argocd-application.md

@@ -0,0 +1,32 @@
+# 6.3 — Argo Application in the homelab-root GitOps repo (GREEN)
+
+Phase: 6 — Deploy and cutover
+Stage: GREEN
+Depends on: [6.2](6.2-kubernetes-manifests.md)
+
+Verified live 2026-08-19: zero Argo Applications anywhere in the cluster source from
+any Forgejo URL. `github.com/Riotpiaole/riotpiao.homelab.com` is authoritative — do
+not introduce a second source of truth.
+
+- [ ] An Argo Application for the gateway is committed under `k8s/argocd/apps/` in `github.com/Riotpiaole/riotpiao.homelab.com`
+- [ ] `repoURL` is that GitHub repo. No Forgejo URL, no local path, no second remote
+- [ ] It targets the `api` namespace, alongside the existing Kong deployment
+- [ ] Its sync wave orders it so the gateway is healthy before anything that depends on it, and does not disturb Kong's existing wave-7 Application
+- [ ] Automated sync with prune and selfHeal is enabled, so drift is corrected without a human
+- [ ] The Application reaches `Synced` and `Healthy` and stays there across a resync
+- [ ] Adding it changes nothing about live traffic — Kong still serves `api.riotpiao.com` after this lands
+- [ ] The commit is pushed and Argo picks it up on its own. No `kubectl apply` of the Application itself (G7)
+
+## Verify
+
+```bash
+kubectl -n argocd get app homelab-frontend \
+  -o jsonpath='{.spec.source.repoURL}{" "}{.status.sync.status}{" "}{.status.health.status}{"\n"}'
+# expected: the GitHub repo URL, Synced, Healthy
+
+kubectl -n argocd get app -o json | grep -ci forgejo
+# expected: 0 — GitHub remains the only Application source
+
+curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/v1/models
+# expected: 200, still served by Kong — this task changed no live traffic
+```

tasks/6.4-deploy-alongside-kong.md

@@ -0,0 +1,36 @@
+# 6.4 — Deploy unexposed alongside Kong (GREEN)
+
+Phase: 6 — Deploy and cutover
+Stage: GREEN
+Depends on: [6.3](6.3-argocd-application.md)
+
+The gateway runs in the cluster against the real predictors while Kong continues to
+serve every byte of live traffic. This is the last step before anything user-visible
+changes, and it is fully reversible — deleting the Application removes it.
+
+- [ ] The gateway is running in namespace `api` and is reachable only in-cluster at `http://<svc>.api.svc.cluster.local`
+- [ ] No Ingress points at the gateway. Ingress `api/api` still sends `/` on `api.riotpiao.com` to `kong-proxy:80`
+- [ ] Its configured upstreams are the real Services: `reasoning-predictor.llm-serving:80`, `ornith-predictor.llm-serving:80`, `embeddings-predictor.llm-serving:80`, `reranker-predictor.llm-serving:80`, and `agent-hub.agent-pod:9090`
+- [ ] Note `reasoning-predictor` listens on port 80, not 8080
+- [ ] A real chat completion succeeds in-cluster against `reasoning`, returning `reasoning_content` and `content` as separate fields
+- [ ] A streaming chat completion delivers tokens incrementally in-cluster, not as one buffered blob at completion
+- [ ] Embeddings and rerank both answer correctly, with rerank reaching the upstream's `/rerank` path
+- [ ] A client disconnect mid-stream is observed to cancel the upstream generation and release its sequence slot
+- [ ] Prometheus is scraping the gateway and the metrics reflect this in-cluster traffic
+- [ ] `api.riotpiao.com` is unaffected throughout — verified before and after
+
+## Verify
+
+```bash
+kubectl -n api get ingress api -o jsonpath='{.spec.rules[0].http.paths[0].backend.service.name}{"\n"}'
+# expected: kong-proxy — the gateway is still unexposed
+
+kubectl -n api run probe --rm -it --image=curlimages/curl --restart=Never -- sh -c '
+curl -sN -X POST http://homelab-frontend.api.svc.cluster.local/v1/chat/completions \
+  -H "content-type: application/json" \
+  -d "{\"model\":\"reasoning\",\"stream\":true,\"messages\":[{\"role\":\"user\",\"content\":\"count to 5\"}]}"'
+# expected: multiple data: chunks arriving over time, terminated by data: [DONE]
+
+curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/v1/models
+# expected: 200 — public traffic still served by Kong, untouched
+```

tasks/6.5-cutover.md

@@ -0,0 +1,45 @@
+# 6.5 — Cutover: repoint Ingress `api/api` (GREEN)
+
+Phase: 6 — Deploy and cutover
+Stage: GREEN
+Depends on: [2.8](2.8-kong-parity-test.md), [6.4](6.4-deploy-alongside-kong.md)
+
+THIS is the cutover. Ingress `api/api` catch-alls `/` on `api.riotpiao.com`; today its
+backend is `kong-proxy:80`. Changing that one backend moves all live traffic to the
+gateway. Reverting is the same one-line change back to `kong-proxy:80`, committed and
+synced — Kong stays running and untouched, so the revert takes effect in seconds.
+
+Preconditions, all required before starting:
+
+- [ ] 2.8 Kong parity is passing — gateway and Kong return equivalent responses for every route in the migration inventory, including a streaming request and a mid-stream disconnect
+- [ ] 6.4 is complete: the gateway is healthy in-cluster against the real predictors, and Prometheus is scraping it
+- [ ] The inventory has been re-verified immediately beforehand. Kong's Deployment has eleven ReplicaSets and is being actively iterated; a stale inventory is a stale plan
+- [ ] Kong remains deployed and serving-capable throughout. Nothing about Kong is deleted in this task
+
+The change itself:
+
+- [ ] The `api/api` Ingress backend becomes the gateway Service, changed in git and synced by Argo. No `kubectl apply`, no `kubectl edit` (G7)
+- [ ] The nginx annotations stay exactly as they are: `proxy-read-timeout: 3600`, `proxy-send-timeout: 3600`, `proxy-buffering: off`, `proxy-body-size: 0`. These are what make token streaming work and the gateway needs the same treatment
+- [ ] The revert is a single-line commit reverting the backend to `kong-proxy:80`, and it has been rehearsed at least once
+- [ ] After the change, public streaming works end to end through nginx, unbuffered
+- [ ] pi keeps working on the legacy `/v1/{reasoning,ornith,qwen}/chat/completions` aliases
+- [ ] A soak period follows, watching gateway metrics, rejection counters and pi traffic. Do not proceed to 6.6 until the soak is clean
+
+## Verify
+
+```bash
+kubectl -n api get ingress api -o jsonpath='{.spec.rules[0].http.paths[0].backend.service.name}{"\n"}'
+# expected: the gateway Service, no longer kong-proxy
+
+curl -sN -X POST https://api.riotpiao.com/v1/chat/completions \
+  -H 'content-type: application/json' \
+  -d '{"model":"reasoning","stream":true,"messages":[{"role":"user","content":"count to 5"}]}'
+# expected: incremental data: chunks over time through nginx, ending in data: [DONE]
+
+curl -s -o /dev/null -w '%{http_code}\n' -X POST https://api.riotpiao.com/v1/reasoning/chat/completions \
+  -H 'content-type: application/json' -d '{"messages":[{"role":"user","content":"hi"}]}'
+# expected: 200 — pi's legacy alias still works
+
+kubectl -n api get deploy kong-kong -o jsonpath='{.status.readyReplicas}{"\n"}'
+# expected: 2 — Kong still running and ready, so the revert is one commit away
+```

tasks/6.6-kong-teardown.md

@@ -0,0 +1,45 @@
+# 6.6 — Kong teardown (REFACTOR)
+
+Phase: 6 — Deploy and cutover
+Stage: REFACTOR
+Depends on: [6.5](6.5-cutover.md)
+
+IRREVERSIBLE. Every step before this one could be undone in seconds by repointing
+Ingress `api/api` back to `kong-proxy:80`. Once the Kong Application is removed and
+Argo prunes the Helm release and its CRDs, that escape hatch is gone — recovery means
+reinstalling Kong from scratch and rebuilding six plugin CRs.
+
+Do not start until the soak after 6.5 has been clean for a deliberate, agreed period.
+
+Preconditions:
+
+- [ ] 6.5 is complete: `api.riotpiao.com` has been served entirely by the gateway through the soak, with no reverts
+- [ ] Gateway metrics over the soak show no elevated 5xx rate and no unexplained rejections
+- [ ] pi and every other known caller have been confirmed working against the gateway
+- [ ] The inventory has been re-verified immediately beforehand — Kong's config has been actively iterated
+
+Order matters. Delete the routing objects first, the Application last:
+
+- [ ] The 7 `ingressClassName: kong` Ingresses are deleted from git — 6 in `llm-serving`, 1 in `agent-pod` for `/console`, `/run`, `/sessions`
+- [ ] The 6 `KongPlugin` CRs are deleted from git: the three `llm-rewrite-*` chat rewrites, `llm-rewrite-rerank`, `llm-models-list`, and the cluster-wide `prometheus` plugin
+- [ ] Removing the cluster-wide `prometheus` plugin does not blind any dashboard, because 5.1 parity metrics are already being scraped from the gateway
+- [ ] The `kong` Application is then removed from `k8s/argocd/apps/55-api-gateway.yaml`
+- [ ] Argo prunes the Helm release, the Kong CRDs and the namespace leftovers on its own. No `helm uninstall`, no `kubectl delete` (G7)
+- [ ] Public traffic is verified unaffected after each deletion, not only at the end
+- [ ] No orphaned Kong CRDs, ReplicaSets or Services remain in namespace `api`
+
+## Verify
+
+```bash
+kubectl get ingress -A -o jsonpath='{range .items[*]}{.spec.ingressClassName}{"\n"}{end}' | sort | uniq -c
+# expected: no kong entries remain; nginx and istio counts unchanged
+
+kubectl get kongplugins -A 2>&1; kubectl -n argocd get app kong 2>&1
+# expected: CRD not found, and Application "kong" not found
+
+kubectl -n api get all | grep -i kong
+# expected: no output — nothing Kong-related left
+
+curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/v1/models
+# expected: 200, served by the gateway
+```

tasks/7.1-cluster-prefix-atlas.md

@@ -0,0 +1,36 @@
+# 7.1 — `/cluster/*` proxies to atlas (GREEN)
+
+Phase: 7 — Additional capability prefixes
+Stage: GREEN
+Depends on: [6.5](6.5-cutover.md)
+
+atlas lives in a separate repo, `riotpiao-backend`. It keeps its own informers and its
+own RBAC. The gateway proxies to it and holds no Kubernetes credentials of its own —
+this is G2, and it is the whole reason cluster-read capability lives behind atlas
+rather than in the edge process.
+
+- [ ] `/cluster/*` on `api.riotpiao.com` proxies to the atlas Service
+- [ ] The gateway gains no ServiceAccount token, no kubeconfig and no RBAC as part of this. Any authorization decision about cluster data is atlas's, not the gateway's (G2)
+- [ ] Path rewriting between the `/cluster` prefix and atlas's own paths is explicit in configuration
+- [ ] Connect, read and write timeouts and a body cap are explicit for this route, with no silent defaults (G6)
+- [ ] The route requires authentication, and the token is checked for cluster capability — a token minted for queue access must not read cluster state
+- [ ] The NetworkPolicy is extended to allow egress to atlas and nothing more
+- [ ] `/v1/*` behaviour is byte-identical before and after this route is added — the prefixes cannot collide
+- [ ] Metrics and rejection counters cover this route with its own route label
+
+## Verify
+
+```bash
+curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/cluster/healthz
+# expected: 401 without a token
+
+curl -s -H "authorization: Bearer $CLUSTER_TOKEN" https://api.riotpiao.com/cluster/healthz
+# expected: atlas's own response body, proxied unmodified
+
+curl -s -o /dev/null -w '%{http_code}\n' -H "authorization: Bearer $QUEUE_ONLY_TOKEN" \
+  https://api.riotpiao.com/cluster/healthz
+# expected: 403 — a queue token does not grant cluster capability
+
+kubectl -n api get pod -l app=homelab-frontend -o jsonpath='{.items[0].spec.serviceAccountName}{"\n"}'
+# expected: a ServiceAccount with no RBAC bindings; the gateway still holds no cluster credentials
+```

tasks/7.2-sqs-prefix.md

@@ -0,0 +1,35 @@
+# 7.2 — `/sqs/*` to kmsvc management-service and Kafka (GREEN)
+
+Phase: 7 — Additional capability prefixes
+Stage: GREEN
+Depends on: [6.5](6.5-cutover.md)
+
+`management-service` already exposes gRPC at `kmsvc.riotpiao.com`. This prefix is a
+NEW surface, not a replacement — that gRPC endpoint stays exactly as it is and nothing
+here migrates off it.
+
+- [ ] `/sqs/*` on `api.riotpiao.com` proxies to kmsvc `management-service` and the Kafka/Strimzi surfaces in the `sqs` namespace
+- [ ] `kmsvc.riotpiao.com` is unchanged and still serving after this lands
+- [ ] Which sub-paths map to which upstream is explicit in configuration; there is no catch-all fallback
+- [ ] Timeouts and body caps are explicit per sub-route, with no silent defaults (G6)
+- [ ] The route requires authentication and the token is checked for queue capability
+- [ ] The NetworkPolicy is extended to reach only the named `sqs` upstreams
+- [ ] `kmsvc-redis-master.sqs:6379` has NO authentication — `ALLOW_EMPTY_PASSWORD=yes`, TLS off. Any workload with network reach has full unauthenticated read/write. It is not proxied, and the NetworkPolicy must not grant the gateway egress to it
+- [ ] `/v1/*` behaviour is unchanged before and after
+- [ ] Metrics and rejection counters cover this route with its own route label
+
+## Verify
+
+```bash
+curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/sqs/healthz
+# expected: 401 without a token
+
+curl -s -H "authorization: Bearer $QUEUE_TOKEN" https://api.riotpiao.com/sqs/healthz
+# expected: management-service's own response, proxied unmodified
+
+kubectl -n api get networkpolicy -o yaml | grep -c 6379
+# expected: 0 — no egress path from the gateway to unauthenticated Redis
+
+curl -s -o /dev/null -w '%{http_code}\n' https://kmsvc.riotpiao.com
+# expected: unchanged from before this task — the existing gRPC surface is untouched
+```

tasks/7.3-workflow-prefix.md

@@ -0,0 +1,34 @@
+# 7.3 — `/workflow/*` to Temporal (GREEN)
+
+Phase: 7 — Additional capability prefixes
+Stage: GREEN
+Depends on: [6.5](6.5-cutover.md)
+
+Temporal runs in the `temporal` namespace. Temporal namespace registration is
+automatic via queue-operator and is NEVER done manually — this route must not create,
+register or mutate Temporal namespaces, only proxy to what queue-operator has already
+provisioned.
+
+- [ ] `/workflow/*` on `api.riotpiao.com` proxies to the Temporal Service in the `temporal` namespace
+- [ ] Nothing in this route registers a Temporal namespace. Registration stays with queue-operator
+- [ ] Path rewriting between the `/workflow` prefix and Temporal's own paths is explicit in configuration
+- [ ] Timeouts and body caps are explicit, with no silent defaults (G6). Long-poll semantics are accounted for rather than truncated by a short read timeout
+- [ ] The route requires authentication and the token is checked for workflow capability — a GPU token must not drive workflows
+- [ ] The NetworkPolicy is extended to reach only Temporal
+- [ ] Streaming or long-poll responses pass through unbuffered, and a client disconnect cancels the upstream call rather than orphaning it (G4)
+- [ ] `/v1/*` behaviour is unchanged before and after
+- [ ] Metrics and rejection counters cover this route with its own route label
+
+## Verify
+
+```bash
+curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/workflow/health
+# expected: 401 without a token
+
+curl -s -H "authorization: Bearer $WORKFLOW_TOKEN" https://api.riotpiao.com/workflow/health
+# expected: Temporal's own response, proxied unmodified
+
+kubectl -n temporal exec svc/temporal-admintools -- tctl --ad temporal-frontend:7233 namespace list | sort > /tmp/ns.after
+diff /tmp/ns.before /tmp/ns.after
+# expected: no diff — the gateway registered nothing; queue-operator remains the only registrar
+```

tasks/7.4-db-prefix.md

@@ -0,0 +1,41 @@
+# 7.4 — `/db/*` read surfaces (GREEN)
+
+Phase: 7 — Additional capability prefixes
+Stage: GREEN
+Depends on: [6.5](6.5-cutover.md)
+
+The most dangerous prefix in this phase. G2 says the gateway holds no cluster
+credentials — so it cannot be the thing that authenticates to CloudNativePG, MinIO or
+Prometheus on a caller's behalf. If designing this route makes you want to give the
+gateway a secret, the design is wrong: put the credential-holding logic in a service
+behind the gateway and proxy to that.
+
+- [ ] `/db/*` on `api.riotpiao.com` exposes read surfaces backed by CloudNativePG, MinIO and monitoring
+- [ ] The gateway holds no database password, no MinIO access key and no cluster credential of any kind (G2). Credentials, if any are needed, live in the service being proxied to
+- [ ] Exposed operations are read-only. There is no write, no delete and no schema-changing path on this prefix
+- [ ] Which sub-paths reach which upstream is enumerated explicitly in configuration. No catch-all, no pass-through of arbitrary query text
+- [ ] Timeouts and body caps are explicit per sub-route, with no silent defaults (G6)
+- [ ] The route requires authentication and the token is checked for a distinct read capability
+- [ ] Result sets are paginated with a bounded page size; an unbounded read cannot be requested
+- [ ] The NetworkPolicy is extended only to the specific upstreams reached. Note `prometheus-operated.monitoring` is headless with ClusterIP `None`, so it needs a pod selector, not a ClusterIP
+- [ ] `/v1/*` behaviour is unchanged before and after
+- [ ] Metrics and rejection counters cover this route with its own route label
+
+## Verify
+
+```bash
+curl -s -o /dev/null -w '%{http_code}\n' https://api.riotpiao.com/db/healthz
+# expected: 401 without a token
+
+curl -s -o /dev/null -w '%{http_code}\n' -X DELETE -H "authorization: Bearer $DB_READ_TOKEN" \
+  https://api.riotpiao.com/db/anything
+# expected: 405 or 404 — no mutating method is routable on this prefix
+
+kubectl -n api get pod -l app=homelab-frontend -o jsonpath='{range .items[0].spec.containers[0].env[*]}{.name}{"\n"}{end}' \
+  | grep -Ei 'password|secret|access_key'
+# expected: no output — the gateway carries no database or object-store credentials
+
+curl -s -H "authorization: Bearer $DB_READ_TOKEN" 'https://api.riotpiao.com/db/...?limit=100000' \
+  | python3 -c "import json,sys;d=json.load(sys.stdin);print(len(d['items']))"
+# expected: capped at the configured maximum page size, not 100000
+```

tasks/AGENT-PROMPT.md

@@ -0,0 +1,101 @@
+# Agent prompt template
+
+Every task is worked by an agent starting from **zero context**. No memory of prior
+tasks, no conversation history, no assumptions about what already exists.
+
+This is deliberate. Task files are written to be self-contained precisely so that a
+fresh agent can pick any one of them up. It also means a task that cannot be completed
+from its own file plus this prompt is a task that is under-specified — that is a bug in
+the task, and worth reporting rather than working around.
+
+Rendered and invoked by [`scripts/run-task.sh`](../scripts/run-task.sh). Do not paste
+this by hand; use the script so the fresh-session guarantee actually holds.
+
+---
+
+## Template
+
+`{{TASK_ID}}` and `{{TASK_FILE}}` are substituted by the runner.
+
+```
+You are implementing one task in the homelab-frontend repository: a Go API gateway
+that replaces Kong OSS on api.riotpiao.com.
+
+You are starting from zero context. Everything you need is below or in the files named
+below. Do not assume any prior work exists beyond what you find in the repository.
+
+## Your task
+
+Read tasks/{{TASK_FILE}} and implement it. That file states what must be true; it does
+not state how. The design is yours to reason out. The checkboxes are the contract.
+
+## Before writing code
+
+1. Read tasks/{{TASK_FILE}} in full.
+2. Check its `Depends on:` line. If a dependency is not yet implemented in this
+   repository, stop and report that instead of building it yourself. One task per run.
+3. Look at how the surrounding code is written and match it. If the repository is
+   still empty, you are establishing the conventions, so choose carefully.
+
+## Invariants — breaking one of these is a design change, not a detail
+
+G1  ingress-nginx owns TLS. The gateway never terminates TLS.
+G2  The gateway holds no Kubernetes credentials. Config comes from git, not a CRD.
+G3  Public surfaces use standard protocol shapes. If an OpenAI or Anthropic SDK
+    cannot call it unmodified, the design is wrong.
+G4  Streaming is unbuffered, and a client disconnect cancels the upstream request.
+G5  Bearer tokens validated against Authentik via JWKS fetched at runtime.
+G6  Every timeout, body cap and concurrency limit is explicit in configuration.
+G7  Deployment flows through git and Argo. Never run kubectl apply, helm upgrade,
+    or terraform apply.
+
+## How to work
+
+Test-driven. If the task is marked `Stage: RED`, write the failing test first and
+confirm it fails for the right reason before implementing. If `GREEN`, write the
+minimum code that passes. If `REFACTOR`, keep the tests green while improving shape.
+
+Go standards for this repository:
+- Never discard errors with `_ =`. Wrap them with context.
+- Every upstream call carries a context.Context.
+- No naked returns. Use `any`, not `interface{}`.
+- Table-driven tests with subtests where there is more than one case.
+
+## Definition of done
+
+Run the `## Verify` block from the task file. It must pass.
+
+Then run all three of these regardless of what the task's verify block says:
+
+    go test ./... -race
+    CGO_ENABLED=0 go build ./...
+    go vet ./...
+
+`-race` is mandatory. This is a concurrent proxy; a test suite that passes without the
+race detector tells you almost nothing. A detected race is a failure, not a warning.
+
+Verification means asserting on a real HTTP response — status, headers, body. "It
+compiles" and "it starts" are not verification. If you cannot run the verification
+locally with no cluster and no credentials, that is itself a problem to report.
+
+Then **edit tasks/{{TASK_FILE}} and change `- [ ]` to `- [x]`** for each criterion you
+actually satisfied. This is a file edit, not something to state in your summary. Leave
+unticked anything you did not complete. A summary that claims `[x]` while the file still
+reads `[ ]` is a false report.
+
+## Hard rules
+
+- Kong is serving live traffic on api.riotpiao.com right now. Change no cluster state.
+- Do not commit or push.
+- Do not implement tasks other than {{TASK_ID}}. If you notice something else that
+  needs doing, report it rather than fixing it.
+- Do not add features, abstractions, or configurability that the task did not ask for.
+- If the task is ambiguous or appears wrong, stop and say so. Do not guess and proceed.
+
+## Report when finished
+
+- What you implemented, and the files you touched.
+- The verification command you ran and its actual output.
+- Which checkboxes you ticked and which you did not, with reasons.
+- Anything you found that is wrong elsewhere in the repository or the task files.
+```

tasks/INDEX.md

@@ -0,0 +1,151 @@
+# Task board — homelab-frontend
+
+The Go API gateway replacing Kong OSS on `api.riotpiao.com`.
+
+Contract: [REQUIREMENTS.md](../REQUIREMENTS.md).
+Why: [ADR-0001](../docs/adr/ADR-0001-retire-kong-for-go-gateway.md).
+What Kong does today and the cutover order: [docs/MIGRATION-kong.md](../docs/MIGRATION-kong.md).
+
+## Rules carried from the ADR and requirements
+
+- G1 — ingress-nginx owns TLS. The gateway never terminates TLS.
+- G2 — the gateway holds no Kubernetes credentials. Config comes from git, not a CRD.
+- G3 — public surfaces use standard protocol shapes. If an OpenAI SDK can't call it unmodified, it's wrong.
+- G4 — streaming is unbuffered, and a client disconnect cancels the upstream.
+- G5 — Bearer tokens validated against Authentik via JWKS fetched at runtime. No pinned keys.
+- G6 — every timeout, body cap and concurrency limit is explicit in config.
+- G7 — deployment flows through git and Argo. No `kubectl apply`, no `helm upgrade`.
+
+## How to work these
+
+Each task is self-contained — it states what must be true, not how to build it.
+Reason out the implementation; the acceptance criteria are the contract.
+
+Stages follow red-green-refactor. A task marked RED means the test comes first and
+must fail for the right reason before any implementation exists.
+
+**Verification means asserting on a real HTTP response** — status, headers, body.
+"It compiles" and "it starts" are not verification. Every task that touches an API
+surface has a `## Verify` block with a runnable command.
+
+Kong is serving live traffic throughout phases 0–5. Nothing in those phases may
+change cluster state.
+
+## Phase 0 — Foundations
+
+| Task | Description |
+|---|---|
+| [0.1](0.1-module-and-entrypoint.md) | Go module, entrypoint, graceful shutdown |
+| [0.2](0.2-route-configuration.md) | Declarative route/upstream config from git, fail-loud validation |
+| [0.3](0.3-health-endpoints.md) | `/healthz` and `/readyz` |
+| [0.4](0.4-local-dev-harness.md) | Run with no cluster, no kubeconfig, no credentials — stub upstreams |
+| [0.5](0.5-structured-logging.md) | Structured logs, no secrets or bodies |
+| [0.6](0.6-ci-pipeline.md) | CI: build, vet, test, `govulncheck` |
+
+## Phase 1 — Proxy core
+
+| Task | Description |
+|---|---|
+| [1.1](1.1-reverse-proxy.md) | Reverse proxy to a configured upstream, connection reuse |
+| [1.2](1.2-streaming-passthrough.md) | SSE and chunked responses pass through unbuffered |
+| [1.3](1.3-disconnect-propagation.md) | Client disconnect cancels the upstream request |
+| [1.4](1.4-per-route-timeouts.md) | Explicit connect/read/write timeouts per route |
+| [1.5](1.5-header-hygiene.md) | Hop-by-hop stripping, `X-Forwarded-*` from nginx |
+| [1.6](1.6-websocket-upgrade.md) | WebSocket upgrade — `agent-pod/console` needs it |
+| [1.7](1.7-body-size-caps.md) | Per-route request body limits |
+
+## Phase 2 — LLM surfaces (`/v1/*` and `/llm/*`)
+
+Two protocol dialects over the same models and the same slot controller. Wire
+formats are documented in [docs/API-llm.md](../docs/API-llm.md).
+
+### OpenAI dialect — `/v1/*`
+
+| Task | Description |
+|---|---|
+| [2.1](2.1-model-registry.md) | Model → upstream map from config |
+| [2.2](2.2-body-based-dispatch.md) | `POST /v1/chat/completions` routes on the body's `model` — the reason this project exists |
+| [2.3](2.3-unknown-model-errors.md) | Unknown/missing model → RFC 9457 problem+json listing valid values |
+| [2.4](2.4-legacy-path-aliases.md) | Keep `/v1/{reasoning,ornith,qwen}/chat/completions` working during cutover |
+| [2.5](2.5-models-endpoint.md) | `GET /v1/models` derived from config, never hardcoded |
+| [2.6](2.6-embeddings-passthrough.md) | `POST /v1/embeddings` — no rewrite needed |
+| [2.7](2.7-rerank-rewrite.md) | `POST /v1/rerank` → upstream `/rerank` |
+| [2.8](2.8-kong-parity-test.md) | Gateway and Kong return equivalent responses for every migrated route |
+
+### Anthropic dialect — `/llm/*`
+
+| Task | Description |
+|---|---|
+| [2.9](2.9-canonical-request-model.md) | Dialect-neutral internal request both surfaces translate into |
+| [2.10](2.10-anthropic-request-translation.md) | `POST /llm/v1/messages` request → canonical; `system`, blocks, required `max_tokens` |
+| [2.11](2.11-anthropic-response-translation.md) | Upstream response → Messages shape; `reasoning_content` becomes a `thinking` block |
+| [2.12](2.12-anthropic-sse-state-machine.md) | Named-event SSE with block indices — the hardest task in the phase |
+| [2.13](2.13-anthropic-error-shape.md) | Anthropic error shape, not RFC 9457 — same rejection, two renderings |
+| [2.14](2.14-queue-position-event.md) | Custom `event: queue` before `message_start` — deliberate non-standard extension |
+| [2.15](2.15-dialect-scope-boundary.md) | Enforce what is deliberately unimplemented: tools, images, caching, batch |
+
+## Phase 3 — Authentication (Authentik)
+
+| Task | Description |
+|---|---|
+| [3.1](3.1-jwks-fetch-and-rotation.md) | Fetch and cache Authentik JWKS, handle rotation without a runbook |
+| [3.2](3.2-bearer-validation.md) | Validate `Authorization: Bearer` — the thing Kong OSS could not do |
+| [3.3](3.3-authentik-service-account.md) | Service account + `client_credentials` provider in Authentik |
+| [3.4](3.4-flag-gated-rollout.md) | Auth defaults off; enabling it is deliberate |
+| [3.5](3.5-capability-authorization.md) | A queue token must not invoke a GPU |
+| [3.6](3.6-pi-client-migration.md) | Move pi off the `apikey` header onto Bearer |
+
+## Phase 4 — Limits and budgets
+
+| Task | Description |
+|---|---|
+| [4.1](4.1-gpu-slot-semaphore.md) | Cap concurrent `reasoning` requests below 8 slots, bounded queue |
+| [4.2](4.2-per-caller-budgets.md) | Request budget per identified caller per window |
+| [4.3](4.3-problem-json-errors.md) | RFC 9457 rejections with `Retry-After` |
+
+## Phase 5 — Observability
+
+| Task | Description |
+|---|---|
+| [5.1](5.1-prometheus-parity.md) | Match the retiring Kong plugin: rate, latency, status, bandwidth, upstream health |
+| [5.2](5.2-gateway-metrics.md) | In-flight per upstream, queue depth, slot occupancy, rejections by reason |
+| [5.3](5.3-servicemonitor.md) | ServiceMonitor so Prometheus scrapes it |
+
+## Phase 6 — Deploy and cutover
+
+| Task | Description |
+|---|---|
+| [6.1](6.1-hardened-image.md) | Distroless, non-root, read-only rootfs, no shell, SHA tags |
+| [6.2](6.2-kubernetes-manifests.md) | Deployment, Service, NetworkPolicy |
+| [6.3](6.3-argocd-application.md) | Argo Application in the homelab-root GitOps repo |
+| [6.4](6.4-deploy-alongside-kong.md) | Deploy unexposed, verify in-cluster against the real upstreams |
+| [6.5](6.5-cutover.md) | Repoint the nginx Ingress from `kong-proxy` to the gateway — reversible |
+| [6.6](6.6-kong-teardown.md) | Delete kong Ingresses, plugins, Helm release. **Irreversible** |
+
+## Phase 7 — Additional capability prefixes
+
+Deliberately after cutover. Each is additive and must not disturb `/v1/*`.
+
+| Task | Description |
+|---|---|
+| [7.1](7.1-cluster-prefix-atlas.md) | `/cluster/*` → atlas (`riotpiao-backend`) |
+| [7.2](7.2-sqs-prefix.md) | `/sqs/*` → kmsvc management-service, Kafka |
+| [7.3](7.3-workflow-prefix.md) | `/workflow/*` → Temporal |
+| [7.4](7.4-db-prefix.md) | `/db/*` → CloudNativePG, MinIO, monitoring reads |
+
+## Progress
+
+Status as of 2026-08-19: scaffolded, nothing implemented. Kong is still serving all
+live traffic on `api.riotpiao.com`, currently **unauthenticated**.
+
+50 tasks. Suggested first slice: 0.1 → 0.2 → 0.4 → 1.1 → 1.2 → 2.1 → 2.2. That
+reaches the single capability Kong could not provide — body-based model dispatch —
+with a verification loop that needs no cluster.
+
+The Anthropic dialect (2.9-2.15) can be worked in parallel with the OpenAI dialect
+once 2.9 lands, since both translate into the same canonical request. Do not build
+either surface's admission control separately — 4.1 owns it for both.
+
+Decided 2026-08-19: authentication is `Authorization: Bearer` on **both** surfaces.
+A stock Anthropic SDK sends `x-api-key` and will get a 401; that is accepted because
+the `/llm` client is first-party. The 401 must say so rather than being bare.