chore: initial commit of Go API gateway
Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
Story Crater Bot
@@ -0,0 +1,136 @@
|
||||
# ADR-0001 — Retire Kong OSS in favour of a Go API gateway
|
||||
|
||||
Status: Accepted
|
||||
Date: 2026-08-19
|
||||
Deciders: rock
|
||||
|
||||
## Context
|
||||
|
||||
`api.riotpiao.com` is currently served by Kong OSS 3.4.1 (Helm, DB-less, namespace `api`,
|
||||
Argo wave 7), sitting behind ingress-nginx which owns TLS. Kong routes to the KServe
|
||||
model predictors in `llm-serving` via seven `ingressClassName: kong` Ingresses and six
|
||||
`KongPlugin` CRs.
|
||||
|
||||
Three separate capabilities were attempted on Kong OSS. All three failed, and each
|
||||
failure is already documented in-repo by the person who hit it:
|
||||
|
||||
**1. Body-based model dispatch is not expressible.**
|
||||
From `k8s/apps/api/llm-routes.yaml`:
|
||||
|
||||
> a single `/v1/chat/completions` endpoint that dispatches on the body's `model` field is
|
||||
> not expressible in Kong OSS (`ai-proxy-advanced`, which does multi-target model routing,
|
||||
> is Enterprise-only).
|
||||
|
||||
The workaround is a path-per-model surface (`/v1/reasoning/chat/completions`,
|
||||
`/v1/ornith/...`, `/v1/qwen/...`) with a `request-transformer` force-overwriting the body's
|
||||
`model` field. This is not OpenAI-standard, so every client needs bespoke configuration —
|
||||
visible today in `~/.pi/agent/models.json`, which carries three separate provider entries
|
||||
for what should be one endpoint.
|
||||
|
||||
**2. OIDC is Enterprise-only.**
|
||||
`k8s/apps/api/AUTH-PLAN.md` routes around the missing `openid-connect` plugin using the
|
||||
built-in `jwt` plugin, which requires pinning Authentik's RSA public key onto a
|
||||
KongConsumer. That plan lists its own consequence:
|
||||
|
||||
> Pinning `rsa_public_key`: Authentik key rotation would break it — document a rotation
|
||||
> runbook, or have the provision script re-export the cert PEM into the Kong credential on
|
||||
> each run.
|
||||
|
||||
A rotation runbook is a standing operational liability accepted only because the gateway
|
||||
cannot fetch JWKS itself.
|
||||
|
||||
**3. `key-auth` cannot read `Authorization: Bearer`.**
|
||||
From `k8s/apps/api/model-auth.yaml`:
|
||||
|
||||
> a raw `apikey: <key>` header succeeds (200), the same request with only
|
||||
> `Authorization: Bearer <key>` fails (401). No OpenAI-SDK-compatible client (pi included)
|
||||
> sends a raw apikey header or lets you customize the header name, so every such client was
|
||||
> hard-blocked.
|
||||
|
||||
Consequence: authentication on the model routes is **currently disabled**. Verified live
|
||||
2026-08-19 — `api.riotpiao.com/v1/reasoning/chat/completions` answers unauthenticated.
|
||||
|
||||
Separately, the intended surface has grown beyond LLM routing. The target is a
|
||||
capability-per-subdomain API over cluster services — `sqs.riotpiao.com` for queue
|
||||
operations, `workflow.riotpiao.com` for Temporal, `cluster.riotpiao.com` for atlas — each
|
||||
needing request shaping, per-caller budgets and streaming semantics that are application
|
||||
concerns, not gateway-plugin concerns.
|
||||
|
||||
## Decision
|
||||
|
||||
Retire Kong OSS entirely. Replace it with a purpose-built Go service,
|
||||
`homelab-frontend`, which owns north-south routing, authentication, and request shaping
|
||||
for every public capability on `*.riotpiao.com`.
|
||||
|
||||
ingress-nginx keeps the edge and TLS. It forwards to the gateway instead of `kong-proxy`.
|
||||
|
||||
Authentication is Authentik OIDC, validated by fetching JWKS from
|
||||
`https://authentik.riotpiao.com` at runtime.
|
||||
|
||||
## Options considered
|
||||
|
||||
**A. Stay on Kong OSS, accept the workarounds.**
|
||||
Keeps a battle-tested proxy and its Prometheus plugin. But the path-per-model surface stays
|
||||
non-standard, the RSA pinning runbook stays, and auth stays off until someone writes a
|
||||
`request-transformer` shim to copy Bearer into an `apikey` header. Every new capability
|
||||
(`sqs`, `workflow`) inherits the same constraints.
|
||||
|
||||
**B. Buy Kong Enterprise.**
|
||||
`ai-proxy-advanced` and `openid-connect` solve 1 and 2. Does not solve the genuinely
|
||||
application-level requirements at all — signed session cookies, per-session daily message
|
||||
budgets, a 6-of-8 GPU sequence-slot semaphore with a bounded queue, and
|
||||
disconnect-cancels-upstream are not gateway features in any tier. Cost for a homelab is not
|
||||
justifiable.
|
||||
|
||||
**C. Go gateway, Kong retained for LLM paths only.**
|
||||
Gradual migration, lower risk. But it means running two gateways indefinitely, splitting the
|
||||
routing table across Kong CRDs and Go code, and keeping the Kong Helm release and its CRDs.
|
||||
The split is the thing most likely to drift.
|
||||
|
||||
**D. Go gateway, Kong retired entirely.** — chosen
|
||||
One routing table, one auth implementation, one place to reason about timeouts. The logic
|
||||
being replaced is small: four `request-transformer` plugins that set a body field and
|
||||
rewrite a URI, one `request-termination` serving a static JSON model list, and one
|
||||
`prometheus` plugin. That is on the order of a hundred lines of Go, against roughly 480
|
||||
lines of YAML it retires.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Gained
|
||||
|
||||
- **Standard OpenAI surface.** One `POST /v1/chat/completions`, model selected from the
|
||||
request body. Any OpenAI SDK works unmodified. The three pi provider entries collapse to
|
||||
one.
|
||||
- **Working authentication.** Bearer tokens are read from the header, because it is our
|
||||
code. JWKS is fetched and cached with automatic rotation handling, so the AUTH-PLAN.md
|
||||
rotation runbook is deleted rather than written.
|
||||
- **Application-level policy becomes possible.** GPU slot semaphore, per-session budgets,
|
||||
disconnect propagation and SSE handling live where the state is.
|
||||
- **One timeout story.** Kong currently sets `read-timeout: 3600000` (1 hour) on chat
|
||||
routes, which silently defeats any shorter server-side cap. Retiring Kong removes the
|
||||
conflicting layer.
|
||||
- **~480 lines of gateway YAML deleted**, plus the Kong CRDs, the Helm release, and its
|
||||
`ServerSideApply` workaround for oversized CRD annotations.
|
||||
|
||||
### Lost / assumed
|
||||
|
||||
- **We now own proxy correctness.** Connection pooling, retries, timeout propagation,
|
||||
streaming passthrough, header hygiene, graceful shutdown. `net/http/httputil.ReverseProxy`
|
||||
covers most of it, but it is our bug surface now.
|
||||
- **Kong's Prometheus plugin goes away.** The gateway must expose equivalent metrics itself
|
||||
(bandwidth, latency, status codes, upstream health) or observability regresses.
|
||||
- **Migration touches live traffic.** pi depends on `api.riotpiao.com` today. Cutover must
|
||||
be reversible — see `docs/MIGRATION-kong.md`.
|
||||
- **`agent-pod/console` is a kong-class Ingress** exposing `/console` (WebSocket), `/run`
|
||||
and `/sessions`. It must migrate too, and it is currently unauthenticated and publicly
|
||||
routed while accepting free-form prompts into a shell-capable container. Putting it behind
|
||||
the gateway's Authentik auth is a security improvement, not just a port.
|
||||
|
||||
### Risks
|
||||
|
||||
- Enabling Authentik auth will break any client currently relying on the unauthenticated
|
||||
surface — including pi, until its `models.json` is updated. Auth must ship behind a flag
|
||||
and be enabled deliberately.
|
||||
- Kong's `request-termination` for `/v1/models` returns a **static** list that can drift
|
||||
from what the engines actually serve. Porting it verbatim ports the bug; the gateway
|
||||
should derive the list from configured upstreams instead.
|
||||
Reference in New Issue
Block a user