Story Crater BotandClaude Opus 5
058f11cf2b
Baseline for the Kong replacement on api.riotpiao.com. Brings the working tree under version control for the first time: gateway source, the task board that drives the agent runs, test fixtures, and K8s manifests. Anchor the gateway ignore rule to the repo root. Unanchored, "gateway" also matched the cmd/gateway/ source directory, so the program entrypoint was excluded from every commit. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
137 lines
6.8 KiB
Markdown
137 lines
6.8 KiB
Markdown
# ADR-0001 — Retire Kong OSS in favour of a Go API gateway
|
|
|
|
Status: Accepted
|
|
Date: 2026-08-19
|
|
Deciders: rock
|
|
|
|
## Context
|
|
|
|
`api.riotpiao.com` is currently served by Kong OSS 3.4.1 (Helm, DB-less, namespace `api`,
|
|
Argo wave 7), sitting behind ingress-nginx which owns TLS. Kong routes to the KServe
|
|
model predictors in `llm-serving` via seven `ingressClassName: kong` Ingresses and six
|
|
`KongPlugin` CRs.
|
|
|
|
Three separate capabilities were attempted on Kong OSS. All three failed, and each
|
|
failure is already documented in-repo by the person who hit it:
|
|
|
|
**1. Body-based model dispatch is not expressible.**
|
|
From `k8s/apps/api/llm-routes.yaml`:
|
|
|
|
> a single `/v1/chat/completions` endpoint that dispatches on the body's `model` field is
|
|
> not expressible in Kong OSS (`ai-proxy-advanced`, which does multi-target model routing,
|
|
> is Enterprise-only).
|
|
|
|
The workaround is a path-per-model surface (`/v1/reasoning/chat/completions`,
|
|
`/v1/ornith/...`, `/v1/qwen/...`) with a `request-transformer` force-overwriting the body's
|
|
`model` field. This is not OpenAI-standard, so every client needs bespoke configuration —
|
|
visible today in `~/.pi/agent/models.json`, which carries three separate provider entries
|
|
for what should be one endpoint.
|
|
|
|
**2. OIDC is Enterprise-only.**
|
|
`k8s/apps/api/AUTH-PLAN.md` routes around the missing `openid-connect` plugin using the
|
|
built-in `jwt` plugin, which requires pinning Authentik's RSA public key onto a
|
|
KongConsumer. That plan lists its own consequence:
|
|
|
|
> Pinning `rsa_public_key`: Authentik key rotation would break it — document a rotation
|
|
> runbook, or have the provision script re-export the cert PEM into the Kong credential on
|
|
> each run.
|
|
|
|
A rotation runbook is a standing operational liability accepted only because the gateway
|
|
cannot fetch JWKS itself.
|
|
|
|
**3. `key-auth` cannot read `Authorization: Bearer`.**
|
|
From `k8s/apps/api/model-auth.yaml`:
|
|
|
|
> a raw `apikey: <key>` header succeeds (200), the same request with only
|
|
> `Authorization: Bearer <key>` fails (401). No OpenAI-SDK-compatible client (pi included)
|
|
> sends a raw apikey header or lets you customize the header name, so every such client was
|
|
> hard-blocked.
|
|
|
|
Consequence: authentication on the model routes is **currently disabled**. Verified live
|
|
2026-08-19 — `api.riotpiao.com/v1/reasoning/chat/completions` answers unauthenticated.
|
|
|
|
Separately, the intended surface has grown beyond LLM routing. The target is a
|
|
capability-per-subdomain API over cluster services — `sqs.riotpiao.com` for queue
|
|
operations, `workflow.riotpiao.com` for Temporal, `cluster.riotpiao.com` for atlas — each
|
|
needing request shaping, per-caller budgets and streaming semantics that are application
|
|
concerns, not gateway-plugin concerns.
|
|
|
|
## Decision
|
|
|
|
Retire Kong OSS entirely. Replace it with a purpose-built Go service,
|
|
`homelab-frontend`, which owns north-south routing, authentication, and request shaping
|
|
for every public capability on `*.riotpiao.com`.
|
|
|
|
ingress-nginx keeps the edge and TLS. It forwards to the gateway instead of `kong-proxy`.
|
|
|
|
Authentication is Authentik OIDC, validated by fetching JWKS from
|
|
`https://authentik.riotpiao.com` at runtime.
|
|
|
|
## Options considered
|
|
|
|
**A. Stay on Kong OSS, accept the workarounds.**
|
|
Keeps a battle-tested proxy and its Prometheus plugin. But the path-per-model surface stays
|
|
non-standard, the RSA pinning runbook stays, and auth stays off until someone writes a
|
|
`request-transformer` shim to copy Bearer into an `apikey` header. Every new capability
|
|
(`sqs`, `workflow`) inherits the same constraints.
|
|
|
|
**B. Buy Kong Enterprise.**
|
|
`ai-proxy-advanced` and `openid-connect` solve 1 and 2. Does not solve the genuinely
|
|
application-level requirements at all — signed session cookies, per-session daily message
|
|
budgets, a 6-of-8 GPU sequence-slot semaphore with a bounded queue, and
|
|
disconnect-cancels-upstream are not gateway features in any tier. Cost for a homelab is not
|
|
justifiable.
|
|
|
|
**C. Go gateway, Kong retained for LLM paths only.**
|
|
Gradual migration, lower risk. But it means running two gateways indefinitely, splitting the
|
|
routing table across Kong CRDs and Go code, and keeping the Kong Helm release and its CRDs.
|
|
The split is the thing most likely to drift.
|
|
|
|
**D. Go gateway, Kong retired entirely.** — chosen
|
|
One routing table, one auth implementation, one place to reason about timeouts. The logic
|
|
being replaced is small: four `request-transformer` plugins that set a body field and
|
|
rewrite a URI, one `request-termination` serving a static JSON model list, and one
|
|
`prometheus` plugin. That is on the order of a hundred lines of Go, against roughly 480
|
|
lines of YAML it retires.
|
|
|
|
## Consequences
|
|
|
|
### Gained
|
|
|
|
- **Standard OpenAI surface.** One `POST /v1/chat/completions`, model selected from the
|
|
request body. Any OpenAI SDK works unmodified. The three pi provider entries collapse to
|
|
one.
|
|
- **Working authentication.** Bearer tokens are read from the header, because it is our
|
|
code. JWKS is fetched and cached with automatic rotation handling, so the AUTH-PLAN.md
|
|
rotation runbook is deleted rather than written.
|
|
- **Application-level policy becomes possible.** GPU slot semaphore, per-session budgets,
|
|
disconnect propagation and SSE handling live where the state is.
|
|
- **One timeout story.** Kong currently sets `read-timeout: 3600000` (1 hour) on chat
|
|
routes, which silently defeats any shorter server-side cap. Retiring Kong removes the
|
|
conflicting layer.
|
|
- **~480 lines of gateway YAML deleted**, plus the Kong CRDs, the Helm release, and its
|
|
`ServerSideApply` workaround for oversized CRD annotations.
|
|
|
|
### Lost / assumed
|
|
|
|
- **We now own proxy correctness.** Connection pooling, retries, timeout propagation,
|
|
streaming passthrough, header hygiene, graceful shutdown. `net/http/httputil.ReverseProxy`
|
|
covers most of it, but it is our bug surface now.
|
|
- **Kong's Prometheus plugin goes away.** The gateway must expose equivalent metrics itself
|
|
(bandwidth, latency, status codes, upstream health) or observability regresses.
|
|
- **Migration touches live traffic.** pi depends on `api.riotpiao.com` today. Cutover must
|
|
be reversible — see `docs/MIGRATION-kong.md`.
|
|
- **`agent-pod/console` is a kong-class Ingress** exposing `/console` (WebSocket), `/run`
|
|
and `/sessions`. It must migrate too, and it is currently unauthenticated and publicly
|
|
routed while accepting free-form prompts into a shell-capable container. Putting it behind
|
|
the gateway's Authentik auth is a security improvement, not just a port.
|
|
|
|
### Risks
|
|
|
|
- Enabling Authentik auth will break any client currently relying on the unauthenticated
|
|
surface — including pi, until its `models.json` is updated. Auth must ship behind a flag
|
|
and be enabled deliberately.
|
|
- Kong's `request-termination` for `/v1/models` returns a **static** list that can drift
|
|
from what the engines actually serve. Porting it verbatim ports the bug; the gateway
|
|
should derive the list from configured upstreams instead.
|