homelab-frontend/docs/adr/ADR-0001-retire-kong-for-go-gateway.md

ADR-0001 — Retire Kong OSS in favour of a Go API gateway

Status: Accepted Date: 2026-08-19 Deciders: rock

Context

api.riotpiao.com is currently served by Kong OSS 3.4.1 (Helm, DB-less, namespace api, Argo wave 7), sitting behind ingress-nginx which owns TLS. Kong routes to the KServe model predictors in llm-serving via seven ingressClassName: kong Ingresses and six KongPlugin CRs.

Three separate capabilities were attempted on Kong OSS. All three failed, and each failure is already documented in-repo by the person who hit it:

1. Body-based model dispatch is not expressible. From k8s/apps/api/llm-routes.yaml:

a single /v1/chat/completions endpoint that dispatches on the body's model field is not expressible in Kong OSS (ai-proxy-advanced, which does multi-target model routing, is Enterprise-only).

The workaround is a path-per-model surface (/v1/reasoning/chat/completions, /v1/ornith/..., /v1/qwen/...) with a request-transformer force-overwriting the body's model field. This is not OpenAI-standard, so every client needs bespoke configuration — visible today in ~/.pi/agent/models.json, which carries three separate provider entries for what should be one endpoint.

2. OIDC is Enterprise-only. k8s/apps/api/AUTH-PLAN.md routes around the missing openid-connect plugin using the built-in jwt plugin, which requires pinning Authentik's RSA public key onto a KongConsumer. That plan lists its own consequence:

Pinning rsa_public_key: Authentik key rotation would break it — document a rotation runbook, or have the provision script re-export the cert PEM into the Kong credential on each run.

A rotation runbook is a standing operational liability accepted only because the gateway cannot fetch JWKS itself.

3. key-auth cannot read Authorization: Bearer. From k8s/apps/api/model-auth.yaml:

a raw apikey: <key> header succeeds (200), the same request with only Authorization: Bearer <key> fails (401). No OpenAI-SDK-compatible client (pi included) sends a raw apikey header or lets you customize the header name, so every such client was hard-blocked.

Consequence: authentication on the model routes is currently disabled. Verified live 2026-08-19 — api.riotpiao.com/v1/reasoning/chat/completions answers unauthenticated.

Separately, the intended surface has grown beyond LLM routing. The target is a capability-per-subdomain API over cluster services — sqs.riotpiao.com for queue operations, workflow.riotpiao.com for Temporal, cluster.riotpiao.com for atlas — each needing request shaping, per-caller budgets and streaming semantics that are application concerns, not gateway-plugin concerns.

Decision

Retire Kong OSS entirely. Replace it with a purpose-built Go service, homelab-frontend, which owns north-south routing, authentication, and request shaping for every public capability on *.riotpiao.com.

ingress-nginx keeps the edge and TLS. It forwards to the gateway instead of kong-proxy.

Authentication is Authentik OIDC, validated by fetching JWKS from https://authentik.riotpiao.com at runtime.

Options considered

A. Stay on Kong OSS, accept the workarounds. Keeps a battle-tested proxy and its Prometheus plugin. But the path-per-model surface stays non-standard, the RSA pinning runbook stays, and auth stays off until someone writes a request-transformer shim to copy Bearer into an apikey header. Every new capability (sqs, workflow) inherits the same constraints.

B. Buy Kong Enterprise. ai-proxy-advanced and openid-connect solve 1 and 2. Does not solve the genuinely application-level requirements at all — signed session cookies, per-session daily message budgets, a 6-of-8 GPU sequence-slot semaphore with a bounded queue, and disconnect-cancels-upstream are not gateway features in any tier. Cost for a homelab is not justifiable.

C. Go gateway, Kong retained for LLM paths only. Gradual migration, lower risk. But it means running two gateways indefinitely, splitting the routing table across Kong CRDs and Go code, and keeping the Kong Helm release and its CRDs. The split is the thing most likely to drift.

D. Go gateway, Kong retired entirely. — chosen One routing table, one auth implementation, one place to reason about timeouts. The logic being replaced is small: four request-transformer plugins that set a body field and rewrite a URI, one request-termination serving a static JSON model list, and one prometheus plugin. That is on the order of a hundred lines of Go, against roughly 480 lines of YAML it retires.

Consequences

Gained

• Standard OpenAI surface. One POST /v1/chat/completions, model selected from the request body. Any OpenAI SDK works unmodified. The three pi provider entries collapse to one.
• Working authentication. Bearer tokens are read from the header, because it is our code. JWKS is fetched and cached with automatic rotation handling, so the AUTH-PLAN.md rotation runbook is deleted rather than written.
• Application-level policy becomes possible. GPU slot semaphore, per-session budgets, disconnect propagation and SSE handling live where the state is.
• One timeout story. Kong currently sets read-timeout: 3600000 (1 hour) on chat routes, which silently defeats any shorter server-side cap. Retiring Kong removes the conflicting layer.
• ~480 lines of gateway YAML deleted, plus the Kong CRDs, the Helm release, and its ServerSideApply workaround for oversized CRD annotations.

Lost / assumed

• We now own proxy correctness. Connection pooling, retries, timeout propagation, streaming passthrough, header hygiene, graceful shutdown. net/http/httputil.ReverseProxy covers most of it, but it is our bug surface now.
• Kong's Prometheus plugin goes away. The gateway must expose equivalent metrics itself (bandwidth, latency, status codes, upstream health) or observability regresses.
• Migration touches live traffic. pi depends on api.riotpiao.com today. Cutover must be reversible — see docs/MIGRATION-kong.md.
• agent-pod/console is a kong-class Ingress exposing /console (WebSocket), /run and /sessions. It must migrate too, and it is currently unauthenticated and publicly routed while accepting free-form prompts into a shell-capable container. Putting it behind the gateway's Authentik auth is a security improvement, not just a port.

Risks

• Enabling Authentik auth will break any client currently relying on the unauthenticated surface — including pi, until its models.json is updated. Auth must ship behind a flag and be enabled deliberately.
• Kong's request-termination for /v1/models returns a static list that can drift from what the engines actually serve. Porting it verbatim ports the bug; the gateway should derive the list from configured upstreams instead.