# homelab-frontend — Requirements The contract for the Go API gateway that replaces Kong OSS on `*.riotpiao.com`. Companion documents: - [docs/adr/ADR-0001-retire-kong-for-go-gateway.md](docs/adr/ADR-0001-retire-kong-for-go-gateway.md) — why Kong is being retired - [docs/MIGRATION-kong.md](docs/MIGRATION-kong.md) — exact inventory of what Kong does today and the cutover order - [tasks/INDEX.md](tasks/INDEX.md) — the task board All cluster facts below were verified live against context `admin@homelab-cluster` on 2026-08-19. Re-verify before relying on any number. --- ## 0. Invariants These hold for every surface. A change that breaks one of these is a design change, not an implementation detail. - **G1** — ingress-nginx owns TLS and the edge. The gateway never terminates TLS. - **G2** — The gateway holds no Kubernetes credentials. It proxies to services that do. Cluster-read permissions stay in atlas, out of the public edge process. - **G3** — Public surfaces use standard protocol shapes. If an OpenAI SDK cannot call it unmodified, the design is wrong. - **G4** — Streaming is unbuffered end to end, and a client disconnect cancels the upstream request rather than orphaning it. - **G5** — Authentication is Bearer-token, validated against Authentik via JWKS fetched at runtime. No pinned public keys, no rotation runbook. - **G6** — Every route's timeout, body cap and concurrency limit is explicit in configuration. No silent defaults. - **G7** — All deployment flows through git and Argo. No `kubectl apply`, no `helm upgrade`, no local `terraform apply`. --- ## 1. Runtime and configuration ### 1.1 Process Single static Go binary. Reads configuration at startup, serves HTTP, exits cleanly on SIGTERM after draining in-flight requests. Must run with **no cluster, no kubeconfig and no credentials** so that behaviour can be verified in a closed loop before touching live traffic. Upstreams are configuration; pointing them at local stubs is the entire mechanism. This is a hard requirement, not a convenience — see §7. ### 1.2 Configuration Route and upstream configuration is declarative and loaded at startup. It must express, per upstream: address, path rewrite, connect/read/write timeouts, maximum body size, and whether the route requires authentication. Configuration errors fail startup loudly. A gateway that starts with a silently dropped route is worse than one that refuses to start. **Configuration lives in git**, mounted as a ConfigMap and synced by Argo. Not a CRD. A CRD would require the gateway to watch the API server, which needs RBAC and contradicts G2 — and CRD-driven routing is precisely the indirection being retired with Kong, where the routing table was split across six `KongPlugin` CRs, seven Ingresses and a Helm values file. A CRD earns its keep when someone other than the repo owner must register routes. That is not true here. If it becomes true, the additive answer is a controller that renders this same ConfigMap — the gateway stays credential-free either way. ### 1.3 Health - `GET /healthz` — liveness, no upstream checks, always cheap. - `GET /readyz` — readiness; may fail while configuration is invalid or JWKS has never been successfully fetched. Neither requires authentication. --- ## 2. Proxy core ### 2.1 Reverse proxying Standard reverse proxy to configured upstreams. Connection reuse across requests. Hop-by-hop headers stripped correctly. `X-Forwarded-*` set from the nginx-supplied values, not fabricated. ### 2.2 Streaming SSE and chunked responses pass through without buffering. Tokens must reach the client as the upstream emits them, not on completion. WebSocket upgrade must work — `agent-pod/console` depends on it. ### 2.3 Disconnect propagation When a client disconnects, the upstream request is cancelled immediately. This is load-bearing: an orphaned generation holds a vLLM sequence slot, and there are only eight in the cluster. ### 2.4 Timeouts Per-route, explicit. Current Kong values, which are deliberate and must be preserved unless changed knowingly: | Route class | connect | read | write | |---|---|---|---| | chat | 10s | 1h | 1h | | embeddings, rerank | 10s | 10m | 10m | The 1-hour read timeout exists because a 32B model on a Volta GPU routinely exceeds 60s. Any shorter application-level cap must be enforced *by the gateway's own logic*, not by shortening the proxy timeout — otherwise long legitimate generations truncate mid-stream. --- ## 3. LLM surfaces — `api.riotpiao.com` Two protocol dialects, permanently. Both translate into one dialect-neutral canonical request, and both pass through **one shared slot controller** before reaching a predictor. | Prefix | Dialect | Primary client | |---|---|---| | `/v1/*` | OpenAI-compatible | pi, generic OpenAI SDKs | | `/llm/*` | Anthropic Messages | the riotpiao frontend (first-party only) | ``` /v1/* (OpenAI) /llm/* (Anthropic) | | +-----------+------------+ v canonical request dialect-neutral v slot controller keyed by UPSTREAM, not by route v reasoning-predictor / ornith-predictor ``` **The slot controller is keyed by upstream and shared across dialects.** Per-dialect semaphores are wrong: the 8 sequence slots are physical, so two independent gates would each believe they were within budget while together exceeding it. Requests from both surfaces contend for the same slots and the same queue, in arrival order. Dispatch, budgets, logging and metrics all operate on the canonical request. Adding a third dialect later must not require touching the controller. ### 3.1 Body-based model dispatch `POST /v1/chat/completions` selects its upstream from the request body's `model` field. This is the single most important requirement in this document: it is the capability Kong OSS lacked, and the reason the gateway exists. Unknown or missing `model` is a client error with a useful message listing valid values — not a 500, and not a silent fallback to a default model. ### 3.2 Upstream map Verified live. `served-model-name` values are what clients send. | `model` in body | Upstream Service | Engine | |---|---|---| | `reasoning` | `reasoning-predictor.llm-serving:80` | vLLM, DeepSeek-R1-Distill-Qwen-32B | | `ornith:35b` | `ornith-predictor.llm-serving:80` | Ollama | | `qwen2.5:3b-instruct` | `ornith-predictor.llm-serving:80` | Ollama, same pods | | `nomic-ai/nomic-embed-text-v2-moe` | `embeddings-predictor.llm-serving:80` | TEI | | `BAAI/bge-reranker-base` | `reranker-predictor.llm-serving:80` | TEI | `reasoning` runs 2 replicas × `--max-num-seqs=4` = **8 concurrent sequence slots total**, `--max-model-len=16384`, `--reasoning-parser=deepseek_r1`, `--enable-auto-tool-choice --tool-call-parser=hermes`. Note `ornith:35b` and `qwen2.5:3b-instruct` share pods; both stay resident via `OLLAMA_MAX_LOADED_MODELS=2` and `OLLAMA_KEEP_ALIVE=-1`, so dispatching between them does not trigger a model swap. ### 3.3 Path rewriting Upstreams expect canonical paths. `/v1/chat/completions` and `/v1/embeddings` pass through unchanged. Rerank is the exception: TEI serves `/rerank`, not `/v1/rerank`, so that route rewrites. ### 3.4 Legacy path aliases `/v1/{reasoning,ornith,qwen}/chat/completions` must keep working during cutover — pi is a live caller. They behave exactly as the canonical endpoint with `model` forced to the corresponding value, overriding whatever the body says. These are temporary. They exist to make the cutover reversible, and are removed once callers have migrated. ### 3.5 `GET /v1/models` Derived from the configured upstream map, never hardcoded. Kong served a static list, and its own manifest flags that the list can drift from what the engines actually serve. The gateway's list must be incapable of disagreeing with what routing will accept. OpenAI list shape: `{"object":"list","data":[{"id","object":"model","owned_by","created"}]}`. ### 3.6 Behaviour to preserve Verified against the live endpoint: - The upstream returns `reasoning_content` separately from `content` for the `reasoning` model. Pass both through untouched. - Tool calling works with explicit `tool_choice`, and is unreliable with `tool_choice: auto` on the R1-distill model. The gateway does not compensate for this — it is a model property, not a gateway concern. Do not add retries or rewriting to work around it. --- ## 4. Authentication — Authentik ### 4.1 Current state **The model API is unauthenticated today.** Confirmed live: `/v1/reasoning/chat/completions` answers with no credentials. Kong's `key-auth` was retired because it accepts a raw `apikey:` header but rejects `Authorization: Bearer`, which hard-blocks every OpenAI-compatible client. See `~/workplace/homelab/k8s/apps/api/model-auth.yaml`. ### 4.2 Requirement Bearer tokens in `Authorization`, validated against Authentik (`https://authentik.riotpiao.com`) by fetching and caching JWKS at runtime. Key rotation must be handled by refetching JWKS, not by pinned PEMs. The pinned-`rsa_public_key` approach in `AUTH-PLAN.md` and its rotation runbook exist only to route around a Kong OSS limitation and must not be carried forward. Service accounts obtain tokens via `client_credentials` against Authentik's token endpoint. ### 4.3 Rollout Auth ships behind a flag, defaulting off, and is enabled deliberately. Enabling it breaks every current caller until they hold a token — pi included, whose `models.json` currently sends a `customHeaders: {apikey: ...}` block that will need replacing with a Bearer token. ### 4.4 Authorization Beyond authentication, a token must be checked for the right to invoke the capability it is calling. A token minted for queue access should not invoke a GPU. --- ## 5. Rate limiting and budgets No `rate-limiting` plugin exists anywhere in the cluster today — this is net new work, not a migration. Verified: six Kong plugins exist, none is `rate-limiting`. Requirements, in priority order: 1. **GPU slot protection.** `reasoning` has 8 total sequence slots. Concurrent in-flight requests to it must be capped below that, leaving operator headroom. Excess requests queue up to a bounded depth, then are rejected with a retryable status. 2. **Per-caller budgets.** Identified callers get a request budget over a window. 3. **Body size caps**, per route. Rejections use RFC 9457 `application/problem+json` and set `Retry-After` where a retry time is knowable. --- ## 6. Observability Kong's cluster-wide `prometheus` plugin is being retired. The gateway must expose at least equivalent signal or observability regresses at cutover: request rate, latency, status codes, bandwidth, and upstream health, labelled by route and upstream. Gateway-specific signals that Kong could not provide, and which are the reason for several requirements above: in-flight requests per upstream, queue depth, GPU slot occupancy, and rejections by reason. Structured logging. Every rejected request is logged with the reason. No secrets, no tokens, no request bodies in logs. --- ## 7. Local development and verification An agent must be able to close a change/verify loop with no cluster, no kubeconfig and no credentials. This is a hard requirement because it determines whether work can proceed unattended. Concretely: it must be possible to start the gateway locally, point it at stub upstreams, issue requests, and assert on the responses — including streaming responses and client disconnects. Verification of any API-shaped task means asserting on the **actual HTTP response**: status, headers, and body. "It compiles" and "it starts" are not verification. Parity with Kong is verified by comparing gateway and Kong responses for the same request, for every route in the migration inventory, before cutover. --- ## 8. Deployment Container: distroless or scratch, `runAsNonRoot`, read-only root filesystem, all capabilities dropped, `seccompProfile: RuntimeDefault`, no shell. Image tags are commit SHAs, never `:latest` — Argo's `selfHeal` cannot roll out a mutable tag reliably. NetworkPolicy: egress only to the upstreams it proxies plus Authentik; ingress from `ingress-nginx` only. Deployed as an Argo Application in the `homelab-root` GitOps repo. Verified live: zero Argo Applications anywhere in the cluster source from any Forgejo URL, so `github.com/Riotpiaole/riotpiao.homelab.com` is authoritative. --- ## 9. Capability surface — path-based Every capability is a path prefix on the single host `api.riotpiao.com`. One DNS record, one Cloudflare tunnel hostname, one nginx Ingress, one Service. | Prefix | Backs onto | Status | |---|---|---| | `/v1/*` | `llm-serving` predictors | v1 — **reserved**, see below | | `/sqs/*` | kmsvc management-service, Kafka/Strimzi (`sqs` ns) | future | | `/workflow/*` | Temporal (`temporal` ns) | future | | `/cluster/*` | atlas, separate repo `riotpiao-backend` | future | | `/db/*` | CloudNativePG, MinIO, monitoring reads | future | **`/v1/*` is reserved for the OpenAI-compatible surface and nothing else.** G3 pins it: an SDK expects `/v1/chat/completions` at the base URL, so that prefix can never be repurposed or nested. Every other capability gets its own prefix that cannot collide with a current or future OpenAI path. Subdomains are deliberately *not* used. Paths keep hostname configuration to a single entry — and hostname configuration is the demonstrated failure mode here, as the unresolved apex 403 shows. Promoting a prefix to its own subdomain later is an additive host rule that can run alongside the path; the reverse is not, because clients hardcode hostnames. Notes carried from the cluster: - Temporal namespace registration is automatic via queue-operator, never manual. - `management-service` already exposes gRPC at `kmsvc.riotpiao.com`; the `/sqs` prefix is a new surface, not a replacement for it. - atlas keeps its own informers and RBAC. The gateway proxies to it and holds no cluster credentials of its own (G2). - `/db/*` read surfaces need particular care — see G2 before designing them. --- ## 10. Known cluster facts worth not rediscovering - `kmsvc-redis-master.sqs:6379` has **no authentication** — `ALLOW_EMPTY_PASSWORD=yes`, TLS off. Any workload with network reach has full unauthenticated read/write. A NetworkPolicy is the only control. - `reasoning-predictor` listens on port **80**, not 8080. - `prometheus-operated.monitoring` is **headless** (ClusterIP None) — egress policies need pod selectors, not ClusterIPs. - `agent-pod/console` is publicly routed, unauthenticated, accepts free-form prompts into a shell-capable container, and serves a WebSocket. Putting it behind gateway auth is a security fix, not merely a port. - Eight `*.example.com` hosts exist on istio-class Ingresses in `llm-serving`. KServe defaults, not public, out of scope — do not mistake them for gateway routes.