homelab-frontend/REQUIREMENTS.md

homelab-frontend — Requirements

The contract for the Go API gateway that replaces Kong OSS on *.riotpiao.com.

Companion documents:

• docs/adr/ADR-0001-retire-kong-for-go-gateway.md — why Kong is being retired
• docs/MIGRATION-kong.md — exact inventory of what Kong does today and the cutover order
• tasks/INDEX.md — the task board

All cluster facts below were verified live against context admin@homelab-cluster on 2026-08-19. Re-verify before relying on any number.

---

0. Invariants

These hold for every surface. A change that breaks one of these is a design change, not an implementation detail.

• G1 — ingress-nginx owns TLS and the edge. The gateway never terminates TLS.
• G2 — The gateway holds no Kubernetes credentials. It proxies to services that do. Cluster-read permissions stay in atlas, out of the public edge process.
• G3 — Public surfaces use standard protocol shapes. If an OpenAI SDK cannot call it unmodified, the design is wrong.
• G4 — Streaming is unbuffered end to end, and a client disconnect cancels the upstream request rather than orphaning it.
• G5 — Authentication is Bearer-token, validated against Authentik via JWKS fetched at runtime. No pinned public keys, no rotation runbook.
• G6 — Every route's timeout, body cap and concurrency limit is explicit in configuration. No silent defaults.
• G7 — All deployment flows through git and Argo. No kubectl apply, no helm upgrade, no local terraform apply.

---

1. Runtime and configuration

1.1 Process

Single static Go binary. Reads configuration at startup, serves HTTP, exits cleanly on SIGTERM after draining in-flight requests.

Must run with no cluster, no kubeconfig and no credentials so that behaviour can be verified in a closed loop before touching live traffic. Upstreams are configuration; pointing them at local stubs is the entire mechanism. This is a hard requirement, not a convenience — see §7.

1.2 Configuration

Route and upstream configuration is declarative and loaded at startup. It must express, per upstream: address, path rewrite, connect/read/write timeouts, maximum body size, and whether the route requires authentication.

Configuration errors fail startup loudly. A gateway that starts with a silently dropped route is worse than one that refuses to start.

Configuration lives in git, mounted as a ConfigMap and synced by Argo. Not a CRD. A CRD would require the gateway to watch the API server, which needs RBAC and contradicts G2 — and CRD-driven routing is precisely the indirection being retired with Kong, where the routing table was split across six KongPlugin CRs, seven Ingresses and a Helm values file.

A CRD earns its keep when someone other than the repo owner must register routes. That is not true here. If it becomes true, the additive answer is a controller that renders this same ConfigMap — the gateway stays credential-free either way.

1.3 Health

• GET /healthz — liveness, no upstream checks, always cheap.
• GET /readyz — readiness; may fail while configuration is invalid or JWKS has never been successfully fetched.

Neither requires authentication.

---

2. Proxy core

2.1 Reverse proxying

Standard reverse proxy to configured upstreams. Connection reuse across requests. Hop-by-hop headers stripped correctly. X-Forwarded-* set from the nginx-supplied values, not fabricated.

2.2 Streaming

SSE and chunked responses pass through without buffering. Tokens must reach the client as the upstream emits them, not on completion.

WebSocket upgrade must work — agent-pod/console depends on it.

2.3 Disconnect propagation

When a client disconnects, the upstream request is cancelled immediately. This is load-bearing: an orphaned generation holds a vLLM sequence slot, and there are only eight in the cluster.

2.4 Timeouts

Per-route, explicit. Current Kong values, which are deliberate and must be preserved unless changed knowingly:

Route class	connect	read	write
chat	10s	1h	1h
embeddings, rerank	10s	10m	10m

The 1-hour read timeout exists because a 32B model on a Volta GPU routinely exceeds 60s. Any shorter application-level cap must be enforced by the gateway's own logic, not by shortening the proxy timeout — otherwise long legitimate generations truncate mid-stream.

---

3. LLM surfaces — api.riotpiao.com

Two protocol dialects, permanently. Both translate into one dialect-neutral canonical request, and both pass through one shared slot controller before reaching a predictor.

Prefix	Dialect	Primary client
/v1/*	OpenAI-compatible	pi, generic OpenAI SDKs
/llm/*	Anthropic Messages	the riotpiao frontend (first-party only)

  /v1/*  (OpenAI)        /llm/*  (Anthropic)
        |                        |
        +-----------+------------+
                    v
            canonical request        dialect-neutral
                    v
             slot controller         keyed by UPSTREAM, not by route
                    v
        reasoning-predictor / ornith-predictor

The slot controller is keyed by upstream and shared across dialects. Per-dialect semaphores are wrong: the 8 sequence slots are physical, so two independent gates would each believe they were within budget while together exceeding it. Requests from both surfaces contend for the same slots and the same queue, in arrival order.

Dispatch, budgets, logging and metrics all operate on the canonical request. Adding a third dialect later must not require touching the controller.

3.1 Body-based model dispatch

POST /v1/chat/completions selects its upstream from the request body's model field. This is the single most important requirement in this document: it is the capability Kong OSS lacked, and the reason the gateway exists.

Unknown or missing model is a client error with a useful message listing valid values — not a 500, and not a silent fallback to a default model.

3.2 Upstream map

Verified live. served-model-name values are what clients send.

model in body	Upstream Service	Engine
reasoning	reasoning-predictor.llm-serving:80	vLLM, DeepSeek-R1-Distill-Qwen-32B
ornith:35b	ornith-predictor.llm-serving:80	Ollama
qwen2.5:3b-instruct	ornith-predictor.llm-serving:80	Ollama, same pods
nomic-ai/nomic-embed-text-v2-moe	embeddings-predictor.llm-serving:80	TEI
BAAI/bge-reranker-base	reranker-predictor.llm-serving:80	TEI

reasoning runs 2 replicas × --max-num-seqs=4 = 8 concurrent sequence slots total, --max-model-len=16384, --reasoning-parser=deepseek_r1, --enable-auto-tool-choice --tool-call-parser=hermes.

Note ornith:35b and qwen2.5:3b-instruct share pods; both stay resident via OLLAMA_MAX_LOADED_MODELS=2 and OLLAMA_KEEP_ALIVE=-1, so dispatching between them does not trigger a model swap.

3.3 Path rewriting

Upstreams expect canonical paths. /v1/chat/completions and /v1/embeddings pass through unchanged. Rerank is the exception: TEI serves /rerank, not /v1/rerank, so that route rewrites.

3.4 Legacy path aliases

/v1/{reasoning,ornith,qwen}/chat/completions must keep working during cutover — pi is a live caller. They behave exactly as the canonical endpoint with model forced to the corresponding value, overriding whatever the body says.

These are temporary. They exist to make the cutover reversible, and are removed once callers have migrated.

3.5 GET /v1/models

Derived from the configured upstream map, never hardcoded. Kong served a static list, and its own manifest flags that the list can drift from what the engines actually serve. The gateway's list must be incapable of disagreeing with what routing will accept.

OpenAI list shape: {"object":"list","data":[{"id","object":"model","owned_by","created"}]}.

3.6 Behaviour to preserve

Verified against the live endpoint:

• The upstream returns reasoning_content separately from content for the reasoning model. Pass both through untouched.
• Tool calling works with explicit tool_choice, and is unreliable with tool_choice: auto on the R1-distill model. The gateway does not compensate for this — it is a model property, not a gateway concern. Do not add retries or rewriting to work around it.

---

4. Authentication — Authentik

4.1 Current state

The model API is unauthenticated today. Confirmed live: /v1/reasoning/chat/completions answers with no credentials.

Kong's key-auth was retired because it accepts a raw apikey: header but rejects Authorization: Bearer, which hard-blocks every OpenAI-compatible client. See ~/workplace/homelab/k8s/apps/api/model-auth.yaml.

4.2 Requirement

Bearer tokens in Authorization, validated against Authentik (https://authentik.riotpiao.com) by fetching and caching JWKS at runtime.

Key rotation must be handled by refetching JWKS, not by pinned PEMs. The pinned-rsa_public_key approach in AUTH-PLAN.md and its rotation runbook exist only to route around a Kong OSS limitation and must not be carried forward.

Service accounts obtain tokens via client_credentials against Authentik's token endpoint.

4.3 Rollout

Auth ships behind a flag, defaulting off, and is enabled deliberately.

Enabling it breaks every current caller until they hold a token — pi included, whose models.json currently sends a customHeaders: {apikey: ...} block that will need replacing with a Bearer token.

4.4 Authorization

Beyond authentication, a token must be checked for the right to invoke the capability it is calling. A token minted for queue access should not invoke a GPU.

---

5. Rate limiting and budgets

No rate-limiting plugin exists anywhere in the cluster today — this is net new work, not a migration. Verified: six Kong plugins exist, none is rate-limiting.

Requirements, in priority order:

1. GPU slot protection. reasoning has 8 total sequence slots. Concurrent in-flight requests to it must be capped below that, leaving operator headroom. Excess requests queue up to a bounded depth, then are rejected with a retryable status.
2. Per-caller budgets. Identified callers get a request budget over a window.
3. Body size caps, per route.

Rejections use RFC 9457 application/problem+json and set Retry-After where a retry time is knowable.

---

6. Observability

Kong's cluster-wide prometheus plugin is being retired. The gateway must expose at least equivalent signal or observability regresses at cutover: request rate, latency, status codes, bandwidth, and upstream health, labelled by route and upstream.

Gateway-specific signals that Kong could not provide, and which are the reason for several requirements above: in-flight requests per upstream, queue depth, GPU slot occupancy, and rejections by reason.

Structured logging. Every rejected request is logged with the reason. No secrets, no tokens, no request bodies in logs.

---

7. Local development and verification

An agent must be able to close a change/verify loop with no cluster, no kubeconfig and no credentials. This is a hard requirement because it determines whether work can proceed unattended.

Concretely: it must be possible to start the gateway locally, point it at stub upstreams, issue requests, and assert on the responses — including streaming responses and client disconnects.

Verification of any API-shaped task means asserting on the actual HTTP response: status, headers, and body. "It compiles" and "it starts" are not verification.

Parity with Kong is verified by comparing gateway and Kong responses for the same request, for every route in the migration inventory, before cutover.

---

8. Deployment

Container: distroless or scratch, runAsNonRoot, read-only root filesystem, all capabilities dropped, seccompProfile: RuntimeDefault, no shell.

Image tags are commit SHAs, never :latest — Argo's selfHeal cannot roll out a mutable tag reliably.

NetworkPolicy: egress only to the upstreams it proxies plus Authentik; ingress from ingress-nginx only.

Deployed as an Argo Application in the homelab-root GitOps repo. Verified live: zero Argo Applications anywhere in the cluster source from any Forgejo URL, so github.com/Riotpiaole/riotpiao.homelab.com is authoritative.

---

9. Capability surface — path-based

Every capability is a path prefix on the single host api.riotpiao.com. One DNS record, one Cloudflare tunnel hostname, one nginx Ingress, one Service.

Prefix	Backs onto	Status
/v1/*	llm-serving predictors	v1 — reserved, see below
/sqs/*	kmsvc management-service, Kafka/Strimzi (sqs ns)	future
/workflow/*	Temporal (temporal ns)	future
/cluster/*	atlas, separate repo riotpiao-backend	future
/db/*	CloudNativePG, MinIO, monitoring reads	future

/v1/* is reserved for the OpenAI-compatible surface and nothing else. G3 pins it: an SDK expects /v1/chat/completions at the base URL, so that prefix can never be repurposed or nested. Every other capability gets its own prefix that cannot collide with a current or future OpenAI path.

Subdomains are deliberately not used. Paths keep hostname configuration to a single entry — and hostname configuration is the demonstrated failure mode here, as the unresolved apex 403 shows. Promoting a prefix to its own subdomain later is an additive host rule that can run alongside the path; the reverse is not, because clients hardcode hostnames.

Notes carried from the cluster:

• Temporal namespace registration is automatic via queue-operator, never manual.
• management-service already exposes gRPC at kmsvc.riotpiao.com; the /sqs prefix is a new surface, not a replacement for it.
• atlas keeps its own informers and RBAC. The gateway proxies to it and holds no cluster credentials of its own (G2).
• /db/* read surfaces need particular care — see G2 before designing them.

---

10. Known cluster facts worth not rediscovering

• kmsvc-redis-master.sqs:6379 has no authentication — ALLOW_EMPTY_PASSWORD=yes, TLS off. Any workload with network reach has full unauthenticated read/write. A NetworkPolicy is the only control.
• reasoning-predictor listens on port 80, not 8080.
• prometheus-operated.monitoring is headless (ClusterIP None) — egress policies need pod selectors, not ClusterIPs.
• agent-pod/console is publicly routed, unauthenticated, accepts free-form prompts into a shell-capable container, and serves a WebSocket. Putting it behind gateway auth is a security fix, not merely a port.
• Eight *.example.com hosts exist on istio-class Ingresses in llm-serving. KServe defaults, not public, out of scope — do not mistake them for gateway routes.