homelab-frontend/docs/MIGRATION-kong.md

Kong retirement — inventory and cutover

Everything Kong does on api.riotpiao.com today, and where it goes. Inventory verified live against context admin@homelab-cluster on 2026-08-19.

Source of the objects being retired: ~/workplace/homelab/k8s/apps/api/ and k8s/argocd/apps/55-api-gateway.yaml.

What is running now

Kong OSS 3.4.1, Helm chart from https://charts.konghq.com, DB-less, namespace api, Argo Application kong at sync wave 7. Two replicas. Fronted by ingress-nginx via Ingress api/api, which catch-alls / on api.riotpiao.com to kong-proxy:80.

Eleven ReplicaSets exist on the Kong Deployment, the newest minutes old — this config is being actively iterated, so re-verify the inventory immediately before cutover.

Routing table to port

Seven ingressClassName: kong Ingresses. Six in llm-serving, one in agent-pod.

Method	Path	Upstream	Transform applied by Kong
GET	/v1/models	—	request-termination: static 200 JSON, upstream never contacted
POST	/v1/reasoning/chat/completions	reasoning-predictor:80	force body model=reasoning, rewrite URI to /v1/chat/completions
POST	/v1/ornith/chat/completions	ornith-predictor:80	force body model=ornith:35b, rewrite URI
POST	/v1/qwen/chat/completions	ornith-predictor:80	force body model=qwen2.5:3b-instruct, rewrite URI
POST	/v1/embeddings	embeddings-predictor:80	none — TEI already serves the canonical path
POST	/v1/rerank	reranker-predictor:80	rewrite URI to /rerank (TEI does not serve /v1/rerank)
GET/WS	/console, /run, /sessions	agent-hub:9090 (agent-pod ns)	none, strip-path: false

Upstream model map, from the manifest comments and confirmed live:

• reasoning → reasoning-predictor — vLLM, DeepSeek-R1-Distill-Qwen-32B, 2 replicas, --max-num-seqs=4, --max-model-len=16384, --reasoning-parser=deepseek_r1, --enable-auto-tool-choice --tool-call-parser=hermes
• ornith:35b → ornith-predictor — Ollama, 2 replicas
• qwen2.5:3b-instruct → ornith-predictor — same pods; both models stay resident via OLLAMA_MAX_LOADED_MODELS=2, OLLAMA_KEEP_ALIVE=-1
• nomic-ai/nomic-embed-text-v2-moe → embeddings-predictor — TEI
• BAAI/bge-reranker-base → reranker-predictor — TEI

The path-per-model surface goes away

The three chat paths exist only because Kong OSS cannot dispatch on the request body. The gateway serves a single POST /v1/chat/completions and selects the upstream from the body's model field.

Keep the old paths as aliases during cutover so live clients do not break, then remove them once callers have migrated. pi is a live caller today.

/v1/models should not be ported verbatim

Kong serves a hardcoded list via request-termination. The manifest already flags that it can drift from what the engines actually serve. Derive the response from the gateway's configured upstream map instead, so the list cannot disagree with what routing will accept.

Plugins being retired

Plugin	Scope	Replacement
llm-rewrite-reasoning / -ornith / -qwen	llm-serving	body-based dispatch in internal/llm
llm-rewrite-rerank	llm-serving	per-upstream path rewrite in the route table
llm-models-list	llm-serving	derived from the upstream map
prometheus	cluster-wide	internal/observability — must expose bandwidth, latency, status codes, upstream health or observability regresses

No rate-limiting plugin exists anywhere in the cluster. REQUIREMENTS.md §4 Tier 2 describes it as an existing layer; it is not built. Nothing to migrate — it is net new work, and it now belongs in the gateway rather than in Kong.

Auth: currently off, must land on

KongConsumer model-invoker exists in namespace api and stays defined, but the key-auth plugin is commented out and every route has model-key-auth stripped from its konghq.com/plugins annotation.

The model API is unauthenticated right now. Confirmed live 2026-08-19: a request to /v1/reasoning/chat/completions with no credentials returns 200.

The reason is recorded in model-auth.yaml — Kong's key-auth accepts a raw apikey: header but rejects Authorization: Bearer, which blocks every OpenAI-compatible client. That is why ~/.pi/agent/models.json carries a customHeaders: {apikey: ...} block.

The gateway reads Bearer tokens directly and validates them against Authentik via JWKS. AUTH-PLAN.md's pinned-RSA-key approach and its rotation runbook are not needed and should not be carried over.

Ship auth behind a flag. Turning it on breaks every current caller until they hold a token — pi included.

Timeouts

Kong today:

Route class	connect	read	write
chat	10s	1h	1h
embeddings / rerank	10s	10m	10m

nginx in front sets proxy-read-timeout: 3600, proxy-send-timeout: 3600, proxy-buffering: off, proxy-body-size: 0. Those stay — they are what makes token streaming work, and the gateway needs the same treatment from nginx.

The 1-hour read timeout is deliberate: a 32B model on a Volta GPU routinely exceeds 60s. Any shorter server-side cap must be enforced in the gateway, not by shortening the proxy timeout, or long legitimate generations get truncated mid-stream.

Cutover

Reversible at every step. Kong keeps serving until the last step.

1. Deploy the gateway alongside Kong, unexposed. Verify in-cluster against http://homelab-frontend.api.svc.cluster.local.
2. Compare gateway and Kong responses for every route in the table above, including a streaming chat request and a client disconnect mid-stream.
3. Repoint Ingress api/api from kong-proxy:80 to the gateway Service. This is the cutover. Reverting is a one-line change to the same Ingress.
4. Soak. Watch gateway metrics and pi traffic.
5. Delete the seven kong-class Ingresses and the six KongPlugin CRs.
6. Remove the kong Application from k8s/argocd/apps/55-api-gateway.yaml; let Argo prune the Helm release, the CRDs and namespace leftovers.

Steps 1–4 are reversible in seconds. Step 5 onward is not — do not start it until the soak is clean.

All of this flows through git and Argo. No kubectl apply, no helm upgrade.

Loose ends

• agent-pod/console is publicly routed, unauthenticated, accepts free-form prompts into a shell-capable container, and exposes a WebSocket. Migrating it behind the gateway's auth is a security fix, not merely a port. Treat WebSocket upgrade as an explicit requirement of the proxy layer.
• Eight *.example.com hosts exist on istio-class Ingresses in llm-serving ({embeddings,ornith,reasoning,reranker}[-predictor]-llm-serving.example.com). KServe defaults, not public, not Kong's — out of scope here, but they exist and should not be mistaken for gateway routes.
• Ingress class split across the cluster is 7 kong / 17 nginx / 4 istio. Only the 7 kong ones are in scope.