Story Crater Bot
6cf5c3e5a9
http.Server.WriteTimeout is an absolute deadline over the whole response, not an inactivity timeout -- 15s was cutting off SSE streams from the reasoning model mid-generation, surfacing to clients as a "terminated" error well before the model finished. Switch to ReadHeaderTimeout (protects against slow headers without capping body duration) and raise WriteTimeout to match the edge nginx Ingress's proxy-read/send-timeout of 3600s.
homelab-frontend
A Go API gateway for the homelab cluster. One capability per subdomain, one auth implementation, one routing table.
Replaces Kong OSS entirely — see ADR-0001 for why, and docs/MIGRATION-kong.md for the cutover.
Position in the stack
browser / SDK ──▶ Cloudflare ──▶ ingress-nginx (TLS, edge)
│
▼
┌─────────────────────────────┐
│ homelab-frontend │
│ routing · authn · budgets │
└──────────────┬──────────────┘
│
/v1 /sqs /workflow /cluster
│ │ │ │
▼ ▼ ▼ ▼
llm-serving kmsvc/Kafka temporal atlas
(predictors) (sqs ns) (temporal ns) (riotpiao-backend)
└──── in-cluster Services ────┘
ingress-nginx keeps TLS and the edge. The gateway owns everything after it.
Backend services are reached through the gateway rather than published individually — a single place for authentication, budgets, timeouts and observability, and a single hostname surface to reason about.
Capability map
One host, one path prefix per capability.
Prefix on api.riotpiao.com |
Backs onto | Status |
|---|---|---|
/v1/* |
llm-serving predictors (vLLM, Ollama, TEI) |
migrating off Kong |
/sqs/* |
kmsvc management-service + Kafka/Strimzi (sqs ns) |
future |
/workflow/* |
Temporal (temporal ns) |
future |
/cluster/* |
atlas — cluster topology / Argo delivery (separate repo) | future |
/db/* |
CloudNativePG, MinIO, monitoring/metrics reads | future |
/v1/* is reserved for the OpenAI-compatible surface. An SDK expects
/v1/chat/completions at the base URL, so that prefix cannot be repurposed.
Paths rather than subdomains: one DNS record, one tunnel hostname, one Ingress. Promoting a prefix to its own subdomain later is additive and can run alongside the path — the reverse is not, because clients hardcode hostnames.
atlas lives in its own repo (riotpiao-backend) and keeps its own informers and
RBAC. The gateway routes to it; it does not absorb it. Cluster-read permissions
stay out of the public edge process.
Design rules
- Standard protocol shapes.
POST /v1/chat/completionsselects its model from the request body, like every OpenAI-compatible server. No path-per-model, no bespoke client configuration. Kong OSS could not do this; that limitation does not survive into the replacement. - Bearer tokens, validated against Authentik. JWKS is fetched at runtime and cached, so key rotation needs no runbook and no pinned PEM.
- Policy lives where the state is. GPU slot semaphores, per-caller budgets, queue depth and disconnect propagation are application concerns. They belong here, not in a proxy plugin.
- Streaming is first-class. SSE and WebSocket pass through unbuffered, and a client disconnect cancels the upstream request rather than orphaning it.
- The gateway holds no cluster credentials. It proxies to services that do.
Layout
cmd/gateway/ entrypoint
internal/
auth/ Authentik OIDC, JWKS cache, service-account tokens
llm/ model registry, body-based dispatch, upstream map
queue/ sqs.riotpiao.com surface
workflow/ workflow.riotpiao.com surface
proxy/ reverse proxy, streaming, timeouts, disconnect propagation
config/ upstream + route configuration
observability/ Prometheus metrics, structured logging
deploy/
base/ Kubernetes manifests
argocd/ Argo Application
docs/adr/ architecture decision records
tasks/ task board — see tasks/INDEX.md
testdata/ fixtures for offline tests
Local development
The gateway must be runnable with no cluster, no kubeconfig and no credentials, so that changes can be verified in a closed loop before touching live traffic. Upstreams are configuration, so pointing them at local stubs is the whole mechanism. See tasks/INDEX.md.
Status
Scaffolded 2026-08-19. Nothing is wired yet. Kong is still serving live traffic on
api.riotpiao.com.