diff --git a/README.md b/README.md index 167061f..8ff71c8 100644 --- a/README.md +++ b/README.md @@ -1,104 +1,323 @@ # homelab-frontend -A Go API gateway for the homelab cluster. One capability per subdomain, one auth -implementation, one routing table. +Production API gateway for the homelab cluster. Single entry point (`api.riotpiao.com`) for all services: LLM inference, workflows, queues, memory, and cluster operations. -Replaces Kong OSS entirely — see -[ADR-0001](docs/adr/ADR-0001-retire-kong-for-go-gateway.md) for why, and -[docs/MIGRATION-kong.md](docs/MIGRATION-kong.md) for the cutover. +**Status:** Live in production. Replaced Kong OSS entirely. -## Position in the stack +--- + +## Quick Links + +- **API Reference:** See [API.md](API.md) — how to call every service +- **Base URL:** `https://api.riotpiao.com` +- **Source:** `ssh://git.riotpiao.com:2222/rock/homelab-frontend.git` + +--- + +## Architecture ``` -browser / SDK ──▶ Cloudflare ──▶ ingress-nginx (TLS, edge) - │ - ▼ - ┌─────────────────────────────┐ - │ homelab-frontend │ - │ routing · authn · budgets │ - └──────────────┬──────────────┘ - │ - /v1 /sqs /workflow /cluster - │ │ │ │ - ▼ ▼ ▼ ▼ - llm-serving kmsvc/Kafka temporal atlas - (predictors) (sqs ns) (temporal ns) (riotpiao-backend) - - └──── in-cluster Services ────┘ +┌─────────────┬──────────────┬────────────┐ +│ Browser │ SDK │ CLI │ +└──────┬──────┴──────┬───────┴────┬───────┘ + │ │ │ + └─────────────┼────────────┘ + │ + HTTPS/TLS + │ + ┌─────────────┼────────────┐ + │ Cloudflare Edge │ + │ (DDoS, caching) │ + └──────────┬──────────────┘ + │ + ingress-nginx + (SSL termination) + │ + ┌─────────────────────────────┐ + │ homelab-frontend Gateway │ + │ (routing, auth, limits) │ + └──────┬───────────────────────┘ + │ + ┌──────┴──────────────────────────────────┐ + │ │ + /v1/* /workflow /sqs / +(LLM) (Temporal gRPC) (Queues) (X-Service) + │ │ │ │ + ▼ ▼ ▼ ▼ +llm-serving temporal:7233 kmsvc/Kafka IAM, S3 +(vLLM, Ollama) (WorkflowService) Memory +(TEI) (gRPC bridge) (poimen) ``` -ingress-nginx keeps TLS and the edge. The gateway owns everything after it. +**Design principles:** +- ✅ Single hostname, multiple path prefixes +- ✅ HTTP REST gateway → gRPC Temporal bridge +- ✅ Bearer token auth via Authentik (JWT + RBAC) +- ✅ Streaming unbuffered (SSE, WebSocket) +- ✅ Per-route timeouts & rate limits +- ✅ No cluster credentials held by gateway -Backend services are reached through the gateway rather than published -individually — a single place for authentication, budgets, timeouts and -observability, and a single hostname surface to reason about. +--- -## Capability map +## Services & Capabilities -One host, one path prefix per capability. +| Service | Prefix | Upstream | Status | +|---------|--------|----------|--------| +| **LLM Chat** | `/v1/chat/completions` | llm-serving (vLLM) | ✅ Live | +| **Embeddings** | `/v1/embeddings` | llm-serving (TEI) | ✅ Live | +| **Reranking** | `/v1/rerank` | llm-serving (TEI) | ✅ Live | +| **Workflows** | `/workflow` | Temporal gRPC (7233) | ✅ Live (START, DESCRIBE, SIGNAL, QUERY, etc) | +| **Queues** | `/` + `X-Service: sqs` | kmsvc/Kafka | ⏳ Ready (ServiceAdapter) | +| **Memory** | `/` + `X-Service: memory` | poimen-memory | ✅ Live | +| **IAM** | `/` + `X-Service: iam` | Authentik API | ✅ Live | +| **S3** | `/` + `X-Service: s3` | MinIO | ✅ Live | -| Prefix on `api.riotpiao.com` | Backs onto | Status | -|---|---|---| -| `/v1/*` | `llm-serving` predictors (vLLM, Ollama, TEI) | migrating off Kong | -| `/sqs/*` | kmsvc management-service + Kafka/Strimzi (`sqs` ns) | future | -| `/workflow/*` | Temporal (`temporal` ns) | future | -| `/cluster/*` | atlas — cluster topology / Argo delivery (separate repo) | future | -| `/db/*` | CloudNativePG, MinIO, monitoring/metrics reads | future | +--- -`/v1/*` is reserved for the OpenAI-compatible surface. An SDK expects -`/v1/chat/completions` at the base URL, so that prefix cannot be repurposed. +## How to Use -Paths rather than subdomains: one DNS record, one tunnel hostname, one Ingress. -Promoting a prefix to its own subdomain later is additive and can run alongside the -path — the reverse is not, because clients hardcode hostnames. +### 1. Get a token -atlas lives in its own repo (`riotpiao-backend`) and keeps its own informers and -RBAC. The gateway routes to it; it does not absorb it. Cluster-read permissions -stay out of the public edge process. +**Human (OIDC device code):** +```bash +core auth login +export TOKEN=$(cat ~/.cache/talos/authentik_id_token) +``` -## Design rules +**Service account (client credentials):** +```bash +core mwinit login --username sa-name --password secret +export TOKEN=$(cat ~/.talos/.riotpiao-auth) +``` -1. **Standard protocol shapes.** `POST /v1/chat/completions` selects its model from - the request body, like every OpenAI-compatible server. No path-per-model, no - bespoke client configuration. Kong OSS could not do this; that limitation does - not survive into the replacement. -2. **Bearer tokens, validated against Authentik.** JWKS is fetched at runtime and - cached, so key rotation needs no runbook and no pinned PEM. -3. **Policy lives where the state is.** GPU slot semaphores, per-caller budgets, - queue depth and disconnect propagation are application concerns. They belong - here, not in a proxy plugin. -4. **Streaming is first-class.** SSE and WebSocket pass through unbuffered, and a - client disconnect cancels the upstream request rather than orphaning it. -5. **The gateway holds no cluster credentials.** It proxies to services that do. +### 2. Call any service -## Layout +**Chat:** +```bash +curl -X POST https://api.riotpiao.com/v1/chat/completions \ + -H "Authorization: Bearer $TOKEN" \ + -d '{ + "model": "reasoning", + "messages": [{"role": "user", "content": "What is 2+2?"}] + }' +``` + +**Workflow:** +```bash +curl -X POST https://api.riotpiao.com/workflow \ + -H "Authorization: Bearer $TOKEN" \ + -d '{ + "action": "START_WORKFLOW", + "namespace": "default", + "payload": { + "workflow_id": "my-workflow", + "workflow_type": "MyWorkflow", + "task_queue": "default" + } + }' +``` + +**Memory:** +```bash +curl -X GET https://api.riotpiao.com/ \ + -H "Authorization: Bearer $TOKEN" \ + -H 'X-Service: memory' \ + -H 'X-Resource: query' \ + -G --data-urlencode 'query=explain machine learning' +``` + +**Full examples:** See [API.md](API.md) + +--- + +## Available Models + +### LLM (Chat & Reasoning) +- `reasoning` — DeepSeek-R1-Distill-Qwen-32B (8 concurrent slots) +- `ornith:35b` — Ollama 35B +- `qwen2.5:3b-instruct` — Qwen 2.5 3B + +### Embeddings +- `nomic-ai/nomic-embed-text-v2-moe` — Fast, multilingual + +### Reranking +- `BAAI/bge-reranker-base` — Document relevance scoring + +--- + +## Authentication + +All endpoints (except `/healthz`, `/readyz`) require: ``` -cmd/gateway/ entrypoint +Authorization: Bearer +``` + +Tokens validated via Authentik JWKS (runtime fetched, cached, auto-rotated). + +**Capabilities** (RBAC): +- `llm:inference` — `/v1/*` chat/embeddings/rerank +- `workflow:execute` — `/workflow` operations +- `memory:read` / `memory:write` — Memory operations +- `sqs:access` — Queue operations +- `s3:access` — S3 operations +- `iam:admin` — User/group management + +--- + +## Error Handling + +All errors return RFC 9457 `application/problem+json`: + +```json +{ + "type": "https://api.example.com/problems/unknown-model", + "title": "Unknown Model", + "status": 400, + "detail": "Model 'gpt-4' is not available", + "valid_models": ["reasoning", "ornith:35b", ...] +} +``` + +**Common status codes:** +- 200 OK +- 400 Bad Request (validation, unknown model) +- 401 Unauthorized (missing/invalid token) +- 403 Forbidden (insufficient capability) +- 404 Not Found (workflow, resource) +- 429 Too Many Requests (rate limit) +- 503 Service Unavailable (backend down) + +--- + +## Rate Limits + +| Endpoint | Limit | Retry-After | +|----------|-------|-------------| +| `/v1/chat/completions` | 8 concurrent | Yes | +| `/v1/embeddings` | 10 concurrent | Yes | +| `/v1/rerank` | 10 concurrent | Yes | +| `/workflow` | 100 concurrent | Yes | + +Hitting limit returns 429 with `Retry-After` header. + +--- + +## Timeouts + +| Endpoint | Connect | Read | Write | +|----------|---------|------|-------| +| `/v1/chat` | 10s | 1h | 1h | +| `/v1/embeddings` | 10s | 10m | 10m | +| `/v1/rerank` | 10s | 10m | 10m | +| `/workflow` | 10s | 30s | 10s | + +Client disconnects cancel upstream request immediately (no orphaned slots). + +--- + +## Local Development + +Run without cluster, no credentials needed: + +```bash +# Build +go build ./cmd/gateway + +# Run locally +./gateway + +# Test in another terminal +curl http://localhost:8080/healthz +``` + +Points upstreams at local stubs if not connected to cluster (see `internal/config`). + +--- + +## Code Layout + +``` +cmd/gateway/ Server entrypoint internal/ - auth/ Authentik OIDC, JWKS cache, service-account tokens - llm/ model registry, body-based dispatch, upstream map - queue/ sqs.riotpiao.com surface - workflow/ workflow.riotpiao.com surface - proxy/ reverse proxy, streaming, timeouts, disconnect propagation - config/ upstream + route configuration - observability/ Prometheus metrics, structured logging -deploy/ - base/ Kubernetes manifests - argocd/ Argo Application -docs/adr/ architecture decision records -tasks/ task board — see tasks/INDEX.md -testdata/ fixtures for offline tests + server/ Router, health checks + proxy/ Reverse proxy, streaming, timeouts + temporal/ Workflow handler + gRPC bridge + serviceadapter/ X-Service dispatcher (CRD-driven) + config/ Route + upstream configuration + auth/ Authentik JWT validation + observability/ Metrics, structured logging +k8s/ + configmap.yaml Route definitions + rbac.yaml Service account, roles + deployment.yaml Pod spec + networkpolicy.yaml Ingress/egress rules +testdata/ Fixtures for offline tests ``` -## Local development +--- -The gateway must be runnable with no cluster, no kubeconfig and no credentials, so -that changes can be verified in a closed loop before touching live traffic. -Upstreams are configuration, so pointing them at local stubs is the whole -mechanism. See [tasks/INDEX.md](tasks/INDEX.md). +## Deployment -## Status +Deployed to Kubernetes via ArgoCD: -Scaffolded 2026-08-19. Nothing is wired yet. Kong is still serving live traffic on -`api.riotpiao.com`. +```bash +# Check deployment +kubectl -n api get deployment homelab-frontend + +# View logs +kubectl -n api logs -l app=homelab-frontend -f + +# Restart +kubectl -n api rollout restart deployment/homelab-frontend +``` + +Configuration mounted as ConfigMap (`k8s/configmap.yaml`). + +--- + +## Health Checks + +```bash +# Liveness (always succeeds) +curl https://api.riotpiao.com/healthz + +# Readiness (waits for config + JWKS) +curl https://api.riotpiao.com/readyz + +# List available models +curl https://api.riotpiao.com/v1/models +``` + +--- + +## Support + +**Issues:** Check pod logs +```bash +kubectl -n api logs deployment/homelab-frontend +``` + +**Debug config:** +```bash +kubectl -n api get configmap homelab-frontend-config -o yaml +``` + +**Restart pod:** +```bash +kubectl -n api rollout restart deployment/homelab-frontend +``` + +**Test endpoint directly:** +```bash +kubectl -n api port-forward svc/homelab-frontend 8080:8080 +curl http://localhost:8080/healthz +``` + +--- + +## See Also + +- [API.md](API.md) — Complete API reference with examples +- `internal/` — Source code (handlers, routing, auth) +- `k8s/` — Kubernetes manifests