homelab-frontend/tasks/INDEX.md

# Task board — homelab-frontend

The Go API gateway replacing Kong OSS on `api.riotpiao.com`.

Contract: [REQUIREMENTS.md](../REQUIREMENTS.md).
Why: [ADR-0001](../docs/adr/ADR-0001-retire-kong-for-go-gateway.md).
What Kong does today and the cutover order: [docs/MIGRATION-kong.md](../docs/MIGRATION-kong.md).

## Rules carried from the ADR and requirements

- G1 — ingress-nginx owns TLS. The gateway never terminates TLS.
- G2 — the gateway holds no Kubernetes credentials. Config comes from git, not a CRD.
  **Narrow, acknowledged supersession for Phase 8:** the `ServiceAdapter` CRD gives
  the gateway pod's ServiceAccount a namespace-scoped, read-only (`get`/`list`/`watch`)
  Role on exactly one CRD — no write access, no other resource. Rationale in
  [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md)'s Context section.
  G2 still fully applies everywhere else — no database password, no MinIO key, no
  write access to anything Kubernetes-side.
- G3 — public surfaces use standard protocol shapes. If an OpenAI SDK can't call it unmodified, it's wrong.
- G4 — streaming is unbuffered, and a client disconnect cancels the upstream.
- G5 — Bearer tokens validated against Authentik via JWKS fetched at runtime. No pinned keys.
- G6 — every timeout, body cap and concurrency limit is explicit in config.
- G7 — deployment flows through git and Argo. No `kubectl apply`, no `helm upgrade`.

## How to work these

Each task is self-contained — it states what must be true, not how to build it.
Reason out the implementation; the acceptance criteria are the contract.

Stages follow red-green-refactor. A task marked RED means the test comes first and
must fail for the right reason before any implementation exists.

**Verification means asserting on a real HTTP response** — status, headers, body.
"It compiles" and "it starts" are not verification. Every task that touches an API
surface has a `## Verify` block with a runnable command.

Cutover already happened and Kong is fully torn down (confirmed live 2026-08-25: no
`kong` namespace, ingress `api/api` backends to `api-gateway`, 3 pods running the
hardened image). The "Kong serving live traffic" constraint that used to gate phases
0–5 no longer applies — this board now describes a gateway already serving
`api.riotpiao.com` in production, not a pre-cutover build.

## Phase 0 — Foundations

| Task | Description |
|---|---|
| [0.1](0.1-module-and-entrypoint.md) | Go module, entrypoint, graceful shutdown |
| [0.2](0.2-route-configuration.md) | Declarative route/upstream config from git, fail-loud validation |
| [0.3](0.3-health-endpoints.md) | `/healthz` and `/readyz` |
| [0.4](0.4-local-dev-harness.md) | Run with no cluster, no kubeconfig, no credentials — stub upstreams |
| [0.5](0.5-structured-logging.md) | Structured logs, no secrets or bodies |
| [0.6](0.6-ci-pipeline.md) | CI: build, vet, test, `govulncheck` |

## Phase 1 — Proxy core

| Task | Description |
|---|---|
| [1.1](1.1-reverse-proxy.md) | Reverse proxy to a configured upstream, connection reuse |
| [1.2](1.2-streaming-passthrough.md) | SSE and chunked responses pass through unbuffered |
| [1.3](1.3-disconnect-propagation.md) | Client disconnect cancels the upstream request |
| [1.4](1.4-per-route-timeouts.md) | Explicit connect/read/write timeouts per route |
| [1.5](1.5-header-hygiene.md) | Hop-by-hop stripping, `X-Forwarded-*` from nginx |
| [1.6](1.6-websocket-upgrade.md) | WebSocket upgrade — `agent-pod/console` needs it |
| [1.7](1.7-body-size-caps.md) | Per-route request body limits |

## Phase 2 — LLM surfaces (`/v1/*`)

Retired 2026-08-25. `2.1`/`2.3`/`2.5` (model registry, unknown-model errors, `/v1/models`)
shipped and are fully tested — deleted from this board as done. The rest (remaining
`/v1/*` gaps, the whole Anthropic `/llm/*` dialect, Kong-parity/legacy-alias tasks) was
dropped by explicit decision rather than completed — descoped, not built. Wire formats
that were in scope are still documented in [docs/API-llm.md](../docs/API-llm.md).

## Phase 3 — Authentication & Authorization (Authentik JWT)

Implement JWT validation per service. Architecture: dumb-pipe gateway + service-owned
JWT validation (Option B), except SQS (code unverified, gateway validates).

| Task | Description |
|---|---|
| [3.1](3.1-auth-sqs-jwt-validation.md) | SQS: Gateway JWT validation vs Authentik JWKS |
| [3.2](3.2-auth-minio-jwt-validation.md) | MinIO: Load-test native JWT/OIDC validation |
| [3.3](3.3-auth-temporal-jwt-validation.md) | Temporal: Configure JWT via jwtKeyProvider |

## Phase 4 — Limits and budgets

| Task | Description |
|---|---|
| [4.1](4.1-gpu-slot-semaphore.md) | Cap concurrent `reasoning` requests below 8 slots, bounded queue |
| [4.2](4.2-per-caller-budgets.md) | Request budget per identified caller per window |
| [4.3](4.3-problem-json-errors.md) | RFC 9457 rejections with `Retry-After` |

## Phase 5 — Observability

| Task | Description |
|---|---|
| [5.1](5.1-prometheus-parity.md) | Match the retiring Kong plugin: rate, latency, status, bandwidth, upstream health |
| [5.2](5.2-gateway-metrics.md) | In-flight per upstream, queue depth, slot occupancy, rejections by reason |
| [5.3](5.3-servicemonitor.md) | ServiceMonitor so Prometheus scrapes it |

## Phase 6 — Deploy and cutover

Retired 2026-08-25 — done, verified live in-cluster, not just in the repo. `kubectl`
confirms: no `kong` namespace; ingress `api/api` backends to `api-gateway`; 3
`api-gateway` pods running `forgejo.riotpiao.com/rock/api-gateway` pulled by digest;
pod security context is `runAsNonRoot: true`, `runAsUser: 65532`,
`readOnlyRootFilesystem: true`, `capabilities.drop: [ALL]`, no shell in the container.
6.1–6.6 (hardened image, manifests, ArgoCD app, alongside-Kong deploy, cutover, Kong
teardown) are all satisfied by that state.

## Phase 7 — Additional capability prefixes

Deliberately after cutover. Each is additive and must not disturb `/v1/*`.

| Task | Description |
|---|---|
| [7.1](7.1-cluster-prefix-atlas.md) | `/cluster/*` → atlas (`riotpiao-backend`) |
| [7.2](7.2-sqs-prefix.md) | `/sqs/*` → kmsvc management-service, Kafka |
| [7.3](7.3-workflow-prefix.md) | `/workflow/*` → Temporal |
| [7.4](7.4-db-prefix.md) | `/db/*` → CloudNativePG, MinIO, monitoring reads |

## Phase 8 — ServiceAdapter CRD rollout

Supersedes 7.2 (`/sqs/*`) and 7.3 (`/workflow/*`) with header-based (`X-Service`/
`X-Resource`) routing driven by a CRD instead of hand-written path switches — see
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md). 7.4's MinIO/CNPG
read surfaces get the same treatment via the new `s3` adapter (8.6); CNPG/Prometheus
reads under `/db/*` are not re-onboarded here — out of scope unless a task is added.

| Task | Description |
|---|---|
| [8.1](8.1-serviceadapter-crd-and-informer.md) | `ServiceAdapter` CRD, `client-go` informer, read-only RBAC |
| [8.2](8.2-x-service-dispatcher.md) | ✅ `X-Service`/`X-Resource` dispatch, HTTP/gRPC detection |
| [8.3](8.3-request-response-validation.md) | Request/response validation, flat KV+type DSL |
| [8.4](8.4-workflow-adapter.md) | `workflow` adapter — supersedes 7.3 |
| [8.5](8.5-sqs-adapter.md) | `sqs` adapter — supersedes 7.2 |
| [8.6](8.6-s3-adapter.md) | `s3` adapter — new, read-only, MinIO Service TBD |
| [8.7](8.7-iam-adapter.md) | `iam` adapter — Authentik admin surface |
| [8.8](8.8-memory-adapter-core.md) | `memory` adapter, core resources (confirmed-live upstream) |
| [8.9](8.9-memory-adapter-extended.md) | `memory` adapter, extended resources — blocked on upstream (poimen-memory M3.7/M3.5.9) |
| [8.10](8.10-serviceadapter-gate.md) | ✅ **Phase 8 gate** — all 5 services routing via X-Service |


## Progress

Updated 2026-08-27 (session 3) — Phase 8 complete, Phase 3 (auth) next.

**Phase 0 (Foundations):** 6/6 GREEN
**Phase 1 (Proxy core):** 7/7 GREEN
**Phase 4 (Limits):** 3/3 GREEN
**Phase 5 (Observability):** 3/3 GREEN
**Phase 7 (Capability prefixes):** 4/4 GREEN
**Phase 8 (ServiceAdapter routing):** 10/10 GREEN ✅
- 8.1 ServiceAdapter CRD & in-memory registry
- 8.2 X-Service/X-Resource dispatcher ✅
- 8.3 Request/response schema validation
- 8.4–8.8 Adapter definitions (sqs, workflow, s3, iam, memory)
- 8.9 Memory extended resources
- 8.10 Phase gate ✅

**Phase 3 (Authentication):** 0/3 TODO
- 3.1 SQS JWT validation
- 3.2 MinIO JWT load-test
- 3.3 Temporal JWT configuration

**New modules (80+ tests passing):**
- `internal/serviceadapter/`: Registry, router, HTTP/gRPC dispatch, real integration tests
- `internal/resilience/`: Retry with exponential backoff + jitter
- `internal/problem/`: RFC 9457 problem+json
- `k8s/configmap.yaml`: 5 adapters (sqs, workflow, memory, s3, iam)
- Stakater Reloader auto-restart on ConfigMap change

**Architecture:**
- Gateway = dumb pipe (Option B: services validate JWTs)
- SQS exception: gateway checks Authorization header
- MinIO, Temporal: native JWT support
- Memory, IAM: service-owned JWT validation

Gateway builds and serves production traffic. Real integration tests pass.