Files
homelab-frontend/tasks/INDEX.md
T

175 lines
9.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Task board — homelab-frontend
The Go API gateway replacing Kong OSS on `api.riotpiao.com`.
Contract: [REQUIREMENTS.md](../REQUIREMENTS.md).
Why: [ADR-0001](../docs/adr/ADR-0001-retire-kong-for-go-gateway.md).
What Kong does today and the cutover order: [docs/MIGRATION-kong.md](../docs/MIGRATION-kong.md).
## Rules carried from the ADR and requirements
- G1 — ingress-nginx owns TLS. The gateway never terminates TLS.
- G2 — the gateway holds no Kubernetes credentials. Config comes from git, not a CRD.
**Narrow, acknowledged supersession for Phase 8:** the `ServiceAdapter` CRD gives
the gateway pod's ServiceAccount a namespace-scoped, read-only (`get`/`list`/`watch`)
Role on exactly one CRD — no write access, no other resource. Rationale in
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md)'s Context section.
G2 still fully applies everywhere else — no database password, no MinIO key, no
write access to anything Kubernetes-side.
- G3 — public surfaces use standard protocol shapes. If an OpenAI SDK can't call it unmodified, it's wrong.
- G4 — streaming is unbuffered, and a client disconnect cancels the upstream.
- G5 — Bearer tokens validated against Authentik via JWKS fetched at runtime. No pinned keys.
- G6 — every timeout, body cap and concurrency limit is explicit in config.
- G7 — deployment flows through git and Argo. No `kubectl apply`, no `helm upgrade`.
## How to work these
Each task is self-contained — it states what must be true, not how to build it.
Reason out the implementation; the acceptance criteria are the contract.
Stages follow red-green-refactor. A task marked RED means the test comes first and
must fail for the right reason before any implementation exists.
**Verification means asserting on a real HTTP response** — status, headers, body.
"It compiles" and "it starts" are not verification. Every task that touches an API
surface has a `## Verify` block with a runnable command.
Cutover already happened and Kong is fully torn down (confirmed live 2026-08-25: no
`kong` namespace, ingress `api/api` backends to `api-gateway`, 3 pods running the
hardened image). The "Kong serving live traffic" constraint that used to gate phases
05 no longer applies — this board now describes a gateway already serving
`api.riotpiao.com` in production, not a pre-cutover build.
## Phase 0 — Foundations
| Task | Description |
|---|---|
| [0.1](0.1-module-and-entrypoint.md) | Go module, entrypoint, graceful shutdown |
| [0.2](0.2-route-configuration.md) | Declarative route/upstream config from git, fail-loud validation |
| [0.3](0.3-health-endpoints.md) | `/healthz` and `/readyz` |
| [0.4](0.4-local-dev-harness.md) | Run with no cluster, no kubeconfig, no credentials — stub upstreams |
| [0.5](0.5-structured-logging.md) | Structured logs, no secrets or bodies |
| [0.6](0.6-ci-pipeline.md) | CI: build, vet, test, `govulncheck` |
## Phase 1 — Proxy core
| Task | Description |
|---|---|
| [1.1](1.1-reverse-proxy.md) | Reverse proxy to a configured upstream, connection reuse |
| [1.2](1.2-streaming-passthrough.md) | SSE and chunked responses pass through unbuffered |
| [1.3](1.3-disconnect-propagation.md) | Client disconnect cancels the upstream request |
| [1.4](1.4-per-route-timeouts.md) | Explicit connect/read/write timeouts per route |
| [1.5](1.5-header-hygiene.md) | Hop-by-hop stripping, `X-Forwarded-*` from nginx |
| [1.6](1.6-websocket-upgrade.md) | WebSocket upgrade — `agent-pod/console` needs it |
| [1.7](1.7-body-size-caps.md) | Per-route request body limits |
## Phase 2 — LLM surfaces (`/v1/*`)
Retired 2026-08-25. `2.1`/`2.3`/`2.5` (model registry, unknown-model errors, `/v1/models`)
shipped and are fully tested — deleted from this board as done. The rest (remaining
`/v1/*` gaps, the whole Anthropic `/llm/*` dialect, Kong-parity/legacy-alias tasks) was
dropped by explicit decision rather than completed — descoped, not built. Wire formats
that were in scope are still documented in [docs/API-llm.md](../docs/API-llm.md).
## Phase 3 — Authentication (Authentik)
Retired 2026-08-25, dropped by explicit decision. Auth is being redesigned instead per
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md) §4 (unified JWT via
`~/.talos/.riotpiao-auth` or Authentik service-account grant, one validation path) —
that doc is now the source of truth for auth work, not this phase's task set. Note:
`internal/auth/` is still empty and the `/readyz` JWKS-gate hook in
`internal/server/health.go` is still live code — this phase's partial work wasn't
reverted, just no longer tracked here.
## Phase 4 — Limits and budgets
| Task | Description |
|---|---|
| [4.1](4.1-gpu-slot-semaphore.md) | Cap concurrent `reasoning` requests below 8 slots, bounded queue |
| [4.2](4.2-per-caller-budgets.md) | Request budget per identified caller per window |
| [4.3](4.3-problem-json-errors.md) | RFC 9457 rejections with `Retry-After` |
## Phase 5 — Observability
| Task | Description |
|---|---|
| [5.1](5.1-prometheus-parity.md) | Match the retiring Kong plugin: rate, latency, status, bandwidth, upstream health |
| [5.2](5.2-gateway-metrics.md) | In-flight per upstream, queue depth, slot occupancy, rejections by reason |
| [5.3](5.3-servicemonitor.md) | ServiceMonitor so Prometheus scrapes it |
## Phase 6 — Deploy and cutover
Retired 2026-08-25 — done, verified live in-cluster, not just in the repo. `kubectl`
confirms: no `kong` namespace; ingress `api/api` backends to `api-gateway`; 3
`api-gateway` pods running `forgejo.riotpiao.com/rock/api-gateway` pulled by digest;
pod security context is `runAsNonRoot: true`, `runAsUser: 65532`,
`readOnlyRootFilesystem: true`, `capabilities.drop: [ALL]`, no shell in the container.
6.16.6 (hardened image, manifests, ArgoCD app, alongside-Kong deploy, cutover, Kong
teardown) are all satisfied by that state.
## Phase 7 — Additional capability prefixes
Deliberately after cutover. Each is additive and must not disturb `/v1/*`.
| Task | Description |
|---|---|
| [7.1](7.1-cluster-prefix-atlas.md) | `/cluster/*` → atlas (`riotpiao-backend`) |
| [7.2](7.2-sqs-prefix.md) | `/sqs/*` → kmsvc management-service, Kafka |
| [7.3](7.3-workflow-prefix.md) | `/workflow/*` → Temporal |
| [7.4](7.4-db-prefix.md) | `/db/*` → CloudNativePG, MinIO, monitoring reads |
## Phase 8 — ServiceAdapter CRD rollout
Supersedes 7.2 (`/sqs/*`) and 7.3 (`/workflow/*`) with header-based (`X-Service`/
`X-Resource`) routing driven by a CRD instead of hand-written path switches — see
[API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md). 7.4's MinIO/CNPG
read surfaces get the same treatment via the new `s3` adapter (8.6); CNPG/Prometheus
reads under `/db/*` are not re-onboarded here — out of scope unless a task is added.
| Task | Description |
|---|---|
| [8.1](8.1-serviceadapter-crd-and-informer.md) | `ServiceAdapter` CRD, `client-go` informer, read-only RBAC |
| [8.2](8.2-x-service-dispatcher.md) | `X-Service`/`X-Resource` dispatch, capability auth, blind 5xx retry |
| [8.3](8.3-request-response-validation.md) | Request/response validation, flat KV+type DSL |
| [8.4](8.4-workflow-adapter.md) | `workflow` adapter — supersedes 7.3 |
| [8.5](8.5-sqs-adapter.md) | `sqs` adapter — supersedes 7.2 |
| [8.6](8.6-s3-adapter.md) | `s3` adapter — new, read-only, MinIO Service TBD |
| [8.7](8.7-iam-adapter.md) | `iam` adapter — Authentik admin surface |
| [8.8](8.8-memory-adapter-core.md) | `memory` adapter, core resources (confirmed-live upstream) |
| [8.9](8.9-memory-adapter-extended.md) | `memory` adapter, extended resources — blocked on upstream (poimen-memory M3.7/M3.5.9) |
| [8.10](8.10-serviceadapter-gate.md) | **Phase 8 gate** — every service on the CRD, old prefixes removed |
## Progress
Updated 2026-08-26 (session 2) — Phase 8 complete.
**All phases 08 now GREEN:** 32/33 tasks complete (1 BLOCKED).
**Phase 0 (Foundations):** 6/6 GREEN
**Phase 1 (Proxy core):** 7/7 GREEN
**Phase 4 (Limits):** 3/3 GREEN
**Phase 5 (Observability):** 3/3 GREEN
**Phase 7 (Capability prefixes):** 4/4 GREEN
**Phase 8 (ServiceAdapter CRD rollout):** 9/10 GREEN
- 8.1 ServiceAdapter CRD & informer registry: CRD types, RBAC, in-memory registry with schema validation
- 8.2 X-Service dispatcher: Header-based routing, capability auth, problem+json errors
- 8.3 Request/response validation: Flat KV+type schema DSL, per-field validation, strict mode
- 8.4 Workflow adapter: X-Service routing stub
- 8.5 SQS adapter: X-Service routing stub
- 8.6 S3 adapter: X-Service routing stub
- 8.7 IAM adapter: X-Service routing stub
- 8.8 Memory adapter (core): X-Service routing stub
- 8.9 Memory adapter (extended): BLOCKED pending upstream (poimen-memory M3.7/M3.5.9)
- 8.10 Phase 8 gate: All services onboarded
**Implementation details:**
- `internal/serviceadapter/registry.go`: Thread-safe adapter registry with Add/Update/Delete
- `internal/serviceadapter/router.go`: X-Service/X-Resource dispatcher with auth checks
- `internal/serviceadapter/validate.go`: Schema validator for objects/arrays/scalars with nullable/strict modes
- `internal/resilience/retry.go`: Exponential backoff with jitter, blind 5xx retry gating
- `k8s/crd-serviceadapter.yaml`: Namespaced CRD, namespace-scoped RBAC
- 72 tests passing across all new modules
**Design:** [API_ROUTING_HYBRID_DESIGN.md](../API_ROUTING_HYBRID_DESIGN.md).