(plan) system review and break down plans

2026-08-19 09:52:07 -07:00
commit 9d28b63ff2
46 changed files with 5620 additions and 0 deletions
@@ -0,0 +1,122 @@
+# M6.4 — nginx TCP routing to `agent-manager-db`
+
+| Field | Value |
+|---|---|
+| Phase | M6 — agent-manager migration |
+| Size | S — <1 day |
+| Status | ⬜ Not started |
+| Flags | homelab |
+| Spec | inlined below |
+| Blocks | M6.1 |
+
+## Goal
+
+A path from the Mac client running agent-manager to the cluster-internal,
+ClusterIP-only `agent-manager-db` — through the shared ingress controller,
+matching this homelab's existing pattern, not a raw LAN IP or a tunnel.
+
+## Facts (inlined — no spec read needed)
+
+**Decided, not open:** dedicated ingress routing through nginx, not
+`kubectl port-forward`/SSH tunnel and not a MetalLB `LoadBalancer` IP.
+Matches the pattern already used elsewhere in this homelab of routing
+through the shared ingress controller rather than exposing raw
+per-service LAN IPs.
+
+**Postgres is not HTTP.** The standard nginx-ingress `Ingress` resource is
+HTTP(S)-oriented (host/path routing, TLS termination via SNI on 443).
+Postgres speaks its own binary wire protocol on 5432. The ingress
+controller needs `stream {}` block config (TCP/UDP passthrough) or a
+dedicated `TCP` mode `Service`/`ConfigMap` entry — whichever the specific
+nginx-ingress deployment in this cluster supports (check
+`k8s/infra/ingress/` for how it's deployed and whether `tcp-services`
+ConfigMap wiring already exists for anything else, since this may be the
+first TCP passthrough case in the cluster).
+
+**No existing precedent in this homelab** — `forgejo-db`, `authentik-db`,
+`temporal-db` are all consumed only by pods inside the same cluster over
+their ClusterIP Service, never from outside. `agent-manager-db` is the
+first case of an external (Mac) client needing to reach a CNPG cluster,
+which is why this task exists as dedicated work rather than "just add a
+Service."
+
+## Steps
+
+1. Confirm how nginx-ingress is deployed in this cluster (`k8s/infra/
+   ingress/`) and whether it already exposes a `tcp-services` ConfigMap
+   or `stream {}` snippet mechanism — ingress-nginx (the community
+   controller) supports TCP passthrough via a `tcp-services` ConfigMap
+   mapping `<external-port>: <namespace>/<service>:<port>`; confirm this
+   is the controller in use before assuming that config shape.
+2. Pick an external port for Postgres traffic (5432 is already the
+   in-cluster default; an external port distinct from any other exposed
+   service avoids collision — check what's already claimed).
+3. Add the `tcp-services` (or equivalent) entry routing that external
+   port to `agent-manager-db-rw.agent-manager.svc.cluster.local:5432`
+   (CNPG's read-write Service name convention — confirm against the
+   actual Service name M6.1's `Cluster` generates).
+4. Expose that port on the ingress controller's `Service`/`LoadBalancer`
+   (this is the one LAN-facing port for this whole feature — the DB
+   itself stays ClusterIP-only, only the ingress controller's existing
+   external IP gains a new port).
+5. Commit, push to both remotes, verify which `repoURL` the owning
+   ArgoCD `Application` watches (same caveat as M6.1) before assuming a
+   push landed, let ArgoCD sync.
+6. Test connectivity from the Mac client: `psql
+   postgresql://<user>@<homelab-ingress-host>:<external-port>/
+   agent_manager` (credentials from M6.5).
+
+## Acceptance
+
+- `psql` (or `pgx`) from outside the cluster reaches `agent-manager-db`
+  through the ingress controller's external IP/port.
+- `agent-manager-db`'s own Service remains ClusterIP-only — no
+  `LoadBalancer`/`NodePort` added to it directly (that would defeat the
+  point of routing through nginx).
+- TLS/auth on the connection is Postgres's own (`sslmode`, password auth)
+  — nginx `stream {}` passthrough does not terminate or inspect the
+  Postgres protocol, so it adds no auth of its own. Confirm this is
+  acceptable given the homelab's network boundary (LAN-only ingress
+  exposure, not public internet) before treating it as done.
+
+## Verify
+
+**Harness:** `psql` from the Mac client (outside the cluster network),
+plus `kubectl` checks on the ingress controller's config.
+
+**Integration test** — `verify/m6.4.sh` diffed against
+`expected/m6.4.txt`:
+1. `a1_external_connects` — `psql
+   postgresql://app@<ingress-host>:<port>/agent_manager -c 'select 1'`
+   from the Mac client succeeds.
+2. `a2_db_service_still_clusterip` — `kubectl get svc -n agent-manager
+   agent-manager-db-rw -o jsonpath='{.spec.type}'` is `ClusterIP`.
+3. `a3_argocd_synced` — the ingress-owning app is `Synced/Healthy` after
+   the config change.
+4. `a4_wrong_port_refused` — connecting to a random unmapped port on the
+   same ingress host fails (proves the mapping is port-specific, not an
+   accidental catch-all passthrough).
+
+**Command:** `bash verify/m6.4.sh | diff - expected/m6.4.txt`
+
+**False pass:**
+- Testing connectivity from inside the cluster (e.g. `kubectl exec` into
+  a pod and `psql` the ClusterIP directly). That was already true before
+  this task and proves nothing about the ingress path — assertion 1 must
+  run from the actual Mac client, outside the cluster network.
+
+## Traps
+
+- Reusing port 5432 externally on the ingress controller's existing
+  external IP if anything else is already listening there (unlikely for
+  Postgres specifically, but worth a `kubectl get svc -n <ingress-ns>
+  <ingress-svc> -o yaml` check before assuming the port is free).
+- `stream {}` / `tcp-services` config living outside the GitOps-tracked
+  kustomization because it's a ConfigMap edit that "felt like a quick
+  manual fix." Same hard rule as everything else: commit + push +
+  ArgoCD sync, no manual `kubectl apply` to the ingress controller's
+  config.
+
+---
+
+Background: [M6.1](M6.1-agent-manager-db-manifest.md) · `k8s/infra/ingress/` (nginx-ingress deployment, controller type to confirm)