(plan) system review and break down plans

2026-08-19 09:52:07 -07:00
commit 9d28b63ff2
46 changed files with 5620 additions and 0 deletions
@@ -0,0 +1,102 @@
+# M2.2 — CNPG `memory-db` + pgvector
+
+| Field | Value |
+|---|---|
+| Phase | M2 — Projections |
+| Size | M — 1–3 days |
+| Status | ⬜ Not started |
+| Flags | homelab |
+| Spec | inlined below |
+| Blocks | — |
+
+## Goal
+
+A Postgres with pgvector, provisioned the way everything else in the cluster is:
+through git, with no manual `psql`.
+
+## Facts (inlined — no spec read needed)
+
+**pgvector needs no custom image.** Verified on the running cluster:
+
+```
+$ psql -tAc "select name,default_version,installed_version
+             from pg_available_extensions where name='vector'"
+vector|0.7.0|
+```
+
+on the stock `ghcr.io/cloudnative-pg/postgresql:16.2`. Available, not yet
+installed — `CREATE EXTENSION` is all that is missing.
+
+**The operator is CNPG 1.30.0**, which supports declarative extensions on the
+`Database` CRD (`kubectl explain database.spec.extensions` resolves). So the
+extension is git-managed too — no manual step, consistent with the GitOps rule
+that infrastructure changes flow through version control.
+
+Follow `k8s/infra/databases/temporal-db.yaml` exactly: 3 instances, `imageName`
+pinned, `enableSuperuserAccess: false`, `storageClass: longhorn-cnpg`,
+`enablePodMonitor: true`, control-plane tolerations, `podAntiAffinityType:
+preferred`.
+
+Storage: 10Gi matches the existing clusters. At 768 dims × 4 bytes, a vector is
+~3 KB; tens of thousands of nodes is well under a gigabyte, so 10Gi is generous
+and consistent rather than tight.
+
+## Steps
+
+1. `k8s/infra/databases/memory-db.yaml` — `Cluster` + `Database` with
+   `extensions: [{name: vector, ensure: present}]`.
+2. Namespace `memory`, created by the ArgoCD app that owns it.
+3. Add to the owning kustomization's explicit resource list — an unlisted file is
+   silently dropped with no error and no drift shown.
+4. Commit, push, let ArgoCD sync. **No `kubectl apply`.**
+5. Verify the extension installed and the app user can create tables.
+6. Record the connection string convention in the repo README; the password comes
+   from the CNPG-generated secret, never committed.
+
+## Acceptance
+
+- `Cluster` reaches `Cluster in healthy state` with 3 instances.
+- `select extversion from pg_extension where extname='vector'` returns a version.
+- ArgoCD shows the app `Synced/Healthy`.
+- No manual `psql` was run to get there.
+
+## Verify
+
+**Harness:** `kubectl` and `psql` read-only checks after sync.
+
+**Integration test** — `verify/m2.2.sh`, output diffed against `expected/m2.2.txt`:
+1. `a1_cluster_healthy` — `kubectl get cluster -n memory memory-db` reports 3/3
+   ready.
+2. `a2_extension_installed` — `select extname, extversion from pg_extension where
+   extname='vector'` returns one row.
+3. `a3_declarative_not_manual` — `kubectl get database -n memory memory-db-vector
+   -o jsonpath='{.spec.extensions}'` shows the declaration, proving it came from
+   git.
+4. `a4_argocd_synced` — the owning app is `Synced/Healthy`.
+5. `a5_app_user_can_ddl` — as `app`, `CREATE TABLE t(v vector(768)); DROP TABLE t;`
+   succeeds.
+6. `a6_hnsw_available` — `CREATE INDEX ... USING hnsw` on that temp table
+   succeeds, proving 0.7.0 has the index type the schema needs.
+
+**Command:** `bash verify/m2.2.sh | diff - expected/m2.2.txt`
+
+**False pass:**
+- Checking `pg_available_extensions` instead of `pg_extension`. Available means
+  the files are on disk; installed means `CREATE EXTENSION` ran. The whole task
+  is the second one.
+- Verifying after a manual `CREATE EXTENSION`. It passes and proves nothing about
+  the declarative path, which is the actual deliverable. Assertion 3 is the guard.
+
+## Traps
+
+- Forgetting the kustomization resource list. The file sits in git, ArgoCD reports
+  Synced, and the objects never exist — silent, and the failure surfaces later as
+  a connection error.
+- Adding `prune: true` semantics without thinking about operator-created children.
+  CNPG creates Services, Secrets and PVCs owned by the Cluster; if ArgoCD's
+  tracking label propagates to them, prune fights the operator. The
+  `llm-serving` app already had to set `prune: false` for exactly this reason.
+
+---
+
+Background: [DESIGN.md](../DESIGN.md) — pgvector · `k8s/infra/databases/temporal-db.yaml`