docs(CLAUDE.md): document CNPG unified pattern and fix storage topology
ADDED: - CloudNativePG (CNPG) Database Pattern section - Explains shared 'app' user model (not per-app roles) - Documents bootstrap.sh credential distribution pattern - Working examples (Forgejo, Authentik) - Prescriptive DO/DON'T guidance for new apps FIXED: - Storage topology: 3-node HA (not "sole Longhorn node") - Verified: all 17 PVCs have replicas across all 3 nodes - Updated last-modified date This documents the architectural pattern established during CNPG refactor.
This commit is contained in:
@@ -4,13 +4,14 @@
|
||||
|
||||
| Node | IP | Zone | Scheduling | Storage |
|
||||
|------|----|----|-----------|---------|
|
||||
| `talos-cp-1` | .213 | az-a | schedulable (all workloads) | sole Longhorn node |
|
||||
| `talos-cp-2` | .163 | az-b | dedicated (`NoSchedule`) | none |
|
||||
| `talos-cp-3` | .166 | az-c | dedicated (`NoSchedule`) | none |
|
||||
| `talos-cp-1` | .213 | az-a | schedulable (all workloads) | Longhorn replica |
|
||||
| `talos-cp-2` | .163 | az-b | dedicated (`NoSchedule`) | Longhorn replica |
|
||||
| `talos-cp-3` | .166 | az-c | dedicated (`NoSchedule`) | Longhorn replica |
|
||||
|
||||
3 voting etcd members peering on the LAN. Only `talos-cp-1` runs workloads and
|
||||
holds storage → stateful services are single-instance. Full detail + gotchas in
|
||||
`USAGE.md` and memory `reference_talos_etcd_and_ca_gotchas`.
|
||||
3 voting etcd members peering on the LAN. All 3 nodes run Longhorn with 3-replica
|
||||
HA storage (verified: all 17 PVCs have replicas across all nodes). Only `talos-cp-1`
|
||||
is schedulable for workloads (cp-2/cp-3 are control-plane only). Full detail + gotchas
|
||||
in `USAGE.md` and memory `reference_talos_etcd_and_ca_gotchas`.
|
||||
|
||||
## Deployment Model: ArgoCD GitOps (helmfile is retired)
|
||||
|
||||
@@ -70,6 +71,75 @@ Verify no changes (clean output = formatted correctly). If files change, review
|
||||
|
||||
🔴 **Infrastructure changes should flow through GitOps when possible:** git commit → push → CI/CD runner (terraform apply) → ArgoCD sync. Local `terraform apply` is permitted (e.g. for local iteration, config regeneration, or when CI/CD isn't wired up for a given module) — still commit + push the resulting state/config afterward so git remains the record of truth. Manual `kubectl apply` remains disallowed for Terraform-managed resources.
|
||||
|
||||
## CloudNativePG (CNPG) Database Pattern
|
||||
|
||||
**Simple ownership model — all apps use shared 'app' user:**
|
||||
|
||||
```yaml
|
||||
# CNPG Cluster (bootstrap, wave 0)
|
||||
bootstrap:
|
||||
initdb:
|
||||
database: app # Bootstrap database
|
||||
owner: app # Bootstrap user (owns all databases)
|
||||
|
||||
# Database CR (per-app, wave 6)
|
||||
spec:
|
||||
name: authentik # Database name
|
||||
owner: app # ← All apps use 'app' (not per-app roles)
|
||||
cluster:
|
||||
name: ddb-cluster
|
||||
```
|
||||
|
||||
**Credential Distribution (bootstrap.sh pattern):**
|
||||
|
||||
1. **Source of truth:** CNPG creates `ddb-cluster-app` secret in `ddb` namespace
|
||||
2. **Distribution:** `bootstrap.sh` copies secret to app namespaces:
|
||||
```bash
|
||||
# For each app namespace (cicd, iam, etc):
|
||||
kubectl get secret ddb-cluster-app -n ddb -o yaml \
|
||||
| sed 's/namespace: ddb/namespace: <app-namespace>/' \
|
||||
| kubectl apply -f -
|
||||
```
|
||||
3. **Apps reference local copy:**
|
||||
```yaml
|
||||
env:
|
||||
- name: DB_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ddb-cluster-app # Local copy in app's namespace
|
||||
key: username # Always "app"
|
||||
- name: DB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ddb-cluster-app
|
||||
key: password
|
||||
```
|
||||
|
||||
**Key Points:**
|
||||
|
||||
✅ **DO:**
|
||||
- All apps connect as `app` user
|
||||
- Database CRs specify `owner: app`
|
||||
- Isolation via separate database names (not roles)
|
||||
- Copy credentials to app namespace via `bootstrap.sh`
|
||||
- Reference local secret copy via `secretKeyRef`
|
||||
|
||||
❌ **DON'T:**
|
||||
- Create per-app roles in `managed.roles` (CNPG doesn't transfer ownership properly)
|
||||
- Grant permissions via PostSync Jobs (owner already has full rights)
|
||||
- Use cross-namespace `secretKeyRef` (not supported)
|
||||
- Manually patch secrets (ArgoCD will revert)
|
||||
|
||||
**Working Examples:**
|
||||
- Forgejo: `k8s/bootstrap-local/04-forgejo.yaml` (references `ddb-cluster-app` in cicd namespace)
|
||||
- Authentik: `k8s/security/iam/authentik-values.yaml` (references `ddb-cluster-app` in iam namespace)
|
||||
|
||||
**For New Apps:**
|
||||
1. Add Database CR to `k8s/data/schemas/` with `owner: app`
|
||||
2. Add secret copy to `bootstrap.sh` (like cicd/iam examples)
|
||||
3. Reference `ddb-cluster-app` via `secretKeyRef` in app's namespace
|
||||
4. No permission grants needed — app user owns the database
|
||||
|
||||
## GitOps / ArgoCD Gotchas (hard-won, all confirmed live in this cluster)
|
||||
|
||||
🟠 **`kustomization.yaml` with an explicit `resources:` allowlist silently
|
||||
@@ -178,4 +248,4 @@ branch, which may not match your pinned version's schema at all.**
|
||||
|
||||
---
|
||||
|
||||
**Last updated:** 2026-07-22
|
||||
**Last updated:** 2026-07-23
|
||||
|
||||
Reference in New Issue
Block a user