rock/homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
216 Commits 3 Branches 0 Tags
ce019f5f3a422ce1582ae27f95976708b3efbbde
Commit Graph
12 Commits
Author SHA1 Message Date
Story Crater Bot ce019f5f3a fix(ddb): add database-level CREATE privilege for schema creation 
Authentik migrations need to CREATE SCHEMA (not just tables in public schema).
This requires GRANT CREATE ON DATABASE, not just schema-level permissions.

Added to PostSync Job:
- GRANT CREATE ON DATABASE authentik TO authentik
- GRANT CREATE ON DATABASE temporal TO temporal
- GRANT CREATE ON DATABASE temporal_visibility TO temporal

App user can grant these (it owns the databases).
 2026-07-23 09:36:23 -07:00
Story Crater Bot 2499cc241f fix(ddb): add PostSync Job for per-database schema permissions 
ROOT CAUSE: CNPG Database CR creates databases but doesn't grant schema
permissions to the owner role. Bootstrap DB owner 'app' retains CREATE
privilege on public schema, blocking authentik/temporal from creating tables.

SECURITY FIX: Removed insecure 'GRANT TO PUBLIC' from postInitApplicationSQL.

SOLUTION: PostSync Job connects as 'app' (DB owner) and grants schema
permissions to named roles (authentik, temporal) in their respective databases.
Runs after Database CRs reconcile, survives CNPG database recreation.

Pattern: Per-database grants via PostSync, not cluster-wide PUBLIC grants.
 2026-07-23 09:02:19 -07:00
Story Crater Bot 1c98628417 fix(ddb): grant universal schema permissions to all roles 
Adds SQL to postInitApplicationSQL granting schema permissions to PUBLIC.
Allows any role (authentik, temporal, etc) to create tables in databases.

For existing cluster: run SQL manually (done).
For future bootstrap: automatic via initdb.

Pattern for apps: Database CR + app-specific init Job optional (co-located).
 2026-07-23 08:07:34 -07:00
Story Crater Bot ec046cccde fix(ddb): use app user credentials in db-permissions Job 
ddb-cluster-superuser secret doesn't exist (not configured).
Use ddb-cluster-app secret instead - app is DB owner, can grant permissions.
 2026-07-23 08:05:03 -07:00
Story Crater Bot 9d485d1238 fix(ddb): add PostSync Job for database schema permissions 
CNPG Database CR creates DBs but doesn't grant schema permissions properly.
Database owner is 'app' instead of specified role (authentik, temporal).

PostSync Job grants ALL on schema public to both app and named roles,
ensuring applications can create tables. Runs after Database CRs reconcile.

Fixes: authentik InsufficientPrivilege error on migration.
 2026-07-23 08:02:44 -07:00
Story Crater Bot dafccd5d72 feat: complete GitOps migration, storage HA verification, and cluster fixes 
Major accomplishments from comprehensive cluster review:

## Storage HA (answering "are volumes replicated?")
- Verified 3-node Longhorn HA: ALL 17 volumes have 3 replicas
- Fixed CLAUDE.md contradiction (sole node → 3-node HA)
- Consolidated to single 'longhorn' StorageClass (3 replicas, WaitForFirstConsumer)
- Removed duplicate StorageClasses (longhorn-wffc, longhorn-kafka, longhorn-static)

## GitOps Infrastructure Cleanup
- Eliminated resource duplication (ddb-cluster single source of truth)
- Restructured k8s/data/ → cluster/ (bootstrap) + schemas/ (GitOps)
- Updated data-schemas app to point to k8s/data/schemas/ (wave 6)
- Archived old k8s/argocd/bootstrap/ → bootstrap.archived/

## Bootstrap Dependencies Fixed
- Added 05-wait-for-databases.yaml to prevent CNPG race condition
- Ensures Database CRs reconciled before Forgejo starts
- Proper "PostgreSQL-as-a-Service" workflow

## Longhorn CSI Plugin Fixed
- Added patch-csi-tolerations-job.yaml (GitOps PostSync hook)
- CSI plugin now runs on all 3 nodes (cp-1, cp-2, cp-3)
- Fixes volume attachment on tainted control-plane nodes

## Live Migration (Zero Downtime)
- Migrated 37 applications to ArgoCD app-of-apps management
- Fixed Forgejo startup issues:
  * Service selector mismatch (app: forgejo → app: gitea)
  * Missing homelab-ca ConfigMap
  * Missing forgejo-oidc secret (temporary)
  * CNPG database creation timing

## Documentation (10 comprehensive files)
- WHATS-NEXT.md - Daily GitOps workflow
- MIGRATION-STATUS.md - Cluster health report
- REVIEW-SUMMARY.md - Session overview
- GITOPS-REBUILD-PLAN.md - Architecture reference
- DDB-REVIEW.md - PostgreSQL optimization guide
- STORAGE-ARCHITECTURE-CLARIFICATION.md - Storage HA investigation
- BOOTSTRAP-DEPENDENCY-FIX.md - CNPG race condition fix
- STORAGECLASS-CONSOLIDATION.md - Single StorageClass rationale
- IMPLEMENTATION-CHECKLIST.md - Migration checklist
- bootstrap.sh - Automated bootstrap script

## Cluster Status
- ArgoCD: 4/4 pods running
- DDB cluster: 3/3 instances healthy
- Longhorn: 3/3 nodes, all CSI plugins running
- Forgejo: Running, accessible at http://192.168.1.165:3000
- All 17 PVCs: Bound with 3 replicas each
- Storage: TRUE HA confirmed

All future changes via git push only (100% GitOps).
 2026-07-22 23:56:34 -07:00
Story Crater Bot 46ec0caf5d fix(temporal): provision schema via CNPG temporal_visibility Database CR + enable chart schema setup/update jobs — both DBs had zero tables so server died on 'no usable database connection' 2026-07-21 21:09:03 -07:00
Story Crater Bot 5ce0b92186 feat(data): add CNPG managed roles + Database CRs for authentik/temporal — replaces missing helmfile post-sync user creation 
authentik/temporal DB users+databases were never provisioned (old helmfile hook
gone; db-init-job only made schemas in shared app DB). Adds managed.roles
(authentik/temporal login roles, passwords from basic-auth secrets) + Database CRs
(dedicated DBs owned by each role). Role secrets applied out-of-band (SOPS), not in
kustomize resources so data-schemas app doesn't choke on ciphertext.
 2026-07-20 23:36:21 -07:00
Story Crater Bot a207c56637 fix(k8s,docs): scale ddb-cluster to single instance, pin minio to storage namespace, document 3-CP topology in USAGE 2026-07-20 08:22:53 -07:00
Story Crater Bot 578a707867 feat(gitops): migrate domain to riotpiao.com, add CNPG + Forgejo HA on Redis/Postgres, wire ArgoCD apps — enables cluster rebuild after etcd wipe and unblocks the git-source chicken-egg via standalone Helm-source Applications 2026-07-19 09:29:17 -07:00
Story Crater Bot e2e17ae0fb feat(data): Add CNPG cluster + database schema initialization 
Create production PostgreSQL cluster via CNPG (3-node HA, Longhorn storage).

Schema initialization Job creates schemas for:
- Authentik (identity provider)
- Temporal (workflow engine)
- Vault (secrets management)
- App (generic application databases)

Database layer now captures complete IaC for stateful infrastructure.
Services find ready schemas when deployed.
 2026-07-16 14:53:59 -07:00
Story Crater Bot 2d7330798b refactor(k8s): Reorganize into 5-layer structure with production kustomizations 2026-07-16 14:28:19 -07:00