rock/homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
286 Commits 3 Branches 0 Tags
225866945a5765ed1379593676d336764a7bda9e
Commit Graph
8 Commits
Author SHA1 Message Date
Story Crater Bot a816045d3b fix(argocd): clone the public GitHub seed anonymously over HTTPS and delete the SSH deploy-key Secret — its private half had been committed in plaintext to a public remote, and a public repo needs no credential at all 2026-08-13 18:02:52 -07:00
Story Crater Bot 47e7a2b1d6 feat(bootstrap): add Phase 1c nginx ingress controller 
- Add p1_ingress() phase to install nginx-ingress-controller
- Create ingress-nginx namespace with privileged PodSecurity label
- Disable ServiceMonitor during bootstrap (Prometheus CRDs not installed yet)
- Add namespace.yaml with PodSecurity labels (allows hostPort)
- Filter cert-manager CRD errors (will be created by ArgoCD)
- Include ingress phase in bootstrap 'all' flow
 2026-07-25 12:39:07 -07:00
Story Crater Bot 8d63db9f3b fix(bootstrap): complete Phase 4 ArgoCD bootstrap with all permanent fixes 
- Fix ArgoCD Application schema: move syncOptions under syncPolicy (00-secrets.yaml)
- Remove helm install --wait flag (talos-cp-2 slow node timeout issue)
- Add comprehensive progress logging with timestamps to bootstrap.sh
- Fix SOPS key path (/Users/rockliang/.sops/key.txt, not homelab-age.key)
- Add local SOPS decryption for bootstrap secrets
- Add CNPG NetworkPolicy allowing app→database connectivity
- Disable Forgejo bundled dependencies (saves 66Gi storage)
- Inject database credentials via deployment.env (GITEA__DATABASE__*)
- Remove invalid ext4 mount options from StorageClass
- Add namespace manifests with PodSecurity labels
- Add encrypted forgejo-admin secret (SOPS)
- Reduce forgejo-db size 50Gi→25Gi per instance
- Prepare ArgoCD SOPS CMP plugin (for post-bootstrap)
 2026-07-25 12:24:30 -07:00
Story Crater Bot 95ae933489 feat(bootstrap): Phase-0 GitHub-seed bootstrap — root-app-github (SSH seed), deploy-key Secret template, cutover URL, bootstrap.sh runner (cilium→longhorn→cnpg→forgejo-db→argocd→cutover) 2026-07-23 21:01:12 -07:00
Story Crater Bot 1c7395d9e1 feat:Fix the bootstrap to be deploy key application 2026-07-23 19:07:39 -07:00
Story Crater Bot 4ad4df7965 refactor(temporal): adopt unified CNPG pattern - use 'app' user 
CHANGES:
  - temporal-values.yaml: user 'app', existingSecret 'ddb-cluster-app'
  - bootstrap.sh: Copy ddb-cluster-app to temporal namespace
  - Removed db-secret-sync directory (obsolete PostSync Job)
  - 60-applications.yaml: Removed db-secret-sync source from temporal Application

PATTERN (same as Forgejo/Authentik):
  1. Database CR: owner app
  2. bootstrap.sh: Copy ddb-cluster-app to temporal namespace
  3. App values: Reference ddb-cluster-app secret
  4. No PostSync Jobs needed

FIXES:
  - Temporal schema CrashLoopBackOff (wrong credentials)
  - Dropped/recreated databases with app owner (clean state)

Following CLAUDE.md CNPG pattern documentation.
 2026-07-23 10:57:53 -07:00
Story Crater Bot 751ae733d5 refactor(cnpg): adopt unified Forgejo pattern for all apps 
UNIFIED PATTERN: All apps follow same credential distribution

FORGEJO PATTERN (now universal):
  1. CNPG creates ddb-cluster-app in ddb namespace (source)
  2. bootstrap.sh copies to app namespaces (cicd, iam)
  3. Apps reference local copy via secretKeyRef
  4. No PostSync Jobs needed

CHANGES:
  - bootstrap.sh: Copy ddb-cluster-app to iam namespace (like cicd)
  - authentik-values.yaml: Reference local ddb-cluster-app via env vars
  - Removed: sync-db-credentials PostSync Job (not needed)
  - kustomization.yaml: Removed PostSync Job reference

BENEFITS:
  ✅ Same pattern as working Forgejo
  ✅ No complex PostSync Jobs
  ✅ bootstrap.sh handles setup for future clusters
  ✅ Simple secretKeyRef, no cross-namespace issues
  ✅ ArgoCD manages applications, not secrets

Database recreated with app owner (fresh migrations needed).
 2026-07-23 10:12:40 -07:00
Story Crater Bot dafccd5d72 feat: complete GitOps migration, storage HA verification, and cluster fixes 
Major accomplishments from comprehensive cluster review:

## Storage HA (answering "are volumes replicated?")
- Verified 3-node Longhorn HA: ALL 17 volumes have 3 replicas
- Fixed CLAUDE.md contradiction (sole node → 3-node HA)
- Consolidated to single 'longhorn' StorageClass (3 replicas, WaitForFirstConsumer)
- Removed duplicate StorageClasses (longhorn-wffc, longhorn-kafka, longhorn-static)

## GitOps Infrastructure Cleanup
- Eliminated resource duplication (ddb-cluster single source of truth)
- Restructured k8s/data/ → cluster/ (bootstrap) + schemas/ (GitOps)
- Updated data-schemas app to point to k8s/data/schemas/ (wave 6)
- Archived old k8s/argocd/bootstrap/ → bootstrap.archived/

## Bootstrap Dependencies Fixed
- Added 05-wait-for-databases.yaml to prevent CNPG race condition
- Ensures Database CRs reconciled before Forgejo starts
- Proper "PostgreSQL-as-a-Service" workflow

## Longhorn CSI Plugin Fixed
- Added patch-csi-tolerations-job.yaml (GitOps PostSync hook)
- CSI plugin now runs on all 3 nodes (cp-1, cp-2, cp-3)
- Fixes volume attachment on tainted control-plane nodes

## Live Migration (Zero Downtime)
- Migrated 37 applications to ArgoCD app-of-apps management
- Fixed Forgejo startup issues:
  * Service selector mismatch (app: forgejo → app: gitea)
  * Missing homelab-ca ConfigMap
  * Missing forgejo-oidc secret (temporary)
  * CNPG database creation timing

## Documentation (10 comprehensive files)
- WHATS-NEXT.md - Daily GitOps workflow
- MIGRATION-STATUS.md - Cluster health report
- REVIEW-SUMMARY.md - Session overview
- GITOPS-REBUILD-PLAN.md - Architecture reference
- DDB-REVIEW.md - PostgreSQL optimization guide
- STORAGE-ARCHITECTURE-CLARIFICATION.md - Storage HA investigation
- BOOTSTRAP-DEPENDENCY-FIX.md - CNPG race condition fix
- STORAGECLASS-CONSOLIDATION.md - Single StorageClass rationale
- IMPLEMENTATION-CHECKLIST.md - Migration checklist
- bootstrap.sh - Automated bootstrap script

## Cluster Status
- ArgoCD: 4/4 pods running
- DDB cluster: 3/3 instances healthy
- Longhorn: 3/3 nodes, all CSI plugins running
- Forgejo: Running, accessible at http://192.168.1.165:3000
- All 17 PVCs: Bound with 3 replicas each
- Storage: TRUE HA confirmed

All future changes via git push only (100% GitOps).
 2026-07-22 23:56:34 -07:00