rock/homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
179 Commits 3 Branches 0 Tags
1ad52bf89cf687a350b15d2f68d2b9af04cc5536
Commit Graph
33 Commits
Author SHA1 Message Date
Story Crater Bot 2866a2aa93 fix(cilium): restrict L2 announcement to control-plane nodes — GPU worker lacks eno1 (Mellanox enp28s0f*), so when it won the .160 lease it couldn't ARP the VIP, black-holing all ingress (flapped on reboots) 2026-08-18 15:08:04 -07:00
Story Crater Bot c3ffc611f4 fix(homarr): add AUTH_OIDC_URI + email account linking — homarr hides the Authentik sign-in button unless AUTH_OIDC_URI (authorize endpoint) is set alongside AUTH_OIDC_ISSUER (per authentik/homarr SSO docs); was the missing var 2026-08-18 15:08:04 -07:00
Story Crater Bot 61b906cce1 feat(monitoring): enable Alertmanager (null receiver, longhorn PVC, az-a) + fix forgejo-rules ns forgejo->cicd — alerting delivery was disabled; forgejo PrometheusRule targeted a nonexistent namespace 2026-08-18 15:08:04 -07:00
Story Crater Bot 650fcf1b61 fix(coredns): own Corefile+hostname rewrites via Talos inlineManifest (single-source terraform/files/coredns/Corefile), drop ArgoCD coredns-config app — in-cluster *.riotpiao.com now resolves to nginx ingress so MinIO/OIDC discovery works; update cp-2 IP .213->.214 2026-08-18 15:08:04 -07:00
Story Crater Bot cdf1cdeb18 feat(terraform): GPU worker node support (schematic, interface/diskSelector/swap tuning, gpu-node label, NVIDIA LTS extensions) 2026-08-18 15:08:04 -07:00
Story Crater Bot 5b3307ffee fix(bootstrap): correct cluster config + complete Phase4 ArgoCD bootstrap permanent fixes 2026-08-18 15:08:03 -07:00
Story Crater Bot fef07c058a refactor(argocd): remove orphaned infrastructure Applications - bootstrap is source of truth 
REMOVED ORPHANED APPLICATIONS:
  - cnpg-operator (OutOfSync, conflicted with bootstrap)
  - forgejo (OutOfSync, conflicted with bootstrap)
  - ingress-nginx-bootstrap (orphaned, no ownerReferences)

ARCHITECTURE NOW CLEAN:
  ✅ Bootstrap: 7 manifests (infrastructure base for regional deployment)
     - ArgoCD, CNPG operator, DDB, Forgejo, ingress-nginx, namespaces, wait-for-databases
  ✅ ArgoCD: 32 Applications (all services/apps)
  ✅ No duplicate management

DEPLOYMENT FLOW:
  1. kubectl apply -k k8s/bootstrap-local/ (infrastructure)
  2. kubectl apply -k k8s/argocd/root/ (app-of-apps)
  3. ArgoCD auto-syncs from Forgejo (applications)

CLEANUP:
  - Archived old bootstrap configs (k8s/argocd/bootstrap.archived/)
  - Deleted orphaned Applications (ArgoCD tracking only, resources untouched)

Bootstrap remains single source of truth for infrastructure.
ArgoCD manages all applications and services.
 2026-08-18 15:08:03 -07:00
Story Crater Bot 97330d780c feat(terraform): add per-node Cloudflare Tunnel cert SANs to controlplane certSANs — remote talosctl/kubectl over tunnel pass TLS verification 
Adds optional cloudflare_talos_sans (machine.certSANs, talos API :50000) and
cloudflare_apiserver_sans (cluster.apiServer.certSANs, kube-apiserver :6443) per
control-plane node. cp-1 gets cp1.homelab + cp1-talos.homelab; cp-2/cp-3 get
their cpN-talos.homelab. Values set in gitignored tfvars.
 2026-08-18 15:08:02 -07:00
Story Crater Bot 5b34e71111 feat(terraform): restructure control planes into a 3-node map with LAN etcd advertise and live machine CA — enables talos-cp-1/2/3 HA and drops worker configs 2026-08-18 15:08:02 -07:00
Story Crater Bot 54bfb5ade6 feat(gitops): migrate domain to riotpiao.com, add CNPG + Forgejo HA on Redis/Postgres, wire ArgoCD apps — enables cluster rebuild after etcd wipe and unblocks the git-source chicken-egg via standalone Helm-source Applications 2026-08-18 15:08:02 -07:00
Story Crater Bot 491e88e493 feat(ci,iac): Consolidate Forgejo CI workflows and add Talos Terraform IaC 
Consolidate three separate Forgejo Actions (argocd-sync, security-scan, validate-k8s) into single cluster-ci workflow for cleaner CI/CD pipeline with proper job sequencing and reduced auth overhead.

Add Terraform configuration for Talos cluster machine configs:
- Provider setup for Talos
- Centralized variables for CP and worker configs
- Template-based config generation for controlplane.yaml and worker-*.yaml
- Sensitive data separated in terraform.tfvars (gitignored)
- Local state tracking for infrastructure
 2026-08-18 15:08:01 -07:00
Story Crater Bot 563f720d09 refactor: retire Terraform, migrate to pure ArgoCD GitOps + CI validation 2026-08-18 15:08:01 -07:00
Story Crater Bot adbad3d97e fix(ci): use direct in-cluster Kubernetes auth for CI runner — drop kubeconfig file dependency 2026-08-18 15:08:01 -07:00
Story Crater Bot 3833c0d119 fix: terraform fmt — normalize formatting across all files 2026-08-18 15:08:01 -07:00
Story Crater Bot df9d9845c0 fix(ci): correct core-cli auth + S3 backend config for CI runner (iterate) 2026-08-18 15:08:01 -07:00
Story Crater Bot f16f439feb feat: Terraform CI via Forgejo Actions + MinIO S3 state backend 
- ArgoCD manages MinIO (phase 0), Terraform manages infrastructure
- Runner workflow: pulls state from S3, validates, plans, applies
- 34 resources imported to state, S3 backend operational
- Fixed AppProject repos, S3 endpoint deprecation, runner package manager
 2026-08-18 15:08:01 -07:00
Story Crater Bot 292146bce4 feat(phase4): ArgoCD-driven Terraform apply via PVC imports (Pod Job approach tried and reverted) 2026-08-18 15:08:01 -07:00
Story Crater Bot 0b4a79b5ca fix(argocd): use homelab-ca wildcard TLS instead of --insecure mode 2026-08-18 15:08:01 -07:00
Story Crater Bot 08a3fd9bcb feat(phase0): configure ArgoCD SOPS decryption + update encrypted secrets 
Phase 0 continuation: enable ArgoCD to decrypt SOPS-encrypted secrets on sync.

1. Update ArgoCD Helm values (terraform/argocd-bootstrap.tf):
   - Add SOPS_AGE_KEY_FILE env var to repoServer
   - Mount sops-age K8s Secret at /home/argocd/.sops
   - Add ConfigManagementPlugin for SOPS (detects *.enc.yaml files)

2. Update encrypted secrets with real values:
   - k8s/base/secrets.enc.yaml: encrypted with actual service credentials
   - All secret values encrypted at rest in git
   - ArgoCD decrypts on sync using K8s Secret + AGE key

Prerequisites:
  - K8s Secret created: kubectl create secret generic sops-age -n argocd --from-file=keys.txt=/Users/rockliang/.sops/key.txt
  - SOPS_AGE_KEY_FILE env var set in ArgoCD repoServer (done above)

Next: Phase 1 — migrate 9 hookless releases to ArgoCD + create Applications that reference encrypted secrets.
 2026-08-18 15:08:01 -07:00
Story Crater BotandClaude Haiku 4.5 f2b6ad1c60 feat(terraform): import Longhorn StorageClasses and app PVCs to Terraform state 
- Phase 1: longhorn, longhorn-kafka StorageClasses (cluster-wide defaults)
- Phase 2 pilot: grafana, loki, portainer, forgejo PVCs
- All imports protected by lifecycle.prevent_destroy
- Removes Helm annotations (meta.helm.sh/*) to prevent dual-ownership conflicts
- Remote state backend (MinIO S3) syncs automatically on plan/apply
- Import-only approach: zero data loss, existing volumes untouched
- See terraform/LONGHORN_PVC_IMPORT.md for execution record

Co-Authored-By: Claude Haiku 4.5 <[email protected]>
 2026-08-18 15:08:01 -07:00
Story Crater Bot 29d5ba3e6e chore(terraform): clean up TF state git-tracking (gitignore, drop cached state, dep bump) 2026-08-18 15:08:01 -07:00
Story Crater Bot 49eabd0e46 feat(terraform): enable S3 remote state backend (MinIO) 
Migrate terraform state from local file to MinIO S3 bucket (terraform-state).
Backend config: https://minio-api.riotpiao.homelab.com (external endpoint).
State now persisted remotely, shared across team, safe for cluster rebuild.

Also added terraform-state bucket to MinIO managed buckets.
 2026-08-18 15:08:01 -07:00
Story Crater Bot 22e18c6f33 feat(minio): add loki storage buckets (chunks/ruler/admin/index) 2026-08-18 15:08:01 -07:00
Story Crater Bot acfc9cfc04 feat(authentik): import 24 resources to TF; chore(bootstrap): add cilium to TF 
Import all live authentik resources (groups, users, oauth2 providers, applications)
into terraform state via authentik-generated.tf. Provider config in authentik-config.tf.
Resources are drift-free and match live cluster.

Add cilium CNI to bootstrap helm_release.for_each (1.19.5, kube-system).
Cilium was unmanaged (helmfile-only); now IaC-owned. Critical path for
cluster rebuild recovery. Adds cilium repo to helm-repositories.tf.
 2026-08-18 15:08:01 -07:00
Story Crater Bot 80e146e106 feat(minio): migrate to official MinIO Operator chart, TF-owned 2026-08-18 15:08:01 -07:00
Story Crater Bot 07b785e5fc Fix: Inject homelab-ca cert into ArgoCD repo-server 
- Mount homelab-ca-secret for TLS verification
- Allows repo-server to reach forgejo.riotpiao.homelab.com
- Fixes x509 certificate verification error
 2026-08-18 15:08:01 -07:00
Story Crater Bot bb1239ab0d Re-enable cert-manager manifests for TF import 
- ClusterIssuers + Certificates now back in TF
- Will import existing live resources
 2026-08-18 15:08:01 -07:00
Story Crater Bot 68dc426e53 feat(argocd): phase2 app-of-apps for 19 workloads + AppProject sourceRepos fix 2026-08-18 15:08:01 -07:00
Story Crater Bot da0ec35f6c fix(terraform): set ingress-nginx PodSecurity to privileged, keep TF-managed 2026-08-18 15:08:01 -07:00
Story Crater Bot f64fda297e Temp: disable kubernetes_manifest cert-manager resources (already live) 
- Will import separately after helm issues resolved
- Avoids re-create conflicts during bootstrap apply
 2026-08-18 15:08:01 -07:00
Story Crater Bot 1c1b924d1d Fix: downgrade ArgoCD to 7.3.3, ignore helm metadata drift 
- ArgoCD 7.9.1 -> 7.3.3 (match live cluster)
- Ignore helm release metadata in lifecycle rules
- Prevents unnecessary upgrade attempts
 2026-08-18 15:08:00 -07:00
Story Crater Bot 4e473978b5 Step 1 complete: Bootstrap layer with ArgoCD, cert-manager, namespaces imported to TF 
- ArgoCD migrated to argocd namespace
- Cert-manager issuers/certs created
- 20 namespaces imported with pod-security labels
- S3 backend temporarily offline (MinIO), using local backup
- Pending: Remove metadata drift from helm releases, re-apply
 2026-08-18 15:08:00 -07:00
Story Crater Bot 7835ac2932 feat: Terraform foundation for cluster & app bootstrap 
Phase 1 infrastructure-as-code setup:
- Core providers (kubernetes, helm, null)
- 15 Helm repositories (grafana, minio, prometheus, etc.)
- Namespace scaffolding (15 namespaces with pod-security labels)
- Storage classes (longhorn, longhorn-kafka with prevent_destroy)
- TLS certificate bootstrap (selfsigned, CA, wildcard cert)
- Remote state backend config (local for now, S3/GCS TODO)
- Variable definitions for all secrets/OIDC clients

Tested: terraform plan passes with no changes (bootstrap infrastructure ready)
Next: Create 25 helm_release resources (Phase 2-4)

Kept helmfile intact; network/Cilium managed via helmfile (no config risk)
Co-Authored-By: Claude Haiku 4.5 <[email protected]>
 2026-08-18 15:08:00 -07:00