Commit Graph
21 Commits
Author SHA1 Message Date
Story Crater Bot 08a3fd9bcb feat(phase0): configure ArgoCD SOPS decryption + update encrypted secrets
Phase 0 continuation: enable ArgoCD to decrypt SOPS-encrypted secrets on sync.

1. Update ArgoCD Helm values (terraform/argocd-bootstrap.tf):
   - Add SOPS_AGE_KEY_FILE env var to repoServer
   - Mount sops-age K8s Secret at /home/argocd/.sops
   - Add ConfigManagementPlugin for SOPS (detects *.enc.yaml files)

2. Update encrypted secrets with real values:
   - k8s/base/secrets.enc.yaml: encrypted with actual service credentials
   - All secret values encrypted at rest in git
   - ArgoCD decrypts on sync using K8s Secret + AGE key

Prerequisites:
  - K8s Secret created: kubectl create secret generic sops-age -n argocd --from-file=keys.txt=/Users/rockliang/.sops/key.txt
  - SOPS_AGE_KEY_FILE env var set in ArgoCD repoServer (done above)

Next: Phase 1 — migrate 9 hookless releases to ArgoCD + create Applications that reference encrypted secrets.
2026-08-18 15:08:01 -07:00
Story Crater BotandClaude Haiku 4.5 ee385e59a1 feat(phase0): setup SOPS for encrypted secret management
Phase 0 groundwork for helmfile→ArgoCD migration using SOPS (Secrets Operations):

1. Install SOPS + AGE encryption
   - AGE key generated and stored locally at ~/.sops/key.txt
   - Public key embedded in .sops.yaml for file encryption rules

2. Create K8s Secret for AGE private key
   - kubectl: create secret generic sops-age -n argocd --from-file=keys.txt=~/.sops/key.txt
   - ArgoCD will use this key to decrypt secrets at sync time

3. Encrypt initial secrets
   - k8s/base/secrets.enc.yaml: AES256_GCM encrypted secrets for all services
   - Placeholder values (will be replaced with real values per environment)
   - Secrets never visible in git (encrypted at rest)

4. Configure SOPS
   - .sops.yaml: creation rules for k8s/*/secrets.enc.yaml files
   - All future secret files auto-encrypt on edit (sops -e)

Setup: Store AGE key as K8s Secret in argocd namespace:
  export KUBECONFIG=cluster-config/kubeconfig
  kubectl create secret generic sops-age -n argocd --from-file=keys.txt=~/.sops/key.txt

Next: Configure ArgoCD Helm plugin to decrypt secrets on sync (Phase 0 continuation).

Co-Authored-By: Claude Haiku 4.5 <[email protected]>
2026-08-18 15:08:01 -07:00
Story Crater Bot 22e18c6f33 feat(minio): add loki storage buckets (chunks/ruler/admin/index) 2026-08-18 15:08:01 -07:00
Story Crater Bot 80e146e106 feat(minio): migrate to official MinIO Operator chart, TF-owned 2026-08-18 15:08:01 -07:00
Story Crater Bot 68dc426e53 feat(argocd): phase2 app-of-apps for 19 workloads + AppProject sourceRepos fix 2026-08-18 15:08:01 -07:00
Story Crater Bot 4e473978b5 Step 1 complete: Bootstrap layer with ArgoCD, cert-manager, namespaces imported to TF
- ArgoCD migrated to argocd namespace
- Cert-manager issuers/certs created
- 20 namespaces imported with pod-security labels
- S3 backend temporarily offline (MinIO), using local backup
- Pending: Remove metadata drift from helm releases, re-apply
2026-08-18 15:08:00 -07:00
Story Crater Bot db7f9ef125 feat: three-tier log aggregation for Loki
Critical services (iam/monitoring/temporal/cicd) keep 100% logs.
Others get 50% sampling + selective drops (health/debug noise).
Balances log volume (40-50% reduction) with error visibility.
2026-08-18 15:08:00 -07:00
Story Crater Bot 78defa0423 feat: track full CoreDNS Deployment manifest, add topologySpreadConstraints
CoreDNS is Talos-bootstrapped and previously untracked except for its
ConfigMap. Pull the full live spec into one file as the single source of
truth, add topologySpreadConstraints so the 2 replicas don't land on the
same node. ScheduleAnyway (not DoNotSchedule) to avoid blocking scheduling
if a node is briefly unavailable.
2026-08-18 15:08:00 -07:00
Story Crater Bot be69b908c2 feat: spread management-service pods across nodes via topologySpreadConstraints
3-9 replicas (HPA) previously relied on implicit scheduler spreading.
ScheduleAnyway (not DoNotSchedule) so pods still get scheduled if a node
is briefly unavailable, just less evenly.
2026-08-18 15:08:00 -07:00
Story Crater Bot 3c9daa023e fix: temporal service config - add explicit ClusterIP services for history/matching 2026-08-18 15:08:00 -07:00
Story Crater Bot a1dd91e56f feat: point SQS charts to public GHCR image
Forgejo registry unreachable from worker nodes (network isolation +
host-to-ClusterIP routing gaps). Both management-service and queue-operator
now ship from the same public GHCR image, with queue-operator selected via
command override.
2026-08-18 15:08:00 -07:00
Story Crater Bot 9dbe8b6f43 remove: strip all oauth2-proxy deployments
- Delete oauth2-proxy helm releases from helmfile (temporal, kmsvc, longhorn, portainer)
- Remove oauth2-proxy manifests and ingress redirects
- Add direct ingress for kmsvc management service
- Update temporal/portainer/longhorn ingress comments to reflect direct service exposure

Services now accessible without oauth2-proxy layer.
2026-08-18 15:08:00 -07:00
Story Crater Bot 1eef4711e5 k8s/aux: add cert-manager longhorn dashboard forge dev-tools and shadowsocks
- cert-manager ClusterIssuers (LetsEncrypt + homelab-ca)
- Longhorn storage dashboard
- Portainer dashboard config
- Forgejo git service
- Claude terminal remote access
- Shadowsocks tunnel for remote access
2026-08-18 15:08:00 -07:00
Story Crater Bot f64cde0687 k8s/cilium: add lb-ipam pool configuration
- Cilium LB-IPAM pool (192.168.1.160-192.168.1.170)
- Fixed IP assignment for LoadBalancer services
2026-08-18 15:08:00 -07:00
Story Crater Bot 61be24e27f k8s/services: add ingress networking portainer llm and project guides
- Nginx ingress + TLS termination (homelab-ca)
- Portainer container UI
- CoreDNS internal DNS rewrites
- DuckDNS DDNS updater
- Ollama LLM inference
- 8 project-usage guides (team reference)
2026-08-18 15:08:00 -07:00
Story Crater Bot 69ad5c371c k8s/messaging: add kafka kmsvc and temporal workflows
- Kafka 3-broker cluster (RF=3, min-ISR=2)
- kmsvc SQS-like API on Kafka
- Redis dedup (standalone, can extend to HA)
- Temporal workflow orchestration (Cassandra backend)
2026-08-18 15:08:00 -07:00
Story Crater Bot a3f261f548 k8s/ci-cd: add forgejo gitops and argocd deployment
- Forgejo git forge + OCI registry
- Argo CD pull-based GitOps
- Private CA TLS (self-signed 10-year cert)
- Machine credentials scoped to repositories
2026-08-18 15:08:00 -07:00
Story Crater Bot 0af06b1239 k8s/monitoring: add prometheus grafana loki observability
- Loki log aggregation (MinIO backed, 10-day retention)
- Promtail daemonset (pod + talos journal logs)
- Prometheus + kube-state-metrics
- Grafana dashboards (6-row template per service)
2026-08-18 15:08:00 -07:00
Story Crater Bot 831dd50805 k8s/iam: add cloudnativepg postgres and vault + authentik
- PostgreSQL 3-replica HA with pgvector
- Vault S3 storage backend (MinIO)
- Authentik federated OIDC provider
- Vault auto-unseal via postStart hook
2026-08-18 15:08:00 -07:00
Story Crater Bot 7da222e243 k8s/storage: add minio s3 with 3-way replication and oidc
- MinIO 3-node site replication (az-a/b/c)
- S3 backend for Loki chunks (10-day retention)
- OIDC integration with Authentik
- envFrom for secret injection
2026-08-18 15:08:00 -07:00
Story Crater Bot df777fb829 k8s: add base namespace and pod disruption budgets
- Namespace setup script with PSP/RBAC
- PodDisruptionBudgets for all services (zero-downtime drain)
2026-08-18 15:08:00 -07:00