Commit Graph
34 Commits
Author SHA1 Message Date
Story Crater Bot ed7be6f229 TF: Add minio-operator Helm repo to ArgoCD config + AppProject sourceRepos 2026-07-14 14:32:45 -07:00
Story Crater Bot 418ab7bfc2 Fix: minio-operator uses official MinIO Operator Helm chart 2026-07-14 14:26:25 -07:00
Story Crater Bot ca8525c625 Add minio-operator Application to deploy operator before Tenant 2026-07-14 14:26:01 -07:00
Story Crater Bot 8a3a892cbd Fix: Inject homelab-ca cert into ArgoCD repo-server
- Mount homelab-ca-secret for TLS verification
- Allows repo-server to reach forgejo.riotpiao.homelab.com
- Fixes x509 certificate verification error
2026-07-14 14:01:14 -07:00
Story Crater Bot cdacdd8d11 Re-enable cert-manager manifests for TF import
- ClusterIssuers + Certificates now back in TF
- Will import existing live resources
2026-07-14 13:55:08 -07:00
Story Crater Bot 263a48a22d Fix: ArgoCD AppProject sourceRepos for correct forgejo URL
- Changed from forgejo.forge.riotpiao.homelab.com/rock/* to forgejo.riotpiao.homelab.com/riotpiao.com/*
- Allows homelab root app to access workload app manifests
2026-07-14 13:52:53 -07:00
Story Crater Bot 47c0301a43 Step 2: ArgoCD app-of-apps manifests for 19 workloads
Wave 0: minio, strimzi-operator, kmsvc-redis, prometheus
Wave 1: vault, loki, cloudnative-pg, authentik (manual-sync), temporal, kafka-cluster
Wave 2: queue-crd, management-service, grafana, promtail, forgejo
Wave 3: forgejo-runner, portainer

All auto-sync except authentik (manual-sync only for IAM safety)
2026-07-14 13:52:18 -07:00
Story Crater Bot edd4ea7fe4 Fix: set ingress-nginx to privileged PodSecurity level
- privileged level allows hostPort (80/443) required for nginx
- Other namespaces remain at baseline for security
- Cleaner than exempting namespace entirely
2026-07-14 13:43:56 -07:00
Story Crater Bot 086ad9f9a9 Fix: exempt ingress-nginx from PodSecurity policy
- restricted policy forbids hostPort (80/443) — broke nginx
- Remove pod-security labels from ingress-nginx namespace entirely
- Other namespaces remain at baseline level
2026-07-14 13:41:36 -07:00
Story Crater Bot 3eabb847fd Re-add ingress-nginx to TF bootstrap (PodSecurity policy fixed)
- ingress-nginx now has restricted policy level (allows hostPort)
- Previous timeout was due to policy blocking pod deployment
- Re-importing helm release to TF management
2026-07-14 13:37:22 -07:00
Story Crater Bot 6109477bf9 Skip TF management of ingress-nginx (helm timeout issues)
- ingress-nginx already deployed and working in cluster
- Helm updates timeout repeatedly (5+ min with context deadline exceeded)
- Remove from bootstrap releases; manage separately via helm/kubectl
- cert-manager + reloader continue via TF
2026-07-14 13:32:40 -07:00
Story Crater Bot 20c634fd03 Fix: ingress-nginx PodSecurity policy enforcement level
- ingress-nginx requires hostPort (80/443) which is forbidden at baseline level
- Change to restricted enforcement level to allow hostPort
- Other namespaces remain at baseline for security
2026-07-14 13:31:16 -07:00
Story Crater Bot 074b43e1f2 Temp: disable kubernetes_manifest cert-manager resources (already live)
- Will import separately after helm issues resolved
- Avoids re-create conflicts during bootstrap apply
2026-07-14 13:26:26 -07:00
Story Crater Bot c2e084c7c2 Fix: downgrade ArgoCD to 7.3.3, ignore helm metadata drift
- ArgoCD 7.9.1 -> 7.3.3 (match live cluster)
- Ignore helm release metadata in lifecycle rules
- Prevents unnecessary upgrade attempts
2026-07-14 13:16:55 -07:00
Story Crater Bot dd608d3231 Step 1 complete: Bootstrap layer with ArgoCD, cert-manager, namespaces imported to TF
- ArgoCD migrated to argocd namespace
- Cert-manager issuers/certs created
- 20 namespaces imported with pod-security labels
- S3 backend temporarily offline (MinIO), using local backup
- Pending: Remove metadata drift from helm releases, re-apply
2026-07-14 13:14:46 -07:00
Story Crater Bot 9a4d486b86 feat: Terraform foundation for cluster & app bootstrap
Phase 1 infrastructure-as-code setup:
- Core providers (kubernetes, helm, null)
- 15 Helm repositories (grafana, minio, prometheus, etc.)
- Namespace scaffolding (15 namespaces with pod-security labels)
- Storage classes (longhorn, longhorn-kafka with prevent_destroy)
- TLS certificate bootstrap (selfsigned, CA, wildcard cert)
- Remote state backend config (local for now, S3/GCS TODO)
- Variable definitions for all secrets/OIDC clients

Tested: terraform plan passes with no changes (bootstrap infrastructure ready)
Next: Create 25 helm_release resources (Phase 2-4)

Kept helmfile intact; network/Cilium managed via helmfile (no config risk)
Co-Authored-By: Claude Haiku 4.5 <[email protected]>
2026-07-14 09:27:24 -07:00
Story Crater Bot 4d32e2765b feat: three-tier log aggregation for Loki
Critical services (iam/monitoring/temporal/cicd) keep 100% logs.
Others get 50% sampling + selective drops (health/debug noise).
Balances log volume (40-50% reduction) with error visibility.
2026-07-13 17:52:59 -07:00
Story Crater Bot 0d4e98d88b feat: track full CoreDNS Deployment manifest, add topologySpreadConstraints
CoreDNS is Talos-bootstrapped and previously untracked except for its
ConfigMap. Pull the full live spec into one file as the single source of
truth, add topologySpreadConstraints so the 2 replicas don't land on the
same node. ScheduleAnyway (not DoNotSchedule) to avoid blocking scheduling
if a node is briefly unavailable.
2026-07-13 16:51:49 -07:00
Story Crater Bot ebf97f573e feat: spread management-service pods across nodes via topologySpreadConstraints
3-9 replicas (HPA) previously relied on implicit scheduler spreading.
ScheduleAnyway (not DoNotSchedule) so pods still get scheduled if a node
is briefly unavailable, just less evenly.
2026-07-13 16:43:58 -07:00
Story Crater Bot 05b088ca48 docs: update USAGE.md for core CLI + IAM management
- Rename talos → core CLI references
- Add IAM management section (Authentik apps, groups, users)
- Add workflows for secret rotation and user management
- Link to detailed core CLI docs (~/workplace/core/USAGE.md)
- Add port-forwarding and troubleshooting tips
2026-07-13 14:12:12 -07:00
Story Crater Bot b33fdde5b7 fix: temporal service config - add explicit ClusterIP services for history/matching 2026-07-13 13:20:50 -07:00
Story Crater Bot cd1c5691e8 feat: point SQS charts to public GHCR image
Forgejo registry unreachable from worker nodes (network isolation +
host-to-ClusterIP routing gaps). Both management-service and queue-operator
now ship from the same public GHCR image, with queue-operator selected via
command override.
2026-07-13 10:31:22 -07:00
Story Crater Bot e6d7626fb6 remove: strip all oauth2-proxy deployments
- Delete oauth2-proxy helm releases from helmfile (temporal, kmsvc, longhorn, portainer)
- Remove oauth2-proxy manifests and ingress redirects
- Add direct ingress for kmsvc management service
- Update temporal/portainer/longhorn ingress comments to reflect direct service exposure

Services now accessible without oauth2-proxy layer.
2026-07-11 19:36:31 -07:00
Story Crater Bot e1d0cfd70b k8s/aux: add cert-manager longhorn dashboard forge dev-tools and shadowsocks
- cert-manager ClusterIssuers (LetsEncrypt + homelab-ca)
- Longhorn storage dashboard
- Portainer dashboard config
- Forgejo git service
- Claude terminal remote access
- Shadowsocks tunnel for remote access
2026-07-11 19:19:59 -07:00
Story Crater Bot 18f2f94f8e k8s/cilium: add lb-ipam pool configuration
- Cilium LB-IPAM pool (192.168.1.160-192.168.1.170)
- Fixed IP assignment for LoadBalancer services
2026-07-11 19:19:44 -07:00
Story Crater Bot 6d5a0ba205 k8s/services: add ingress networking portainer llm and project guides
- Nginx ingress + TLS termination (homelab-ca)
- Portainer container UI
- CoreDNS internal DNS rewrites
- DuckDNS DDNS updater
- Ollama LLM inference
- 8 project-usage guides (team reference)
2026-07-11 19:17:54 -07:00
Story Crater Bot 1c02e2b831 k8s/messaging: add kafka kmsvc and temporal workflows
- Kafka 3-broker cluster (RF=3, min-ISR=2)
- kmsvc SQS-like API on Kafka
- Redis dedup (standalone, can extend to HA)
- Temporal workflow orchestration (Cassandra backend)
2026-07-11 19:17:42 -07:00
Story Crater Bot 4ab596196e k8s/ci-cd: add forgejo gitops and argocd deployment
- Forgejo git forge + OCI registry
- Argo CD pull-based GitOps
- Private CA TLS (self-signed 10-year cert)
- Machine credentials scoped to repositories
2026-07-11 19:17:34 -07:00
Story Crater Bot 63d7256b9e k8s/monitoring: add prometheus grafana loki observability
- Loki log aggregation (MinIO backed, 10-day retention)
- Promtail daemonset (pod + talos journal logs)
- Prometheus + kube-state-metrics
- Grafana dashboards (6-row template per service)
2026-07-11 19:17:28 -07:00
Story Crater Bot 674c8f0d66 k8s/iam: add cloudnativepg postgres and vault + authentik
- PostgreSQL 3-replica HA with pgvector
- Vault S3 storage backend (MinIO)
- Authentik federated OIDC provider
- Vault auto-unseal via postStart hook
2026-07-11 19:17:22 -07:00
Story Crater Bot 36aea89e47 k8s/storage: add minio s3 with 3-way replication and oidc
- MinIO 3-node site replication (az-a/b/c)
- S3 backend for Loki chunks (10-day retention)
- OIDC integration with Authentik
- envFrom for secret injection
2026-07-11 19:16:56 -07:00
Story Crater Bot 11c26f3f29 k8s: add base namespace and pod disruption budgets
- Namespace setup script with PSP/RBAC
- PodDisruptionBudgets for all services (zero-downtime drain)
2026-07-11 19:16:50 -07:00
Story Crater Bot 11898733e8 infra: add helmfile and talos cluster configuration
- helmfile: 18 releases across 22 namespaces
- Pod disruption budgets for zero-downtime drain
- Nginx ingress with LoadBalancer + Cilium LB-IPAM
- Cluster bootstrap hooks
2026-07-11 19:16:41 -07:00
Story Crater Bot 5b00616f41 docs: add foundation docs and cluster configuration templates
- README: cluster architecture, quick start, use cases
- USAGE: stack topology, custom CLI reference
- TROUBLESHOOTING: operational safety rules
- .env.example: configuration template
- Makefile: build shortcuts
2026-07-11 19:16:34 -07:00