feat(ci,iac): Consolidate Forgejo CI workflows and add Talos Terraform IaC
Consolidate three separate Forgejo Actions (argocd-sync, security-scan, validate-k8s) into single cluster-ci workflow for cleaner CI/CD pipeline with proper job sequencing and reduced auth overhead. Add Terraform configuration for Talos cluster machine configs: - Provider setup for Talos - Centralized variables for CP and worker configs - Template-based config generation for controlplane.yaml and worker-*.yaml - Sensitive data separated in terraform.tfvars (gitignored) - Local state tracking for infrastructure
This commit is contained in:
Story Crater Bot
@@ -1,130 +0,0 @@
|
||||
# Phase 1: Migrate 9 Hookless Releases to ArgoCD
|
||||
|
||||
## Releases to migrate (no presync/postsync hooks)
|
||||
1. strimzi-operator
|
||||
2. kafka-cluster
|
||||
3. kmsvc-redis
|
||||
4. queue-crd
|
||||
5. management-service
|
||||
6. promtail
|
||||
7. blackbox-exporter
|
||||
8. portainer
|
||||
9. claude-terminal
|
||||
|
||||
## Pattern per release
|
||||
|
||||
### 1. Create Application in k8s/argocd/apps/
|
||||
|
||||
**Helmfile source:**
|
||||
```yaml
|
||||
- name: strimzi-operator
|
||||
namespace: sqs
|
||||
createNamespace: true
|
||||
chart: strimzi/strimzi-kafka-operator
|
||||
version: 0.46.0
|
||||
values:
|
||||
- watchNamespaces: ["sqs"]
|
||||
```
|
||||
|
||||
**ArgoCD Application:**
|
||||
```yaml
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: strimzi-operator
|
||||
namespace: argocd
|
||||
annotations:
|
||||
argocd.argoproj.io/sync-wave: "0"
|
||||
spec:
|
||||
project: homelab
|
||||
source:
|
||||
repoURL: https://strimzi.io/charts/
|
||||
chart: strimzi-kafka-operator
|
||||
targetRevision: 0.46.0
|
||||
helm:
|
||||
values: |
|
||||
watchNamespaces: ["sqs"]
|
||||
destination:
|
||||
server: https://kubernetes.default.svc
|
||||
namespace: sqs
|
||||
syncPolicy:
|
||||
automated:
|
||||
prune: true
|
||||
selfHeal: true
|
||||
syncOptions:
|
||||
- CreateNamespace=true
|
||||
```
|
||||
|
||||
### 2. Test ArgoCD application (dry-run)
|
||||
|
||||
```bash
|
||||
# If app created, check diff
|
||||
kubectl apply -f k8s/argocd/apps/{WAVE}-{RELEASE}.yaml --dry-run=client -o yaml
|
||||
|
||||
# Or: use argocd CLI
|
||||
argocd app diff {RELEASE} # should be clean (no diffs) if spec matches helmfile
|
||||
```
|
||||
|
||||
### 3. Remove from helmfile
|
||||
|
||||
Delete the release block from `helmfile.yaml.gotmpl`, commit.
|
||||
|
||||
```bash
|
||||
# Verify no unintended drift
|
||||
helmfile diff
|
||||
```
|
||||
|
||||
### 4. Commit
|
||||
|
||||
Per-release commit (one app at a time).
|
||||
|
||||
## Helmfile → Application mapping
|
||||
|
||||
| Helmfile | Chart | Namespace | Version | Wave | Status |
|
||||
|----------|-------|-----------|---------|------|--------|
|
||||
| strimzi-operator | strimzi/strimzi-kafka-operator | sqs | 0.46.0 | 0 | TODO |
|
||||
| kmsvc-redis | bitnami/redis | sqs | 20.6.0 | 0 | TODO |
|
||||
| prometheus | prometheus-community/kube-prometheus-stack | monitoring | latest | 0 | TODO |
|
||||
| kafka-cluster | ./k8s/sqs/charts/kafka-cluster | sqs | local | 1 | TODO |
|
||||
| queue-crd | ./k8s/sqs/charts/queue-crd | sqs | local | 1 | TODO |
|
||||
| management-service | ./k8s/sqs/charts/management-service | sqs | local | 1 | TODO |
|
||||
| promtail | grafana/promtail | logging | latest | 1 | TODO |
|
||||
| blackbox-exporter | prometheus-community/prometheus-blackbox-exporter | monitoring | ~11 | 1 | TODO |
|
||||
| portainer | portainer/portainer | dashboard | latest | 3 | TODO |
|
||||
| claude-terminal | ./k8s/dev-tools | dev-tools | local | 3 | TODO |
|
||||
|
||||
## Local charts mapping
|
||||
|
||||
For local charts (e.g., `./k8s/sqs/charts/kafka-cluster`), use `source.path` instead of `source.chart`:
|
||||
|
||||
```yaml
|
||||
source:
|
||||
repoURL: https://forgejo.riotpiao.homelab.com/riotpiao.com/homelab.git
|
||||
targetRevision: main
|
||||
path: k8s/sqs/charts/kafka-cluster
|
||||
helm:
|
||||
valueFiles:
|
||||
- values.yaml # or path to values override
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
After all 9 releases migrated:
|
||||
|
||||
```bash
|
||||
argocd app list | grep -E "strimzi-operator|kmsvc-redis|prometheus|kafka-cluster|queue-crd|management-service|promtail|blackbox-exporter|portainer|claude-terminal"
|
||||
|
||||
# All should show: Synced | Healthy
|
||||
```
|
||||
|
||||
Then:
|
||||
```bash
|
||||
helmfile diff # should show no diffs (these releases removed from helmfile)
|
||||
```
|
||||
|
||||
## Rollback
|
||||
|
||||
If an Application breaks the cluster during migration:
|
||||
1. Keep helmfile release block in git (don't delete until verified)
|
||||
2. If needed: `helmfile apply -l name={RELEASE}` restores from helmfile
|
||||
3. Debug the Application spec and retry
|
||||
@@ -1,113 +0,0 @@
|
||||
# Terraform State Management
|
||||
|
||||
## Overview
|
||||
|
||||
Terraform state for the homelab cluster is managed using a hybrid approach:
|
||||
- **Remote backend:** S3 (MinIO) for centralized, shared state
|
||||
- **Local backup:** Git-ignored backups for disaster recovery
|
||||
|
||||
## Backend Configuration
|
||||
|
||||
State is stored in MinIO S3:
|
||||
|
||||
```
|
||||
Bucket: terraform-state
|
||||
Key: homelab/terraform.tfstate
|
||||
Endpoint: https://minio-api.riotpiao.homelab.com
|
||||
Profile: minio
|
||||
```
|
||||
|
||||
Configuration: `terraform/state.tf`
|
||||
|
||||
## Accessing State
|
||||
|
||||
### Pull state from S3
|
||||
```bash
|
||||
cd terraform
|
||||
terraform state pull > terraform.tfstate.backup
|
||||
```
|
||||
|
||||
### View resources
|
||||
```bash
|
||||
terraform state list
|
||||
terraform state show <resource-name>
|
||||
```
|
||||
|
||||
### Import new resources
|
||||
```bash
|
||||
terraform import <resource-type>.<name> <resource-id>
|
||||
```
|
||||
|
||||
## Backup Strategy
|
||||
|
||||
### Automatic backups
|
||||
Run the backup script periodically (e.g., cron):
|
||||
```bash
|
||||
scripts/terraform-state-backup.sh
|
||||
```
|
||||
|
||||
Backups are saved to: `~/.terraform-backups/homelab/`
|
||||
|
||||
### Manual backup
|
||||
```bash
|
||||
cd terraform
|
||||
terraform state pull > /tmp/terraform-$(date +%s).tfstate
|
||||
cp /tmp/terraform-*.tfstate ~/.terraform-backups/homelab/
|
||||
```
|
||||
|
||||
## Disaster Recovery
|
||||
|
||||
If state is corrupted or lost:
|
||||
|
||||
1. **Stop all infrastructure changes:**
|
||||
```bash
|
||||
git revert <commit> # Rollback infrastructure changes
|
||||
```
|
||||
|
||||
2. **Restore from local backup:**
|
||||
```bash
|
||||
BACKUP_FILE=~/.terraform-backups/homelab/<timestamp>-terraform.tfstate
|
||||
cd terraform
|
||||
terraform state push $BACKUP_FILE
|
||||
```
|
||||
|
||||
3. **Verify state:**
|
||||
```bash
|
||||
terraform state list
|
||||
terraform plan
|
||||
```
|
||||
|
||||
## S3 Bucket Setup
|
||||
|
||||
If S3 bucket doesn't exist, create it:
|
||||
|
||||
```bash
|
||||
kubectl exec -n storage <minio-pod> -- mc mb minio/terraform-state --region us-east-1
|
||||
```
|
||||
|
||||
## State Lock (Optional)
|
||||
|
||||
For multi-person teams, enable state locking via DynamoDB (not yet configured).
|
||||
|
||||
## Best Practices
|
||||
|
||||
- ✓ Never commit `*.tfstate` or `*.tfstate.*` to git
|
||||
- ✓ Back up state before major `terraform apply` operations
|
||||
- ✓ Always run `terraform plan` before `terraform apply`
|
||||
- ✓ Review diff carefully for destructive changes
|
||||
- ✓ Keep state backend secure (MinIO has authentication)
|
||||
|
||||
## Monitoring
|
||||
|
||||
Check S3 backend status:
|
||||
```bash
|
||||
kubectl get pods -n storage -l app=minio
|
||||
# Or
|
||||
scripts/terraform-state-backup.sh
|
||||
```
|
||||
|
||||
## Related Files
|
||||
|
||||
- `terraform/state.tf` — Backend configuration
|
||||
- `scripts/terraform-state-backup.sh` — Automated backup script
|
||||
- `.gitignore` — Excludes local state files from git
|
||||
Reference in New Issue
Block a user