# GitOps Architecture: Pure ArgoCD IaC

## Directory Structure (Production-Grade)

```
homelab/
├── k8s/
│   ├── _base/                          # Shared kustomizations, base values
│   │   ├── kustomization.yaml
│   │   ├── secrets-template.yaml       # Secret templates (filled via sops)
│   │   └── namespaces.yaml             # All namespace definitions
│   │
│   ├── infrastructure/                 # Layer 0: Foundation (namespaces, storage, RBAC)
│   │   ├── kustomization.yaml
│   │   ├── namespaces.yaml
│   │   ├── storage-classes.yaml
│   │   ├── service-accounts.yaml
│   │   ├── cluster-roles.yaml
│   │   ├── cluster-rolebindings.yaml
│   │   └── network-policies.yaml
│   │
│   ├── bootstrap/                      # Layer 1: Bootstrap (cert-manager, cilium, ingress-nginx)
│   │   ├── kustomization.yaml
│   │   ├── cert-manager/
│   │   │   ├── kustomization.yaml
│   │   │   └── values.yaml
│   │   ├── cilium/
│   │   │   ├── kustomization.yaml
│   │   │   └── values.yaml
│   │   └── ingress-nginx/
│   │       ├── kustomization.yaml
│   │       └── values.yaml
│   │
│   ├── platform/                       # Layer 2: Platform (storage, observability, state)
│   │   ├── kustomization.yaml
│   │   ├── longhorn/
│   │   │   ├── kustomization.yaml
│   │   │   └── values.yaml
│   │   ├── minio/
│   │   │   ├── kustomization.yaml
│   │   │   ├── values.yaml
│   │   │   └── buckets/
│   │   │       └── terraform-state.yaml
│   │   ├── loki/
│   │   │   ├── kustomization.yaml
│   │   │   └── values.yaml
│   │   ├── prometheus/
│   │   │   ├── kustomization.yaml
│   │   │   ├── values.yaml
│   │   │   └── servicemonitors/
│   │   └── promtail/
│   │       ├── kustomization.yaml
│   │       └── values.yaml
│   │
│   ├── security/                       # Layer 3: Identity & Auth
│   │   ├── kustomization.yaml
│   │   ├── authentik/
│   │   │   ├── kustomization.yaml
│   │   │   ├── values.yaml
│   │   │   └── config/
│   │   │       ├── oauth-apps.yaml
│   │   │       ├── groups.yaml
│   │   │       └── users.yaml
│   │   ├── vault/
│   │   │   ├── kustomization.yaml
│   │   │   └── values.yaml
│   │   └── cert-issuer/
│   │       └── certificate-definitions.yaml
│   │
│   ├── applications/                   # Layer 4: Business Applications
│   │   ├── kustomization.yaml
│   │   ├── forgejo/
│   │   │   ├── kustomization.yaml
│   │   │   ├── values.yaml
│   │   │   ├── deployment.yaml
│   │   │   └── config/
│   │   ├── grafana/
│   │   │   ├── kustomization.yaml
│   │   │   ├── values.yaml
│   │   │   └── dashboards/
│   │   ├── portainer/
│   │   │   ├── kustomization.yaml
│   │   │   └── values.yaml
│   │   ├── temporal/
│   │   │   ├── kustomization.yaml
│   │   │   └── values.yaml
│   │   └── llm/
│   │       ├── kustomization.yaml
│   │       └── values.yaml
│   │
│   ├── data/                           # Layer 5: Data Services
│   │   ├── kustomization.yaml
│   │   ├── postgres/
│   │   │   ├── kustomization.yaml
│   │   │   ├── values.yaml
│   │   │   └── backups/
│   │   ├── redis/
│   │   │   ├── kustomization.yaml
│   │   │   └── values.yaml
│   │   └── kafka/
│   │       ├── kustomization.yaml
│   │       └── values.yaml
│   │
│   └── argocd/                         # ArgoCD Configuration (apps + projects)
│       ├── kustomization.yaml
│       ├── projects/
│       │   └── homelab-project.yaml
│       └── apps/
│           ├── kustomization.yaml
│           ├── root-app.yaml           # Root application (points to k8s/infrastructure/)
│           ├── layer-0-infrastructure.yaml
│           ├── layer-1-bootstrap.yaml
│           ├── layer-2-platform.yaml
│           ├── layer-3-security.yaml
│           ├── layer-4-applications.yaml
│           └── layer-5-data.yaml
│
├── .sops.yaml                          # SOPS encryption config (for secrets)
├── .env.example                        # Environment variables template
├── .github/workflows/                  # (or .forgejo/workflows/)
│   ├── validate-k8s.yaml              # Lint, kubeval, ArgoCD validation
│   └── security-scan.yaml             # OWASP, policy checks
└── GITOPS_ARCHITECTURE.md
```

## Application Sync Layers (Deployment Order)

```
Layer 0: Infrastructure (foundation - 1 app)
  └─ infrastructure/ (namespaces, storage classes, RBAC)

Layer 1: Bootstrap (cluster networking - 1 app)
  └─ bootstrap/ (cert-manager, cilium, ingress-nginx)

Layer 2: Platform (cluster services - 1 app)
  └─ platform/ (longhorn, minio, loki, prometheus, promtail)

Layer 3: Security (identity & auth - 1 app)
  └─ security/ (authentik, vault, cert-issuers)

Layer 4: Applications (business services - 1 app)
  └─ applications/ (forgejo, grafana, portainer, temporal, llm)

Layer 5: Data (stateful services - 1 app)
  └─ data/ (postgres, redis, kafka)
```

**Why layers?**
- Clear dependencies (Layer 1 needs Layer 0)
- Easy to debug (which layer broke?)
- Easy to rollback (one layer at a time)
- Easy to scale (add services without touching others)

## ArgoCD Application Architecture

### Root Application
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: homelab-root
  namespace: argocd
spec:
  project: homelab
  source:
    repoURL: https://forgejo.riotpiao.com/riotpiao.com/homelab.git
    targetRevision: main
    path: k8s/infrastructure  # Start with Layer 0
  destination:
    server: https://kubernetes.default.svc
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
```

### Layer Applications (created by Layer 0)
```yaml
# k8s/infrastructure/argocd-apps.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: layer-1-bootstrap
spec:
  project: homelab
  source:
    repoURL: https://forgejo.riotpiao.com/riotpiao.com/homelab.git
    targetRevision: main
    path: k8s/bootstrap
  destination:
    server: https://kubernetes.default.svc
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: layer-2-platform
spec:
  ...
  path: k8s/platform
---
# Repeat for layers 3, 4, 5
```

## Kustomization Strategy

### Base (Helm chart + values override)
```yaml
# k8s/bootstrap/cert-manager/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

helmCharts:
- name: cert-manager
  repo: https://charts.jetstack.io
  releaseName: cert-manager
  version: v1.21.0
  namespace: cert-manager
  valuesInline:
    crds:
      enabled: true
    prometheus:
      enabled: true
      servicemonitor:
        enabled: true
```

### Patch (customize per environment)
```yaml
# k8s/bootstrap/cert-manager/kustomization.yaml
patchesJson6902:
- target:
    group: helm.sh
    version: v1
    kind: Release
    name: cert-manager
  patch: |-
    - op: add
      path: /spec/values/installCRDs
      value: "true"
```

## Secrets Management (SOPS)

```bash
# Encrypt secrets before committing
sops -e secrets.yaml > secrets.enc.yaml
git add secrets.enc.yaml

# ArgoCD decrypts at sync time (via plugin)
```

## File Naming Conventions

```
Layer directories:
  k8s/{layer}/*.yaml
  
Service subdirectories:
  k8s/{layer}/{service}/
    ├── kustomization.yaml        # Helm chart + patches
    ├── values.yaml               # Helm values
    └── config/                   # Additional manifests
        ├── foo.yaml
        └── bar.yaml

Naming:
  ✓ cert-manager/values.yaml     (service-specific)
  ✓ cluster-roles.yaml            (resource type)
  ✓ storage-classes.yaml          (resource type)
  ✗ cert-manager-helm.yaml        (redundant suffix)
  ✗ my-custom-config.yaml         (non-standard)
```

## CI/CD Validation Pipeline

```yaml
# .forgejo/workflows/validate-k8s.yaml
on: [push, pull_request]

jobs:
  validate:
    runs-on: default
    steps:
      - uses: actions/checkout@v3
      
      # 1. Lint YAML
      - run: yamllint k8s/
      
      # 2. Validate K8s manifests
      - run: kubeval k8s/**/*.yaml
      
      # 3. Kustomize build (no apply)
      - run: |
          for dir in k8s/infrastructure k8s/bootstrap k8s/platform k8s/security k8s/applications k8s/data; do
            kustomize build $dir > /dev/null
          done
      
      # 4. ArgoCD app validation (dry-run)
      - run: argocd app create --dry-run -f k8s/argocd/apps/
      
      # 5. Policy check (optional)
      - run: conftest test -p policy/ k8s/**/*.yaml
```

## Deployment Checklist

**Day 1: Design Review**
- [ ] Review directory structure
- [ ] Confirm layer dependencies
- [ ] Identify secrets (need SOPS)
- [ ] Plan kustomization strategy

**Day 2-3: Build Infrastructure Layer**
- [ ] Create k8s/infrastructure/ with namespaces, storage classes, RBAC
- [ ] Create kustomization.yaml for Layer 0
- [ ] Test: `kustomize build k8s/infrastructure/`

**Day 4-5: Build Bootstrap Layer**
- [ ] Create k8s/bootstrap/{cert-manager,cilium,ingress-nginx}/
- [ ] Add Helm chart references
- [ ] Test: `kustomize build k8s/bootstrap/`

**Day 6-7: Build Platform Layer**
- [ ] Create k8s/platform/{minio,longhorn,loki,prometheus}/
- [ ] Add values overrides
- [ ] Test: `kustomize build k8s/platform/`

**Day 8-9: Build Security Layer**
- [ ] Create k8s/security/{authentik,vault}/
- [ ] Migrate Authentik resources from Terraform
- [ ] Encrypt secrets with SOPS

**Day 10-11: Build Application Layer**
- [ ] Move k8s/talos-ci-cd/ → k8s/applications/forgejo/
- [ ] Consolidate k8s/{logging,monitoring} → k8s/platform/ or k8s/applications/
- [ ] Create kustomization.yaml for services

**Day 12: Create ArgoCD Apps**
- [ ] Create k8s/argocd/apps/ with layer applications
- [ ] Update AppProject permissions
- [ ] Deploy root application (starts with Layer 0)

**Day 13-14: Monitor & Validate**
- [ ] Watch ArgoCD sync for each layer
- [ ] Verify no drift
- [ ] Test manual edits (ArgoCD corrects them)

## Rollback Strategy

```bash
# Rollback entire layer
git revert <commit-hash>
git push

# ArgoCD detects change, syncs back to previous state

# Rollback single service
git checkout <commit-hash> -- k8s/applications/forgejo/
git commit -m "revert: forgejo back to <version>"
git push
```

## Benefits of This Architecture

✓ **Scalability:** Easy to add new services (copy service directory)
✓ **Clarity:** Clear layer dependencies (no surprises)
✓ **Safety:** Manual edits auto-corrected by ArgoCD
✓ **Auditability:** Every change in git (who, when, why)
✓ **Testing:** Kustomize build validates before sync
✓ **Rollback:** Git history = disaster recovery
✓ **Secrets:** SOPS encryption built-in
✓ **CI/CD:** Automated validation on every PR

## Next Steps

1. Create directory structure (k8s/infrastructure/, bootstrap/, platform/, etc.)
2. Migrate existing manifests from scattered k8s/ to new structure
3. Create kustomization.yaml for each layer
4. Create ArgoCD Applications for each layer
5. Test on live cluster (non-disruptive)
6. Update CI/CD pipeline to validate new structure