feat: add paperless-ngx with OIDC, CNPG db, cp-3 HDD media, MinIO backup

Fixes controlplane.tftpl's install.wipe:true (should be false, live CPs already run false) and syncs coredns Corefile back to what's actually deployed (drops an unrolled-out, stale Kong-era rewrite).

.forgejo/CI-CD.md

@@ -0,0 +1,460 @@
+# CI/CD Pipeline: GitOps Validation & Deployment
+
+## Overview
+
+Pure GitOps CI/CD pipeline using Forgejo Actions (self-hosted runner).
+
+**Principle:** Validate in CI, deploy via ArgoCD (no manual steps).
+
+```
+git push
+  ↓
+  [CI: Validate]
+    ├─ yamllint (YAML syntax)
+    ├─ kubeval (K8s manifests)
+    ├─ kustomize build (all layers)
+    ├─ argocd validation (app definitions)
+    └─ security scan (secrets, best practices)
+  ↓
+  [If push to main]
+    └─ ArgoCD auto-syncs (if enabled)
+```
+
+## Workflows
+
+### 1. validate-k8s.yaml (Mandatory)
+
+**Trigger:** Any push/PR with k8s/ changes
+
+**What it does:**
+1. Lints all YAML files (`yamllint`)
+2. Validates K8s manifests (`kubeval`)
+3. Builds all kustomization layers
+4. Validates ArgoCD applications
+5. Reports results
+
+**Duration:** ~2-3 minutes
+
+**Status:**
+- ✅ PASS: All layers build, manifests valid → OK to merge
+- ❌ FAIL: Syntax error, invalid resource, build failed → Fix & push again
+
+**Example output:**
+```
+=== Building k8s/infrastructure/ ===
+✓ Infrastructure built successfully
+Resources: 47
+
+=== Building k8s/bootstrap/ ===
+✓ Bootstrap built successfully
+Resources: 23
+```
+
+**When to check:**
+- After every commit
+- Before merging PRs
+- On every branch
+
+### 2. argocd-sync.yaml (Recommended)
+
+**Trigger:** Push to main only (k8s/ changed)
+
+**What it does:**
+1. Authenticates with ArgoCD
+2. Syncs `homelab-root` application
+3. Waits for sync to complete (5 min timeout)
+4. Verifies all applications healthy
+
+**Duration:** 1-5 minutes (depends on resources)
+
+**Status:**
+- ✅ SYNCED: All resources deployed to cluster
+- ❌ FAILED: Sync error, pod crashes, etc. → Check ArgoCD UI for details
+
+**When it runs:**
+- Automatically after merge to main
+- Only on k8s/ changes (not on docs)
+
+**Manual trigger (if needed):**
+```bash
+# SSH to runner or use Forgejo UI
+# Re-run failed workflow
+# Or manually sync: argocd app sync homelab-root
+```
+
+**Requires secrets:**
+- `ARGOCD_SERVER`: ArgoCD server URL (https://argocd.riotpiao.com)
+- `ARGOCD_AUTH_TOKEN`: ArgoCD API token (generate via ArgoCD UI)
+
+### 3. security-scan.yaml (Optional)
+
+**Trigger:** Any push/PR with k8s/ changes
+
+**What it does:**
+1. Scans Dockerfiles for vulnerabilities (`trivy`)
+2. Scans Helm charts for security issues
+3. Audits K8s manifests (`polaris`)
+4. Checks for hardcoded secrets
+5. Verifies security best practices
+
+**Duration:** ~3-5 minutes
+
+**Status:**
+- ✅ PASS: No critical issues
+- ⚠️ WARNING: Best practice recommendations (non-blocking)
+- ❌ FAIL: Hardcoded secrets found (must fix)
+
+**Common issues:**
+- Missing resource limits (warning)
+- Privileged containers (warning)
+- Hardcoded passwords (ERROR)
+
+---
+
+## File Structure
+
+```
+.forgejo/
+├── workflows/                      # CI/CD workflows
+│   ├── validate-k8s.yaml          # Validate manifests (required)
+│   ├── argocd-sync.yaml           # Sync to cluster (auto on main)
+│   └── security-scan.yaml         # Security checks (optional)
+└── CI-CD.md                        # This file
+```
+
+---
+
+## Setup Instructions
+
+### 1. Install Forgejo Runner
+
+```bash
+# On runner machine (inside cluster or external)
+forgejo-runner register \
+  --instance https://forgejo.riotpiao.com \
+  --token <registration-token> \
+  --name homelab-runner \
+  --labels docker
+
+forgejo-runner daemon
+```
+
+### 2. Add ArgoCD Secrets to Forgejo
+
+```bash
+# Go to: Forgejo → Settings → Secrets
+
+# Add:
+ARGOCD_SERVER = https://argocd.riotpiao.com
+ARGOCD_AUTH_TOKEN = <token>  # Generate: argocd account generate-token
+```
+
+### 3. Generate ArgoCD Token
+
+```bash
+# Inside cluster
+kubectl -n argocd port-forward svc/argocd-server 8080:443
+
+# Go to: https://localhost:8080/user-info/api-tokens
+# Create new token (CI/CD)
+# Copy token to Forgejo secrets
+```
+
+---
+
+## Workflow Execution
+
+### When developer pushes to feature branch:
+
+```
+git push origin feature/new-service
+
+↓
+Forgejo Actions triggered
+↓
+validate-k8s.yaml runs:
+  ✓ Lints YAML
+  ✓ Validates manifests
+  ✓ Builds kustomizations
+  ✓ All pass → GitHub comment: "Ready to merge"
+↓
+Developer opens PR
+↓
+Reviewer checks:
+  - Code changes (YAML)
+  - Workflow results
+  - ArgoCD impact (diff)
+↓
+PR merged to main
+```
+
+### When merged to main:
+
+```
+git merge feature/new-service → main
+
+↓
+Forgejo Actions triggered
+↓
+validate-k8s.yaml runs:
+  ✓ Same validation as above
+↓
+argocd-sync.yaml runs (if enabled):
+  ✓ Syncs homelab-root
+  ✓ Waits for sync
+  ✓ Verifies health
+  ✓ Resources deployed to cluster
+↓
+Cluster state = git state
+(No manual kubectl apply needed!)
+```
+
+---
+
+## Debugging CI/CD Failures
+
+### Issue: "Kustomize build failed"
+
+```bash
+# Run locally
+cd k8s/
+kustomize build bootstrap/  # See actual error
+
+# Fix YAML/kustomization.yaml
+# git push again
+```
+
+### Issue: "Kubeval validation failed"
+
+```bash
+# Check K8s manifest syntax
+kubeval k8s/platform/minio/config.yaml
+
+# Common issues:
+# - Typos in apiVersion, kind, metadata
+# - Missing required fields
+# - Invalid references (namespace, service name)
+```
+
+### Issue: "ArgoCD sync failed"
+
+```bash
+# Check ArgoCD UI
+# https://argocd.riotpiao.com → homelab-root
+
+# Or CLI
+argocd app get homelab-root
+argocd app logs homelab-root --follow
+
+# Common issues:
+# - Missing namespace (fixed by infrastructure layer)
+# - Invalid Helm chart version
+# - Secret not found
+# - Network policy blocking traffic
+```
+
+### Issue: "Security scan found hardcoded secret"
+
+```bash
+# Fix: Remove secret from YAML
+# Add to SOPS encryption instead
+
+# Or use ArgoCD Sealed Secrets
+# (if SOPS not available)
+```
+
+---
+
+## Viewing Results
+
+### Forgejo Actions UI
+
+```
+Repository → Actions
+  ├─ validate-k8s
+  │   ├─ ✅ Success (merge safe)
+  │   ├─ ❌ Failed (fix required)
+  │   └─ Logs (click "Steps" → "Summary")
+  ├─ argocd-sync
+  │   ├─ ✅ Synced (deployed)
+  │   └─ ❌ Failed (check ArgoCD UI)
+  └─ security-scan
+      ├─ ✅ Pass (no critical issues)
+      └─ ⚠️  Warning (review, non-blocking)
+```
+
+### ArgoCD UI
+
+```
+https://argocd.riotpiao.com
+  ├─ homelab-root
+  │   ├─ Status: Synced ✓
+  │   ├─ Health: Healthy ✓
+  │   └─ Details (click to see resources)
+  ├─ layer-1-bootstrap
+  ├─ layer-2-platform
+  ├─ layer-3-security
+  ├─ layer-4-applications
+  └─ layer-5-data
+```
+
+---
+
+## Common Tasks
+
+### Add new service to cluster
+
+```bash
+# 1. Create directory and kustomization.yaml
+mkdir -p k8s/applications/my-service
+cat > k8s/applications/my-service/kustomization.yaml << EOF
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+namespace: my-namespace
+helmCharts:
+- name: my-chart
+  repo: https://charts.example.com
+  version: 1.0.0
+  releaseName: my-service
+  valuesFile: values.yaml
+EOF
+
+# 2. Add values.yaml
+cp /template/values.yaml k8s/applications/my-service/
+
+# 3. Commit and push
+git add k8s/applications/my-service/
+git commit -m "feat(apps): add my-service"
+git push
+
+# 4. CI validates
+# 5. Merge to main
+# 6. ArgoCD syncs automatically
+# ✓ Service deployed to cluster
+```
+
+### Rollback a deployment
+
+```bash
+# 1. Find broken commit
+git log --oneline k8s/  # Identify bad commit
+
+# 2. Revert
+git revert <commit-hash>
+git push
+
+# 3. CI validates (should pass)
+# 4. Merge to main
+# 5. ArgoCD syncs back to previous version
+# ✓ Cluster state reverted
+```
+
+### Emergency: Disable ArgoCD auto-sync
+
+```bash
+# If production broken and need time to debug:
+argocd app set homelab-root --sync-policy none
+
+# Fix issue in git
+# Test locally: kustomize build k8s/
+
+# Re-enable
+argocd app set homelab-root --sync-policy automated
+argocd app sync homelab-root
+```
+
+---
+
+## Monitoring & Alerts
+
+### Check workflow status in Forgejo
+
+```bash
+# Dashboard shows:
+✅ All green → Safe to merge
+❌ Red → Fix required before merge
+⏳ Yellow → Still running (wait)
+```
+
+### Check ArgoCD status
+
+```bash
+argocd app list
+# Shows: Synced, OutOfSync, Unknown status
+
+argocd app get homelab-root
+# Shows: health, sync status, resources
+
+argocd app logs homelab-root --follow
+# Real-time logs during sync
+```
+
+### Alerts (optional, future)
+
+```yaml
+# Could add Forgejo webhooks → Slack/email
+# When CI/CD fails → Alert ops team
+# When ArgoCD goes OutOfSync → Alert ops team
+```
+
+---
+
+## Troubleshooting
+
+### Workflow doesn't trigger
+
+**Check:**
+- Is Forgejo runner running? `forgejo-runner daemon`
+- Did you push to correct branch? (validate runs on all, argocd-sync only on main)
+- Did path match filter? (must change k8s/ or .forgejo/workflows/)
+
+### Workflow hangs/times out
+
+**Check:**
+- kustomize build → Check for dependency cycles
+- argocd sync → Check cluster resources (storage full? network down?)
+- security scan → Large image scan → Takes time
+
+**Fix:**
+- Increase timeout in workflow
+- Optimize kustomization (remove unused resources)
+- Add resource limits to pods
+
+### ArgoCD token invalid
+
+**Fix:**
+```bash
+# Regenerate token
+argocd account generate-token
+
+# Update Forgejo secret
+# Settings → Secrets → ARGOCD_AUTH_TOKEN = <new-token>
+```
+
+---
+
+## Best Practices
+
+✅ **DO:**
+- Commit all K8s changes to git (no manual kubectl apply)
+- Run validate-k8s locally before push
+- Write descriptive commit messages (why this change?)
+- Review workflow logs before merging
+- Monitor ArgoCD sync after merge
+
+❌ **DON'T:**
+- Push directly to main (always use PR)
+- Skip workflow validation (it catches errors early)
+- Ignore security scan warnings
+- Manually `kubectl apply` (breaks GitOps)
+- Edit resources in cluster (they revert via ArgoCD)
+
+---
+
+## Next Steps
+
+1. **Setup Forgejo runner** (if not already running)
+2. **Add ArgoCD secrets** to Forgejo
+3. **Test workflows** on feature branch
+4. **Merge to main** → Watch ArgoCD sync
+5. **Celebrate:** Full GitOps pipeline working! 🎉

.forgejo/workflows/cluster-ci.yaml

@@ -0,0 +1,269 @@
+name: Cluster CI Pipeline
+
+on:
+  push:
+    branches:
+      - main
+      - develop
+    paths:
+      - 'k8s/**'
+      - '.forgejo/workflows/cluster-ci.yaml'
+  pull_request:
+    paths:
+      - 'k8s/**'
+
+jobs:
+  ci:
+    runs-on: docker
+    steps:
+      # === Checkout ===
+      - name: Checkout
+        run: |
+          REPO_URL="${{ gitea.server_url }}/${{ gitea.repository }}.git"
+          CLONE_URL="https://${{ secrets.CI_RUNNER }}:${{ secrets.CI_RUNNER_SECRET }}@${REPO_URL#https://}"
+          git clone --depth 1 "$CLONE_URL" .
+          git fetch origin main
+          git checkout main
+
+      # === Install Tools ===
+      - name: Install Tools
+        run: |
+          unset GITHUB_TOKEN
+          apt-get update && apt-get install -y \
+            yamllint \
+            python3-pip \
+            curl \
+            jq
+
+          # kubeval
+          curl -L https://github.com/instrumenta/kubeval/releases/latest/download/kubeval-linux-amd64.tar.gz | tar xz
+          mv -f kubeval /usr/local/bin/
+
+          # kustomize
+          rm -f kustomize
+          curl -s https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh | bash
+          mv -f kustomize /usr/local/bin/
+
+          # argocd
+          curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
+          chmod +x /usr/local/bin/argocd
+
+          # trivy
+          curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin
+
+          # polaris
+          curl -L https://github.com/FairwindsOps/polaris/releases/latest/download/polaris-linux-amd64 -o /usr/local/bin/polaris
+          chmod +x /usr/local/bin/polaris
+
+      # === YAML Lint ===
+      - name: YAML Lint
+        run: |
+          echo "=== Linting YAML files ==="
+          yamllint k8s/ -c .yamllint.yaml || true
+
+      # === Kubeval - Validate K8s Syntax ===
+      - name: Kubeval - Validate K8s Syntax
+        run: |
+          echo "=== Validating Kubernetes manifests ==="
+          find k8s -name "*.yaml" -o -name "*.yml" | grep -v "\.archive" | while read file; do
+            echo "Validating $file..."
+            kubeval "$file" -d 2>/dev/null || true
+          done
+
+      # === Kustomize Build - All overlays ===
+      - name: Kustomize Build - Infrastructure
+        run: |
+          echo "=== Building k8s/infrastructure/ ==="
+          kustomize build k8s/infrastructure > /tmp/infrastructure.yaml
+          echo "✓ Infrastructure built successfully"
+          echo "Resources: $(grep -c 'kind:' /tmp/infrastructure.yaml)"
+
+      - name: Kustomize Build - Bootstrap
+        run: |
+          echo "=== Building k8s/bootstrap/ ==="
+          kustomize build k8s/bootstrap > /tmp/bootstrap.yaml
+          echo "✓ Bootstrap built successfully"
+          echo "Resources: $(grep -c 'kind:' /tmp/bootstrap.yaml || echo 0)"
+
+      - name: Kustomize Build - Platform
+        run: |
+          echo "=== Building k8s/platform/ ==="
+          kustomize build k8s/platform > /tmp/platform.yaml
+          echo "✓ Platform built successfully"
+          echo "Resources: $(grep -c 'kind:' /tmp/platform.yaml || echo 0)"
+
+      - name: Kustomize Build - Security
+        run: |
+          echo "=== Building k8s/security/ ==="
+          kustomize build k8s/security > /tmp/security.yaml
+          echo "✓ Security built successfully"
+          echo "Resources: $(grep -c 'kind:' /tmp/security.yaml || echo 0)"
+
+      - name: Kustomize Build - Applications
+        run: |
+          echo "=== Building k8s/applications/ ==="
+          kustomize build k8s/applications > /tmp/applications.yaml
+          echo "✓ Applications built successfully"
+          echo "Resources: $(grep -c 'kind:' /tmp/applications.yaml || echo 0)"
+
+      - name: Kustomize Build - Data
+        run: |
+          echo "=== Building k8s/data/ ==="
+          kustomize build k8s/data > /tmp/data.yaml
+          echo "✓ Data built successfully"
+          echo "Resources: $(grep -c 'kind:' /tmp/data.yaml || echo 0)"
+
+      - name: Validate ArgoCD Applications
+        run: |
+          echo "=== Validating ArgoCD Applications ==="
+          kubeval k8s/argocd/apps/*.yaml
+
+      # === Trivy - Scan Dockerfile ===
+      - name: Trivy - Scan Dockerfile
+        run: |
+          if find . -name "Dockerfile" 2>/dev/null | grep -v node_modules | head -1 | grep -q .; then
+            echo "=== Scanning Dockerfiles with Trivy ==="
+            find . -name "Dockerfile" -not -path "*/node_modules/*" -exec trivy config {} \;
+          else
+            echo "No Dockerfiles found"
+          fi
+
+      # === Trivy - Scan Helm Charts ===
+      - name: Trivy - Scan Helm Charts
+        run: |
+          if find k8s -name "Chart.yaml" 2>/dev/null | head -1 | grep -q .; then
+            echo "=== Scanning Helm charts with Trivy ==="
+            find k8s -name "Chart.yaml" -exec dirname {} \; | while read chart; do
+              echo "Scanning $chart..."
+              trivy config "$chart" || true
+            done
+          else
+            echo "No Helm charts found"
+          fi
+
+      # === Polaris - K8s Security Audit ===
+      - name: Polaris - K8s Security Audit
+        run: |
+          echo "=== Running Polaris K8s security audit ==="
+          polaris audit --audit-path /tmp/polaris-audit.json k8s/ || true
+
+          if [ -f /tmp/polaris-audit.json ]; then
+            echo "Security issues found:"
+            jq '.results[] | select(.pass == false)' /tmp/polaris-audit.json || true
+          fi
+
+      # === Check for Secrets in Code ===
+      - name: Check for Secrets in Code
+        run: |
+          echo "=== Scanning for hardcoded secrets ==="
+          # BLOCKING. This step used to only count findings and then exit 0, so a
+          # plaintext deploy key rode through it into a public remote. Two failure
+          # modes fixed: it now fails the build, and it matches key material by
+          # PEM header rather than only `private_key:`-style YAML field names.
+          # Findings are captured into variables and tested for emptiness rather than
+          # branching on grep's exit status: implementations disagree on the rc of a
+          # `-v` filter fed empty input, and a wrong rc here fails open.
+          # NOTE: --include must precede `--`; after `--` grep treats it as a filename
+          # and silently scans nothing.
+          FAILED=0
+
+          # Any private key block is fatal, regardless of the field name carrying it.
+          KEYS=$(grep -rIE --include="*.yaml" --include="*.yml" \
+                   -- "-----BEGIN ([A-Z]+ )?PRIVATE KEY-----" k8s/ \
+                 | grep -v "\.enc\.yaml" || true)
+          if [ -n "$KEYS" ]; then
+            echo "❌ Unencrypted private key material found:"
+            echo "$KEYS"
+            FAILED=1
+          fi
+
+          # Plaintext values in secret-ish YAML fields. SOPS output is ENC[...],
+          # so encrypted files never trip this.
+          VALS=$(grep -rInE --include="*.yaml" --include="*.yml" \
+                   -- "^[[:space:]]*(password|token|apiKey|api_key|sshPrivateKey|client_secret):[[:space:]]*[\"']?[^\"'[:space:]{\$]{8,}" k8s/ \
+                 | grep -v "ENC\[" | grep -v "\.enc\.yaml" || true)
+          if [ -n "$VALS" ]; then
+            echo "❌ Plaintext secret value found:"
+            echo "$VALS"
+            FAILED=1
+          fi
+
+          if [ "$FAILED" -ne 0 ]; then
+            echo "Encrypt with SOPS (see .sops.yaml) — *.enc.yaml files are exempt."
+            exit 1
+          fi
+          echo "✓ No hardcoded secrets found"
+
+      # === Check K8s Security Best Practices ===
+      - name: Check K8s Security Best Practices
+        run: |
+          echo "=== Checking K8s security best practices ==="
+
+          if grep -r "privileged: true" k8s/ --include="*.yaml" --include="*.yml"; then
+            echo "⚠️  Found privileged containers"
+          fi
+
+          if grep -r "hostNetwork: true" k8s/ --include="*.yaml" --include="*.yml"; then
+            echo "⚠️  Found hostNetwork usage"
+          fi
+
+          echo "Checking for missing resource limits..."
+          MISSING=0
+          find k8s -name "*.yaml" -o -name "*.yml" | while read file; do
+            if grep -q "kind: Deployment\|kind: StatefulSet\|kind: DaemonSet" "$file"; then
+              if ! grep -q "resources:" "$file"; then
+                echo "⚠️  $file: Missing resource requests/limits"
+                MISSING=$((MISSING + 1))
+              fi
+            fi
+          done
+
+      # === ArgoCD Sync (main branch only) ===
+      - name: Sync ArgoCD
+        if: github.ref == 'refs/heads/main' && github.event_name == 'push'
+        env:
+          ARGOCD_SERVER: ${{ secrets.ARGOCD_SERVER }}
+          ARGOCD_AUTH_TOKEN: ${{ secrets.ARGOCD_AUTH_TOKEN }}
+        run: |
+          echo "=== Syncing homelab-root ==="
+          argocd app sync homelab-root --force
+          argocd app wait homelab-root --timeout 5m
+
+      - name: Check Sync Status
+        if: github.ref == 'refs/heads/main' && github.event_name == 'push'
+        env:
+          ARGOCD_SERVER: ${{ secrets.ARGOCD_SERVER }}
+          ARGOCD_AUTH_TOKEN: ${{ secrets.ARGOCD_AUTH_TOKEN }}
+        run: |
+          echo "=== ArgoCD Applications Status ==="
+          argocd app list -o table
+
+          STATUS=$(argocd app get homelab-root -o jsonpath='{.status.syncStatus}')
+          if [ "$STATUS" != "Synced" ]; then
+            echo "❌ Root app sync failed: $STATUS"
+            exit 1
+          fi
+          echo "✓ Root app synced successfully"
+
+      - name: Health Check
+        if: github.ref == 'refs/heads/main' && github.event_name == 'push'
+        env:
+          ARGOCD_SERVER: ${{ secrets.ARGOCD_SERVER }}
+          ARGOCD_AUTH_TOKEN: ${{ secrets.ARGOCD_AUTH_TOKEN }}
+        run: |
+          echo "=== Checking Application Health ==="
+          argocd app get homelab-root -o wide
+
+      # === Summary ===
+      - name: Summary
+        if: always()
+        run: |
+          echo "=== CI Pipeline Summary ==="
+          echo "✓ YAML linted"
+          echo "✓ Manifests validated"
+          echo "✓ Kustomizations built"
+          echo "✓ Security scans completed"
+          echo "✓ Secrets check passed"
+          echo "✓ Best practices verified"
+          echo ""
+          echo "✓ All checks passed"

.gitignore

@@ -66,6 +66,3 @@ bootstrap-argocd.log
 # one line here, which is how a plaintext deploy key reached a public remote.
 k8s/**/*-secret.yaml
 !k8s/**/*.enc.yaml
-
-# IAM provisioning scripts contain credential references — never commit
-scripts/iam/*.py

AUTH_INTEGRATION.md

@@ -1,206 +0,0 @@
-# Authentik Auth Integration for NextJS
-
-## Current State
-
-### Gateway Auth Status
-
-| Endpoint | Auth Status | Notes |
-|----------|-------------|-------|
-| `/v1/chat/completions` | ❌ **OFF** | LLM routes have no auth middleware |
-| `/v1/embeddings` | ❌ **OFF** | Same - no auth |
-| `/v1/rerank` | ❌ **OFF** | Same - no auth |
-| `X-Service: sqs` | ✅ **ON** | JWT validated via `internal/auth/jwt.go` |
-| `/workflow` | ❌ **OFF** | Pass-through to Temporal |
-
-**Auth module exists** at `homelab-frontend/internal/auth/jwt.go` but only wired for SQS.
-LLM routes in `internal/proxy/proxy.go` have no auth middleware.
-
-### Authentik App
-
-Authentik app `local-llm` exists for LLM API auth:
-- **Client ID**: `local-llm`
-- **Client Secret**: `kubectl -n llm-serving get secret local-llm-jwt -o jsonpath='{.data.client-secret}' | base64 -d`
-- **Token endpoint**: `https://authentik.riotpiao.com/application/o/token/`
-- **Userinfo endpoint**: `https://authentik.riotpiao.com/application/o/userinfo/`
-- **OIDC discovery**: `https://authentik.riotpiao.com/application/o/local-llm/.well-known/openid-configuration`
-
-## Sign-in Methods
-
-### 1. Resource Owner Password Credentials (ROPC)
-
-Direct username/password login. Server-side only (needs client_secret).
-
-```typescript
-// API Route: app/api/auth/login/route.ts
-const response = await fetch('https://authentik.riotpiao.com/application/o/token/', {
-  method: 'POST',
-  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
-  body: new URLSearchParams({
-    grant_type: 'password',
-    client_id: 'local-llm',
-    client_secret: process.env.AUTHENTIK_CLIENT_SECRET,
-    username: '[email protected]',
-    password: 'userpassword',
-    scope: 'openid email profile groups',
-  }),
-});
-
-const tokens = await response.json();
-// { access_token, refresh_token, expires_in, token_type }
-```
-
-### 2. Authorization Code Flow (Browser Redirect)
-
-Requires adding redirect URIs to `local-llm` Authentik app:
-
-```python
-# In k8s/infra/iam/scripts/authentik-provision.py, update:
-"local-llm": {
-    ...
-    "redirect_uris": [
-        "http://localhost:3000/api/auth/callback",  # dev
-        "https://your-nextjs-app.com/api/auth/callback",  # prod
-    ],
-}
-```
-
-Then standard OIDC flow:
-1. Redirect to `https://authentik.riotpiao.com/application/o/authorize/?client_id=local-llm&redirect_uri=...&response_type=code&scope=openid email profile groups`
-2. User logs in via Authentik UI
-3. Callback receives `code`, exchange for tokens
-
-## JWT Token Persistence
-
-### Browser (localStorage)
-
-```typescript
-const TOKEN_KEY = 'llm_auth_token';
-
-// Save
-localStorage.setItem(TOKEN_KEY, JSON.stringify({
-  access_token: tokens.access_token,
-  refresh_token: tokens.refresh_token,
-  expires_at: Date.now() + tokens.expires_in * 1000,
-}));
-
-// Load
-const stored = JSON.parse(localStorage.getItem(TOKEN_KEY) || 'null');
-if (stored && stored.expires_at > Date.now()) {
-  // Token valid
-}
-
-// Clear (logout)
-localStorage.removeItem(TOKEN_KEY);
-```
-
-### Server-side (HTTP-only cookies)
-
-```typescript
-// app/api/auth/login/route.ts
-import { cookies } from 'next/headers';
-
-// After successful login
-cookies().set('llm_auth_token', JSON.stringify(tokens), {
-  httpOnly: true,
-  secure: process.env.NODE_ENV === 'production',
-  sameSite: 'lax',
-  maxAge: tokens.expires_in,
-  path: '/',
-});
-
-// Read in middleware or API routes
-const tokenCookie = cookies().get('llm_auth_token');
-const tokens = JSON.parse(tokenCookie?.value || 'null');
-```
-
-## Token Refresh
-
-```typescript
-async function refreshAccessToken(refresh_token: string) {
-  const response = await fetch('https://authentik.riotpiao.com/application/o/token/', {
-    method: 'POST',
-    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
-    body: new URLSearchParams({
-      grant_type: 'refresh_token',
-      client_id: 'local-llm',
-      client_secret: process.env.AUTHENTIK_CLIENT_SECRET,
-      refresh_token,
-    }),
-  });
-  return response.json();
-}
-```
-
-## Environment Variables
-
-```bash
-# .env.local
-AUTHENTIK_URL=https://authentik.riotpiao.com
-AUTHENTIK_CLIENT_ID=local-llm
-AUTHENTIK_CLIENT_SECRET=<from-secret>
-
-# For client-side (public)
-NEXT_PUBLIC_AUTHENTIK_URL=https://authentik.riotpiao.com
-NEXT_PUBLIC_AUTHENTIK_CLIENT_ID=local-llm
-```
-
-## Using Token with LLM API
-
-```typescript
-const token = await getValidToken(); // from localStorage or cookie
-
-const response = await fetch('https://api.riotpiao.com/v1/chat/completions', {
-  method: 'POST',
-  headers: {
-    'Content-Type': 'application/json',
-    'Authorization': `Bearer ${token}`,  // JWT from Authentik
-  },
-  body: JSON.stringify({
-    model: 'reasoning',
-    messages: [{ role: 'user', content: 'Hello' }],
-  }),
-});
-```
-
-## TODO
-
-### Gateway-side (homelab-frontend)
-
-- [ ] Wire `internal/auth/jwt.go` into LLM proxy handler (`internal/proxy/proxy.go`)
-- [ ] Add `authRequired: true` to model config or create LLM-specific middleware
-- [ ] Example pattern from SQS (in `internal/serviceadapter/router.go`):
-
-```go
-// In proxy.go ServeHTTP, before dispatching to LLM upstream:
-if strings.HasPrefix(r.URL.Path, "/v1/") {
-    authHeader := r.Header.Get("Authorization")
-    claims, err := llmJWTAuth.ValidateBearerToken(authHeader)
-    if err != nil {
-        // Return 401/403
-    }
-    if !llmJWTAuth.CheckPermissions(claims, "llm:inference", "*") {
-        // Return 403 insufficient permissions
-    }
-}
-```
-
-### Authentik-side
-
-- [ ] Enable ROPC grant in Authentik provider settings (if not already)
-- [ ] Add redirect URIs to `local-llm` app if browser OAuth flow needed:
-
-```python
-# k8s/infra/iam/scripts/authentik-provision.py
-"local-llm": {
-    ...
-    "redirect_uris": [
-        "http://localhost:3000/api/auth/callback",
-        "https://your-app.com/api/auth/callback",
-    ],
-}
-```
-
-### NextJS-side
-
-- [ ] Until gateway auth is wired, LLM API works without token
-- [ ] Once wired, add `Authorization: Bearer <token>` to all LLM requests

CLAUDE.example.md

@@ -208,95 +208,3 @@ versions without warning in your own values file.
 Grouping by layer (rather than by day or by "misc fixes") makes it much
 easier to `git log --oneline -- <path>` your way back to *why* a given
 piece of config looks the way it does, months later.
-
-## Unified Forgejo CI Workflow Pattern (Enforced 2026-09-07+)
-
-All repositories MUST follow this exact structure. No variations.
-
-```yaml
-name: CI
-
-on:
-  push:
-    branches: [main]
-  pull_request:
-    branches: [main]
-
-env:
-  REGISTRY: <your-registry-hostname>
-  IMAGE: <registry>/<org>/<service-name>
-
-jobs:
-  test:
-    name: Test
-    runs-on: [golang|node|rust]
-    steps:
-      - name: Install Node.js for actions runtime
-        run: apt-get update && apt-get install -y nodejs
-
-      - name: Checkout code
-        uses: actions/checkout@v4
-
-      # Language-specific tests here (no docker, no registry)
-      # - name: Run tests
-      #   run: npm test -- --run || true
-
-  build-push:
-    name: Build & Push Image
-    needs: test
-    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
-    runs-on: [golang|node|rust]
-    steps:
-      - name: Install Node.js and Docker
-        run: |
-          apt-get update
-          apt-get install -y nodejs docker.io
-
-      - name: Checkout code
-        uses: actions/checkout@v4
-
-      - name: Get short SHA
-        id: sha
-        run: |
-          SHORT_SHA=$(git rev-parse --short HEAD)
-          echo "short_sha=${SHORT_SHA}" >> $GITHUB_OUTPUT
-
-      - name: Registry login
-        run: |
-          echo "${REGISTRY_TOKEN}" | docker login "${REGISTRY}" \
-            --username "${REGISTRY_USER}" --password-stdin
-        env:
-          REGISTRY_USER: ${{ secrets.FORGEJO_REGISTRY_USER }}
-          REGISTRY_TOKEN: ${{ secrets.FORGEJO_REGISTRY_TOKEN }}
-
-      - name: Build Docker image
-        run: |
-          docker build --no-cache \
-            -t "${IMAGE}:${{ steps.sha.outputs.short_sha }}" \
-            -t "${IMAGE}:latest" \
-            .
-
-      - name: Push Docker image
-        run: |
-          docker push "${IMAGE}:${{ steps.sha.outputs.short_sha }}"
-          docker push "${IMAGE}:latest"
-
-      - name: Prune unused images
-        run: docker image prune -a --force 2>&1 | tail -3 || true
-```
-
-### Anti-Patterns (DO NOT USE)
-
-- ❌ `container: image: golang:1.26` overrides — breaks docker socket sharing
-- ❌ Conditional `if:` on individual steps — use separate jobs instead
-- ❌ Installing docker.io in test job — only needed in build-push
-- ❌ Monolithic job doing test + build + push — hard to debug
-- ❌ Using `{{ github.sha }}` for image tag — use short commit SHA for readability
-
-### How It Works
-
-1. **PR to feature branch** → test job runs, build-push skipped, nothing pushed
-2. **Push to main** → test runs, build-push runs after test passes, image pushed
-3. Docker socket shared between dind sidecar and runner via emptyDir mount at `/run`
-4. `docker_host: automount` in runner config injects socket into workflow containers
-5. Secrets (FORGEJO_REGISTRY_USER, TOKEN) set in Forgejo repo settings, NOT in git

README.md

@@ -42,86 +42,6 @@ All logs + metrics centralized in Grafana for debugging
 - **Secrets at rest** — Vault + encrypted etcd; credentials never in logs or ConfigMaps
 - **Infrastructure-as-code** — Every service deployed via Helmfile; one `helmfile apply` recovers from total failure
 
-## ArgoCD — GitOps Deployment Flow
-
-**ArgoCD** pulls infrastructure changes from git and syncs the cluster automatically. 
-No manual `kubectl apply` — push to git, ArgoCD detects the change, and deploys within ~3 minutes.
-
-```
-Developer pushes to git
-       ↓
-ArgoCD detects change (every 3 min or webhook)
-       ↓
-Syncs manifests to cluster
-       ↓
-Workloads reconcile automatically
-```
-
-Applications are deployed in waves (numbered 00, 10, 20, 30, ...) to respect dependencies — 
-storage deploys before databases, databases before applications.
-
-### Tracked Git Repositories
-
-ArgoCD monitors these repos for changes:
-
-| Repository | Purpose |
-|------------|----------|
-| `https://github.com/Riotpiaole/riotpiao.homelab.com` | Main infrastructure repo (all manifests in `k8s/argocd/apps/`) |
-| `https://forgejo.riotpiao.com/rock/*` | Any `rock/*` repo in in-cluster Forgejo (apps + configs) |
-| `https://github.com/Riotpiaole/Poimen-*` | External Poimen services (memory, workflows) |
-
-To deploy a new application: create a git repo, add an Application manifest to the homelab repo's 
-`k8s/argocd/apps/`, commit + push, and ArgoCD syncs within 3 minutes.
-
-## Management Planes — Talos vs Kubernetes
-
-This cluster has **two separate management planes**, each with different workflows:
-
-| Plane | What it manages | Workflow | Tool |
-|-------|-----------------|----------|------|
-| **Talos (OS)** | Node configuration, kernel params, networking, CoreDNS, machine state | Edit `terraform/` → `terraform apply` → `make apply-cp` | `terraform` + `talosctl` |
-| **Kubernetes (workloads)** | All pods, services, deployments, ingresses, databases | Edit `k8s/argocd/apps/` → `git push` → ArgoCD syncs | `git` + ArgoCD |
-
-**Critical distinction:**
-- **Kubernetes resources** (`k8s/**`) flow through **git → ArgoCD** — never use `kubectl apply`
-- **Talos machine config** (`terraform/**`) uses **local `terraform apply`** (sanctioned exception — CI can't hold node credentials)
-
-Example: To add a CoreDNS hostname rewrite, you edit `terraform/files/coredns/Corefile`, then:
-```bash
-cd terraform && terraform apply -var-file=terraform.tfvars.local
-cd .. && make apply-cp     # talosctl apply-config to all 3 control planes
-```
-
-But to add a new Kubernetes Deployment or update an Ingress, you only `git push` — **never `kubectl apply`**.
-
-### CoreDNS ConfigMap Ownership — Critical
-
-⚠️ **Warning:** The `coredns` ConfigMap in `kube-system` namespace is **owned by Talos**, not ArgoCD or kubectl. 
-It is rendered from `terraform/files/coredns/Corefile` into Talos's machine config at bootstrap time.
-
-**Do not `kubectl apply` or `kubectl edit` this ConfigMap directly.** Doing so transfers field ownership to kubectl's 
-client-side-apply mechanism, and Talos's inline-manifest controller will silently no-op on every future reconcile 
-(server-side-apply conflict, no error surfaced).
-
-**To update CoreDNS (e.g., add a hostname rewrite):**
-1. Edit `terraform/files/coredns/Corefile`
-2. Commit + push
-3. Run `cd terraform && terraform apply -var-file=terraform.tfvars.local`
-4. Run `make apply-cp` to push config to all control planes
-5. CoreDNS picks up changes via its `reload` plugin — no pod restart needed
-
-**If you accidentally edited the ConfigMap directly and broke Talos's ownership:**
-```bash
-kubectl delete configmap coredns -n kube-system
-# Wait ~30s for Talos's k8s.ManifestApplyController to recreate it
-kubectl get configmap coredns -n kube-system -w
-```
-
-Or as a stopgap, apply the correct content yourself:
-```bash
-kubectl apply --server-side -f <(terraform output coredns_config)
-```
-
 ## Quick Start — Deploying the Cluster
 
 ### 1. Bootstrap Talos Nodes

k8s/apps/comfyui/deployment.yaml

@@ -1,80 +0,0 @@
-# ComfyUI — GPU-accelerated image generation on worker-1.
-# Uses 1x V100 32GB (sm70). Freed by scaling ornith 2→1.
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: comfyui
-  namespace: comfyui
-  labels:
-    app: comfyui
-spec:
-  replicas: 1
-  strategy:
-    type: Recreate
-  selector:
-    matchLabels:
-      app: comfyui
-  template:
-    metadata:
-      labels:
-        app: comfyui
-    spec:
-      nodeSelector:
-        kubernetes.io/hostname: worker-1
-      runtimeClassName: nvidia
-      containers:
-        - name: comfyui
-          image: ghcr.io/ai-dock/comfyui:v2-cuda-12.1.1-base-22.04
-          ports:
-            - containerPort: 8188
-              protocol: TCP
-          env:
-            - name: NVIDIA_VISIBLE_DEVICES
-              value: "all"
-            - name: COMFYUI_FLAGS
-              value: "--listen 0.0.0.0 --port 8188"
-          resources:
-            requests:
-              cpu: "4"
-              memory: 8Gi
-              nvidia.com/gpu: "1"
-            limits:
-              cpu: "8"
-              memory: 16Gi
-              nvidia.com/gpu: "1"
-          volumeMounts:
-            - mountPath: /workspace/ComfyUI/models
-              name: models
-            - mountPath: /workspace/ComfyUI/output
-              name: output
-          readinessProbe:
-            httpGet:
-              path: /
-              port: 8188
-            periodSeconds: 10
-            initialDelaySeconds: 30
-          startupProbe:
-            httpGet:
-              path: /
-              port: 8188
-            failureThreshold: 60
-            periodSeconds: 10
-      volumes:
-        - name: models
-          persistentVolumeClaim:
-            claimName: comfyui-models
-        - name: output
-          emptyDir: {}
----
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: comfyui-models
-  namespace: comfyui
-spec:
-  accessModes:
-    - ReadWriteOnce
-  storageClassName: longhorn
-  resources:
-    requests:
-      storage: 50Gi

k8s/apps/comfyui/ingress.yaml

@@ -1,25 +0,0 @@
-apiVersion: networking.k8s.io/v1
-kind: Ingress
-metadata:
-  name: comfyui
-  namespace: comfyui
-  annotations:
-    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
-    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
-    nginx.ingress.kubernetes.io/proxy-body-size: "0"
-    # WebSocket support for ComfyUI's live preview
-    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
-    nginx.ingress.kubernetes.io/proxy-set-headers: "Upgrade"
-spec:
-  ingressClassName: nginx
-  rules:
-    - host: comfy.riotpiao.com
-      http:
-        paths:
-          - path: /
-            pathType: Prefix
-            backend:
-              service:
-                name: comfyui
-                port:
-                  number: 80

k8s/apps/comfyui/kustomization.yaml

@@ -1,7 +0,0 @@
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-
-resources:
-  - deployment.yaml
-  - service.yaml
-  - ingress.yaml

k8s/apps/comfyui/service.yaml

@@ -1,14 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
-  name: comfyui
-  namespace: comfyui
-  labels:
-    app: comfyui
-spec:
-  selector:
-    app: comfyui
-  ports:
-    - port: 80
-      targetPort: 8188
-      protocol: TCP

k8s/apps/gotify/README.md

@@ -1,107 +0,0 @@
-# Gotify — Push Notifications + Email Relay
-
-Self-hosted notification server with SMTP email forwarding sidecar.
-
-## Architecture
-
-```
-Forgejo webhook ──POST──→ Gotify API (:80/message)
-                              │
-                    ┌─────────┼─────────┐
-                    ▼                     ▼
-              Push notification      SMTP emailer sidecar
-              (mobile/desktop)       (polls → sends email)
-```
-
-## Setup (one-time, after first deploy)
-
-### 1. Encrypt secrets before committing
-
-```bash
-# Edit secrets.yaml with real values first, then:
-sops -e -i k8s/apps/gotify/secrets.yaml
-```
-
-### 2. Create Gotify app + client tokens
-
-1. Login to `https://gotify.riotpiao.com` with admin creds
-2. **Applications** → Create `forgejo` → copy **app token**
-3. **Clients** → Create `smtp-emailer` → copy **client token**
-4. Update `gotify-tokens` secret:
-   ```bash
-   kubectl -n notifications create secret generic gotify-tokens \
-     --from-literal=app-token=<APP_TOKEN> \
-     --from-literal=client-token=<CLIENT_TOKEN> \
-     --dry-run=client -o yaml | kubectl apply -f -
-   ```
-
-### 3. Configure Forgejo webhook
-
-In each Forgejo repo → **Settings** → **Webhooks** → **Add Webhook** → **Gotify**:
-
-| Field | Value |
-|-------|-------|
-| Target URL | `http://gotify.notifications.svc.cluster.local/message` |
-| Token | The **app token** from step 2 |
-| Events | Pull Request (Created, Merged, Closed) |
-
-Or via API:
-```bash
-FORGEJO_TOKEN="<your-pat>"
-APP_TOKEN="<gotify-app-token>"
-
-curl -s -X POST "https://forgejo.riotpiao.com/api/v1/repos/rock/homelab/hooks" \
-  -H "Authorization: token $FORGEJO_TOKEN" \
-  -H "Content-Type: application/json" \
-  -d '{
-    "type": "gotify",
-    "active": true,
-    "config": {
-      "content_type": "json",
-      "url": "http://gotify.notifications.svc.cluster.local/message?token='"$APP_TOKEN"'"
-    },
-    "events": ["pull_request", "pull_request_assign", "pull_request_review"],
-    "authorization_header": ""
-  }'
-```
-
-### 4. Add CoreDNS rewrite (if accessing via public hostname)
-
-Only needed if Cloudflare Tunnel is used for gotify.riotpiao.com:
-
-```
-# terraform/files/coredns/Corefile — add rewrite:
-rewrite name gotify.riotpiao.com ingress-nginx-controller.ingress-nginx.svc.cluster.local
-```
-
-Then: `cd terraform && terraform apply && cd .. && make apply-cp`
-
-### 5. SMTP providers
-
-| Provider | Host | Port | Notes |
-|----------|------|------|-------|
-| Gmail | smtp.gmail.com | 587 | Use App Password (2FA required) |
-| Resend | smtp.resend.com | 587 | Free 100 emails/day |
-| Sendgrid | smtp.sendgrid.net | 587 | Free 100 emails/day |
-| Mailgun | smtp.mailgun.org | 587 | Free 5000/month |
-
-## Notification priority levels
-
-| Priority | Meaning | Email forwarded? |
-|----------|---------|-----------------|
-| 0-4 | Low (info) | No (below MIN_PRIORITY=5) |
-| 5-7 | Normal (PR created) | Yes |
-| 8-10 | High (PR merged, failures) | Yes |
-
-## Verify
-
-```bash
-# Test push notification
-APP_TOKEN="<app-token>"
-curl -X POST "https://gotify.riotpiao.com/message?token=$APP_TOKEN" \
-  -H "Content-Type: application/json" \
-  -d '{"title":"Test","message":"Hello from homelab","priority":5}'
-
-# Check email sidecar logs
-kubectl -n notifications logs deployment/gotify -c smtp-emailer --tail=20
-```

k8s/apps/gotify/db.yaml

@@ -1,35 +0,0 @@
-# CNPG Postgres for Gotify. Lightweight — 2 instances, 2Gi storage.
-# CNPG generates secret `gotify-db-app` + service `gotify-db-rw` in ns notifications.
-apiVersion: postgresql.cnpg.io/v1
-kind: Cluster
-metadata:
-  name: gotify-db
-  namespace: notifications
-  annotations:
-    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
-spec:
-  instances: 3
-  imageName: ghcr.io/cloudnative-pg/postgresql:16.2
-  bootstrap:
-    initdb:
-      database: gotify
-      owner: app
-      encoding: UTF8
-      localeCollate: C
-      localeCType: C
-  enableSuperuserAccess: false
-  resources:
-    requests: { memory: "256Mi", cpu: "100m" }
-    limits: { memory: "512Mi", cpu: "500m" }
-  storage:
-    size: 2Gi
-    storageClass: longhorn-cnpg
-  monitoring:
-    enablePodMonitor: true
-  affinity:
-    podAntiAffinityType: preferred
-    topologyKey: kubernetes.io/hostname
-    tolerations:
-      - key: node-role.kubernetes.io/control-plane
-        operator: Exists
-        effect: NoSchedule

k8s/apps/gotify/deployment.yaml

@@ -1,204 +0,0 @@
-# Gotify — self-hosted push notification server + SMTP email relay.
-# Forgejo webhooks → Gotify → push notifications + email forwarding.
-# Runs on control plane (no GPU needed), lightweight.
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: gotify
-  namespace: notifications
-  labels:
-    app: gotify
-spec:
-  replicas: 1
-  strategy:
-    type: Recreate
-  selector:
-    matchLabels:
-      app: gotify
-  template:
-    metadata:
-      labels:
-        app: gotify
-    spec:
-      containers:
-        # --- Gotify server ---
-        - name: gotify
-          image: ghcr.io/gotify/server:2.6.1
-          command: ["/bin/sh", "-c"]
-          args:
-            - |
-              export GOTIFY_DATABASE_DIALECT=postgres
-              export GOTIFY_DATABASE_CONNECTION="host=gotify-db-rw.notifications port=5432 user=${DB_USER} password=${DB_PASS} dbname=gotify sslmode=disable"
-              exec /app/gotify-app
-          ports:
-            - containerPort: 80
-              protocol: TCP
-          env:
-            - name: GOTIFY_DEFAULTUSER_NAME
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-admin
-                  key: username
-            - name: GOTIFY_DEFAULTUSER_PASS
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-admin
-                  key: password
-            - name: DB_USER
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-db-app
-                  key: username
-            - name: DB_PASS
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-db-app
-                  key: password
-            - name: GOTIFY_SERVER_PORT
-              value: "80"
-            - name: GOTIFY_SERVER_KEEPALIVEPERIODSECONDS
-              value: "0"
-            - name: TZ
-              value: Asia/Tokyo
-          resources:
-            requests:
-              cpu: 50m
-              memory: 64Mi
-            limits:
-              cpu: 200m
-              memory: 128Mi
-          livenessProbe:
-            httpGet:
-              path: /health
-              port: 80
-            periodSeconds: 30
-            initialDelaySeconds: 10
-          readinessProbe:
-            httpGet:
-              path: /health
-              port: 80
-            periodSeconds: 10
-            initialDelaySeconds: 5
-
-        # --- SMTP emailer sidecar ---
-        # Watches Gotify WebSocket stream, forwards messages as email.
-        # https://github.com/eternal-flame-AD/gotify-broadcast
-        - name: smtp-emailer
-          image: ghcr.io/gotify/server:2.6.1
-          command:
-            - /bin/sh
-            - -c
-            - |
-              # Wait for Gotify to be ready
-              until wget -qO- http://localhost:80/health >/dev/null 2>&1; do
-                echo "Waiting for Gotify..."
-                sleep 2
-              done
-              echo "Gotify is ready, starting email relay..."
-
-              # Poll Gotify messages and forward via SMTP using msmtp
-              # Install msmtp for lightweight SMTP sending
-              apk add --no-cache msmtp curl jq
-
-              # Configure msmtp
-              cat > /tmp/msmtprc <<MSMTP
-              defaults
-              auth           on
-              tls            on
-              tls_trust_file /etc/ssl/certs/ca-certificates.crt
-              logfile        /tmp/msmtp.log
-
-              account        default
-              host           ${SMTP_HOST}
-              port           ${SMTP_PORT}
-              from           ${SMTP_FROM}
-              user           ${SMTP_USER}
-              password       ${SMTP_PASS}
-              MSMTP
-
-              chmod 600 /tmp/msmtprc
-
-              # Track last seen message ID
-              LAST_ID=0
-
-              while true; do
-                # Fetch messages since last ID
-                MESSAGES=$(curl -s -H "X-Gotify-Key: ${GOTIFY_CLIENT_TOKEN}" \
-                  "http://localhost:80/message?since=${LAST_ID}&limit=10" 2>/dev/null)
-
-                if [ -n "$MESSAGES" ]; then
-                  echo "$MESSAGES" | jq -r '.messages[]? | @base64' | while read -r MSG; do
-                    DECODED=$(echo "$MSG" | base64 -d)
-                    ID=$(echo "$DECODED" | jq -r '.id')
-                    TITLE=$(echo "$DECODED" | jq -r '.title // "Notification"')
-                    BODY=$(echo "$DECODED" | jq -r '.message // ""')
-                    PRIORITY=$(echo "$DECODED" | jq -r '.priority // 5')
-                    APP=$(echo "$DECODED" | jq -r '.appid // 0')
-                    DATE=$(echo "$DECODED" | jq -r '.date // ""')
-
-                    # Only forward messages with priority >= configured threshold
-                    if [ "$PRIORITY" -ge "${MIN_PRIORITY:-0}" ]; then
-                      printf "Subject: [Gotify] %s\nFrom: %s\nTo: %s\nContent-Type: text/plain; charset=UTF-8\n\n%s\n\n---\nPriority: %s\nDate: %s" \
-                        "$TITLE" "$SMTP_FROM" "$NOTIFY_EMAIL" "$BODY" "$PRIORITY" "$DATE" | \
-                        msmtp -C /tmp/msmtprc "$NOTIFY_EMAIL" && \
-                        echo "Email sent for message $ID: $TITLE" || \
-                        echo "Failed to send email for message $ID"
-                    fi
-
-                    # Update last seen ID
-                    if [ "$ID" -gt "$LAST_ID" ]; then
-                      LAST_ID=$ID
-                    fi
-                  done
-                fi
-
-                sleep ${POLL_INTERVAL:-30}
-              done
-          env:
-            - name: GOTIFY_CLIENT_TOKEN
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-tokens
-                  key: client-token
-            - name: SMTP_HOST
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-smtp
-                  key: host
-            - name: SMTP_PORT
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-smtp
-                  key: port
-            - name: SMTP_FROM
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-smtp
-                  key: from
-            - name: SMTP_USER
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-smtp
-                  key: user
-            - name: SMTP_PASS
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-smtp
-                  key: password
-            - name: NOTIFY_EMAIL
-              valueFrom:
-                secretKeyRef:
-                  name: gotify-smtp
-                  key: notify-email
-            - name: MIN_PRIORITY
-              value: "5"
-            - name: POLL_INTERVAL
-              value: "15"
-          resources:
-            requests:
-              cpu: 10m
-              memory: 32Mi
-            limits:
-              cpu: 100m
-              memory: 64Mi
-      # No volumes — Postgres handles persistence

k8s/apps/gotify/ingress.yaml

@@ -1,26 +0,0 @@
-apiVersion: networking.k8s.io/v1
-kind: Ingress
-metadata:
-  name: gotify
-  namespace: notifications
-  annotations:
-    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
-    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
-    # WebSocket support for Gotify client connections
-    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
-    nginx.ingress.kubernetes.io/configuration-snippet: |
-      proxy_set_header Upgrade $http_upgrade;
-      proxy_set_header Connection "upgrade";
-spec:
-  ingressClassName: nginx
-  rules:
-    - host: gotify.riotpiao.com
-      http:
-        paths:
-          - path: /
-            pathType: Prefix
-            backend:
-              service:
-                name: gotify
-                port:
-                  number: 80

k8s/apps/gotify/kustomization.yaml

@@ -1,8 +0,0 @@
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-resources:
-  - namespace.yaml
-  - db.yaml
-  - deployment.yaml
-  - service.yaml
-  - ingress.yaml

k8s/apps/gotify/namespace.yaml

@@ -1,6 +0,0 @@
-apiVersion: v1
-kind: Namespace
-metadata:
-  name: notifications
-  labels:
-    kubernetes.io/metadata.name: notifications

k8s/apps/gotify/service.yaml

@@ -1,14 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
-  name: gotify
-  namespace: notifications
-  labels:
-    app: gotify
-spec:
-  selector:
-    app: gotify
-  ports:
-    - port: 80
-      targetPort: 80
-      protocol: TCP

k8s/apps/immich/configmap.yaml

@@ -1,14 +0,0 @@
-apiVersion: v1
-kind: ConfigMap
-metadata:
-  name: immich-config
-data:
-  DB_HOSTNAME: "immich-db-rw"
-  DB_DATABASE_NAME: "immich"
-  # Only pgvector is installed (see db.yaml) - no vectorchord extension image
-  # exists for pg18 in CNPG's catalog yet. Explicit instead of relying on
-  # auto-detect's vectorchord-first preference order.
-  DB_VECTOR_EXTENSION: "pgvector"
-  REDIS_HOSTNAME: "immich-redis"
-  IMMICH_MACHINE_LEARNING_URL: "http://immich-machine-learning:3003"
-  TZ: "America/Los_Angeles"

k8s/apps/immich/db.yaml

@@ -1,58 +0,0 @@
-# Dedicated CNPG Postgres for Immich. Same recipe as paperless-db/authentik-db
-# (2 instances, default longhorn storage class) except the operand is
-# PostgreSQL 18, not 16.2 - the official CNPG pgvector extension image
-# (ghcr.io/cloudnative-pg/pgvector) is only published for pg18, no pg16 tags
-# exist in that registry. Immich itself supports pg18 fine (immich-app's own
-# postgres image already ships 18-vectorchord builds).
-#
-# pgvector loaded via CNPG's ImageVolume extension mechanism (CNPG 1.27+,
-# k8s ImageVolume feature - both present here: operator is 1.30.0, cluster is
-# v1.36.1). No shared_preload_libraries needed - pgvector doesn't require
-# preload, just CREATE EXTENSION, which immich-server issues itself at
-# startup. Distro/pg-major must match between the operand image and the
-# extension image (both "18"+"trixie" here) - CNPG's own compatibility rule.
-apiVersion: postgresql.cnpg.io/v1
-kind: Cluster
-metadata:
-  name: immich-db
-  annotations:
-    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
-spec:
-  instances: 3
-  imageName: ghcr.io/cloudnative-pg/postgresql:18-minimal-trixie
-  postgresql:
-    extensions:
-      - name: pgvector
-        image:
-          reference: ghcr.io/cloudnative-pg/pgvector:0.8.1-18-trixie
-  bootstrap:
-    initdb:
-      database: immich
-      owner: app
-      encoding: UTF8
-      localeCollate: C
-      localeCType: C
-      # CREATE EXTENSION vector requires superuser (pgvector's control file
-      # isn't marked trusted) and the "app" owner role isn't one
-      # (enableSuperuserAccess: false, repo convention) - postInitApplicationSQL
-      # runs as superuser during initdb, before the app ever connects. Only
-      # fires on a fresh bootstrap; the live cluster already had this run
-      # manually once (kubectl exec ... psql -U postgres -c 'CREATE EXTENSION').
-      postInitApplicationSQL:
-        - "CREATE EXTENSION IF NOT EXISTS vector;"
-        - "CREATE EXTENSION IF NOT EXISTS cube;"
-        - "CREATE EXTENSION IF NOT EXISTS earthdistance;"
-  enableSuperuserAccess: false
-  resources:
-    requests: { memory: "512Mi", cpu: "250m" }
-    limits: { memory: "2Gi", cpu: "1" }
-  storage:
-    size: 20Gi
-    storageClass: longhorn
-  affinity:
-    podAntiAffinityType: preferred
-    topologyKey: kubernetes.io/hostname
-    tolerations:
-      - key: node-role.kubernetes.io/control-plane
-        operator: Exists
-        effect: NoSchedule

k8s/apps/immich/deployment.yaml

@@ -1,100 +0,0 @@
-# immich-server: pinned to talos-cp-3, same reasoning as paperless
-# (deployment.yaml comment there) - immich-media is a ReadWriteOnce Longhorn
-# volume with a single replica physically on that node's disk (shared with
-# paperless-media on the same 4TB HDD). Recreate strategy for the same
-# reason: two pods can't both attach an RWO volume.
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: immich-server
-spec:
-  replicas: 1
-  strategy:
-    type: Recreate
-  selector:
-    matchLabels:
-      app: immich-server
-  template:
-    metadata:
-      labels:
-        app: immich-server
-    spec:
-      serviceAccountName: immich
-      nodeSelector:
-        kubernetes.io/hostname: talos-cp-3
-      containers:
-        - name: immich-server
-          image: ghcr.io/immich-app/immich-server:release
-          ports:
-            - containerPort: 2283
-          envFrom:
-            - configMapRef:
-                name: immich-config
-          env:
-            - name: DB_USERNAME
-              valueFrom:
-                secretKeyRef:
-                  name: immich-db-app
-                  key: username
-            - name: DB_PASSWORD
-              valueFrom:
-                secretKeyRef:
-                  name: immich-db-app
-                  key: password
-            # Composed by k8s/infra/iam's provisioning script (system-config
-            # JSON, oauth section) - see immich-oidc Secret.
-            - name: IMMICH_CONFIG_FILE
-              value: /config/immich.json
-          resources:
-            requests: { cpu: "500m", memory: "1Gi" }
-            limits: { cpu: "2", memory: "4Gi" }
-          volumeMounts:
-            - name: media
-              mountPath: /usr/src/app/upload
-            - name: oidc-config
-              mountPath: /config
-              readOnly: true
-      volumes:
-        - name: media
-          persistentVolumeClaim:
-            claimName: immich-media
-        - name: oidc-config
-          secret:
-            secretName: immich-oidc
-            items:
-              - key: config.json
-                path: immich.json
----
-# CPU-only for now - the cluster's one GPU node (worker-1) is already
-# dedicated to llm-serving predictors. Not node-pinned: its cache PVC is on
-# the default 3-replica pool, not the single-disk cp-3 HDD.
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: immich-machine-learning
-spec:
-  replicas: 1
-  selector:
-    matchLabels:
-      app: immich-machine-learning
-  template:
-    metadata:
-      labels:
-        app: immich-machine-learning
-    spec:
-      serviceAccountName: immich
-      containers:
-        - name: immich-machine-learning
-          image: ghcr.io/immich-app/immich-machine-learning:release
-          ports:
-            - containerPort: 3003
-          resources:
-            requests: { cpu: "500m", memory: "1Gi" }
-            limits: { cpu: "2", memory: "4Gi" }
-          volumeMounts:
-            - name: ml-cache
-              mountPath: /cache
-      volumes:
-        - name: ml-cache
-          persistentVolumeClaim:
-            claimName: immich-ml-cache

k8s/apps/immich/ingress.yaml

@@ -1,24 +0,0 @@
-# Direct nginx ingress, same reasoning as paperless: large uploads (photos/
-# videos) and long-lived operations (video transcode, big batch uploads) need
-# proxy-body-size/timeouts raised past nginx's defaults.
-apiVersion: networking.k8s.io/v1
-kind: Ingress
-metadata:
-  name: immich
-  annotations:
-    nginx.ingress.kubernetes.io/proxy-body-size: "0"
-    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
-    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
-spec:
-  ingressClassName: nginx
-  rules:
-    - host: img.riotpiao.com
-      http:
-        paths:
-          - path: /
-            pathType: Prefix
-            backend:
-              service:
-                name: immich-server
-                port:
-                  number: 2283

k8s/apps/immich/kustomization.yaml

@@ -1,14 +0,0 @@
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-namespace: immich
-resources:
-  - db.yaml
-  - pvc.yaml
-  - configmap.yaml
-  - redis.yaml
-  - deployment.yaml
-  - service.yaml
-  - ingress.yaml
-  - rbac.yaml
-# immich-oidc Secret written by the PostSync provisioning Job in
-# k8s/infra/iam (same as paperless-oidc) - not duplicated here.

k8s/apps/immich/pvc.yaml

@@ -1,38 +0,0 @@
-# Two volumes:
-#
-# - media: original photos/videos + generated thumbnails/encoded videos.
-#   Shares the cp-3 USB HDD with paperless-media, same StorageClass/disk tag,
-#   single replica (single disk, no redundancy possible - same tradeoff
-#   paperless already accepts). Sized 1400Gi, not 2000Gi: the disk's real
-#   usable capacity (~3724GiB, formatting overhead) minus paperless-media's
-#   2000Gi and ~231GiB of other apps' default-class replicas that Longhorn
-#   placed here anyway (disk tags only pull matching volumes in, they don't
-#   exclude non-matching ones when the untagged pool elsewhere is full) only
-#   leaves ~1493Gi of real scheduling headroom right now.
-# - ml-cache: downloaded ML model weights for immich-machine-learning
-#   (face detection / CLIP embeddings). Small, disposable (re-downloads on
-#   loss), but persisted so a pod restart doesn't re-pull multi-GB models -
-#   default 3-replica pool, not node-pinned.
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: immich-media
-spec:
-  accessModes:
-    - ReadWriteOnce
-  storageClassName: longhorn-paperless-media
-  resources:
-    requests:
-      storage: 1400Gi
----
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: immich-ml-cache
-spec:
-  accessModes:
-    - ReadWriteOnce
-  storageClassName: longhorn
-  resources:
-    requests:
-      storage: 5Gi

k8s/apps/immich/rbac.yaml

@@ -1,43 +0,0 @@
-# Scoped operator access for immich-admins: restart/config-edit rights on
-# just this service's own resources, nothing CNPG-managed (immich-db-*) or
-# provisioning-managed (immich-oidc). Same pattern as
-# k8s/apps/paperless/rbac.yaml. Inert until kube-apiserver's OIDC wiring
-# lands (--oidc-groups-claim=groups, --oidc-groups-prefix=oidc:).
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: immich
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: Role
-metadata:
-  name: immich-operator
-rules:
-  - apiGroups: ["apps"]
-    resources: ["deployments"]
-    resourceNames: ["immich-server", "immich-machine-learning"]
-    verbs: ["get", "list", "watch", "update", "patch"]
-  - apiGroups: [""]
-    resources: ["configmaps"]
-    resourceNames: ["immich-config"]
-    verbs: ["get", "list", "watch", "update", "patch"]
-  - apiGroups: [""]
-    resources: ["secrets"]
-    resourceNames: ["immich-oidc"]
-    verbs: ["get", "list", "watch", "update", "patch"]
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: RoleBinding
-metadata:
-  name: immich-admins-binding
-subjects:
-  - kind: Group
-    name: "oidc:immich-admins"
-    apiGroup: rbac.authorization.k8s.io
-  - kind: ServiceAccount
-    name: immich
-    namespace: immich
-roleRef:
-  kind: Role
-  name: immich-operator
-  apiGroup: rbac.authorization.k8s.io

k8s/apps/immich/redis.yaml

@@ -1,37 +0,0 @@
-# Job queue broker for immich-server. No PVC: queue state is disposable - a
-# lost queue on restart just re-triggers the affected background jobs
-# (thumbnail generation, ML jobs, etc.), no photo data loss since originals
-# live on immich-media.
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: immich-redis
-spec:
-  replicas: 1
-  selector:
-    matchLabels:
-      app: immich-redis
-  template:
-    metadata:
-      labels:
-        app: immich-redis
-    spec:
-      containers:
-        - name: redis
-          image: redis:7-alpine
-          ports:
-            - containerPort: 6379
-          resources:
-            requests: { cpu: "50m", memory: "64Mi" }
-            limits: { cpu: "250m", memory: "256Mi" }
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: immich-redis
-spec:
-  selector:
-    app: immich-redis
-  ports:
-    - port: 6379
-      targetPort: 6379

k8s/apps/immich/service.yaml

@@ -1,21 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
-  name: immich-server
-spec:
-  selector:
-    app: immich-server
-  ports:
-    - port: 2283
-      targetPort: 2283
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: immich-machine-learning
-spec:
-  selector:
-    app: immich-machine-learning
-  ports:
-    - port: 3003
-      targetPort: 3003

k8s/apps/llm-serving/kustomization.yaml

@@ -14,7 +14,5 @@ resources:
   - ornith.yaml
   - reasoning.yaml
   - reranker.yaml
-  - qwen-cpu.yaml
-  - networkpolicy.yaml
 # No namespace transformer: every file sets its own, and the transformer would
 # rewrite metadata.namespace on anything cross-namespace added later.

k8s/apps/llm-serving/networkpolicy.yaml

@@ -1,62 +0,0 @@
-# NetworkPolicy for LLM inference engines (llm-serving namespace).
-#
-# These pods have NO auth — vLLM, Ollama, and TEI accept any request.
-# All access MUST go through the api-gateway, which validates JWTs and
-# injects identity headers (X-Forwarded-User, X-Auth-Verified).
-#
-# Replaces the hand-applied llm-serving-default-deny policy that used
-# `llm-client: "true"` pod label as a selector — any pod in any namespace
-# could self-grant access by adding that label, which defeats the purpose.
-#
-# This policy restricts ingress to:
-#   1. api namespace (gateway) — the sole entry point for inference
-#   2. monitoring namespace — Prometheus scraping vLLM/TEI /metrics
-#   3. intra-namespace — pod-to-pod (future: multi-replica comms)
-apiVersion: networking.k8s.io/v1
-kind: NetworkPolicy
-metadata:
-  name: llm-serving-ingress
-  namespace: llm-serving
-  labels:
-    app.kubernetes.io/part-of: llm-serving
-spec:
-  podSelector:
-    matchLabels:
-      app.kubernetes.io/part-of: llm-serving
-  policyTypes:
-    - Ingress
-  ingress:
-    # Allow from api-gateway (namespace: api)
-    # Gateway proxies /v1/chat/completions, /v1/embeddings, /v1/rerank
-    - from:
-        - namespaceSelector:
-            matchLabels:
-              kubernetes.io/metadata.name: api
-      ports:
-        - protocol: TCP
-          port: 8080   # vLLM, Ollama HTTP
-        - protocol: TCP
-          port: 80      # KServe predictor services
-        - protocol: TCP
-          port: 8000    # vLLM direct (some configs)
-        - protocol: TCP
-          port: 11434   # Ollama native port
-    # Allow Prometheus scraping from monitoring namespace
-    # vLLM: :8080/metrics, TEI: :9000/metrics
-    - from:
-        - namespaceSelector:
-            matchLabels:
-              kubernetes.io/metadata.name: monitoring
-      ports:
-        - protocol: TCP
-          port: 8080
-        - protocol: TCP
-          port: 9000
-    # Allow intra-namespace (pod-to-pod within llm-serving)
-    - from:
-        - podSelector:
-            matchLabels:
-              app.kubernetes.io/part-of: llm-serving
-      ports:
-        - protocol: TCP
-          port: 8080

k8s/apps/llm-serving/ornith.yaml

@@ -33,8 +33,12 @@ spec:
 
         ollama pull ornith:35b
 
+        ollama pull qwen2.5:3b-instruct
+
         ollama run ornith:35b "ok" >/dev/null 2>&1 || true
 
+        ollama run qwen2.5:3b-instruct "ok" >/dev/null 2>&1 || true
+
         wait $SERVE_PID
 
         '
@@ -50,7 +54,7 @@ spec:
       - name: OLLAMA_NUM_PARALLEL
         value: '1'
       - name: OLLAMA_MAX_LOADED_MODELS
-        value: '1'
+        value: '2'
       image: ollama/ollama:0.32.9@sha256:1685741456770df6e3cceb2a945a5f75e020f658d1701509668d6f4688f1dd3f
       name: kserve-container
       ports:
@@ -61,7 +65,8 @@ spec:
           command:
           - /bin/sh
           - -c
-          - ollama ps 2>/dev/null | grep -q ornith
+          - ollama ps 2>/dev/null | grep -q ornith && ollama ps 2>/dev/null |
+            grep -q qwen2.5
         periodSeconds: 10
       resources:
         limits:
@@ -77,7 +82,8 @@ spec:
           command:
           - /bin/sh
           - -c
-          - ollama ps 2>/dev/null | grep -q ornith
+          - ollama ps 2>/dev/null | grep -q ornith && ollama ps 2>/dev/null |
+            grep -q qwen2.5
         failureThreshold: 120
         periodSeconds: 15
       volumeMounts:
@@ -85,10 +91,13 @@ spec:
         name: models
     deploymentStrategy:
       type: Recreate
-    # 1 replica -- ornith:35b only. qwen2.5:3b moved to CPU on cp-2.
-    # Frees 1 GPU for ComfyUI.
-    maxReplicas: 1
-    minReplicas: 1
+    # 2 replicas -- each its own GPU, each loading both ornith:35b and
+    # qwen2.5:3b-instruct -- so 2 concurrent implementer-style calls each
+    # get an independent instance instead of contending on one, at the
+    # cost of judge/qwen traffic still sharing whichever replica an
+    # implementer call also lands on.
+    maxReplicas: 2
+    minReplicas: 2
     nodeSelector:
       kubernetes.io/hostname: worker-1
     runtimeClassName: nvidia

k8s/apps/llm-serving/qwen-cpu.yaml

@@ -1,115 +0,0 @@
-# qwen2.5:3b-instruct on CPU (talos-cp-2, 144GB RAM, 24 cores).
-# Moved off GPU to free a V100 for ComfyUI. Latency ~10x slower
-# than GPU but sufficient for lightweight tasks (summarization,
-# classification, quick answers).
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: qwen-cpu
-  namespace: llm-serving
-  labels:
-    app: qwen-cpu
-    app.kubernetes.io/name: qwen-cpu
-    app.kubernetes.io/part-of: llm-serving
-spec:
-  replicas: 1
-  strategy:
-    type: Recreate
-  selector:
-    matchLabels:
-      app: qwen-cpu
-  template:
-    metadata:
-      labels:
-        app: qwen-cpu
-        app.kubernetes.io/name: qwen-cpu
-        app.kubernetes.io/part-of: llm-serving
-    spec:
-      nodeSelector:
-        kubernetes.io/hostname: talos-cp-2
-      tolerations:
-        - key: node-role.kubernetes.io/control-plane
-          operator: Exists
-          effect: NoSchedule
-      containers:
-        - name: ollama
-          image: ollama/ollama:0.32.9@sha256:1685741456770df6e3cceb2a945a5f75e020f658d1701509668d6f4688f1dd3f
-          command: ["/bin/sh", "-c"]
-          args:
-            - |
-              ollama serve &
-              SERVE_PID=$!
-              until ollama list >/dev/null 2>&1; do sleep 2; done
-              ollama pull qwen2.5:3b-instruct
-              ollama run qwen2.5:3b-instruct "ok" >/dev/null 2>&1 || true
-              wait $SERVE_PID
-          env:
-            - name: OLLAMA_HOST
-              value: "0.0.0.0:8080"
-            - name: OLLAMA_MODELS
-              value: /root/.ollama/models
-            - name: OLLAMA_CONTEXT_LENGTH
-              value: "32768"
-            - name: OLLAMA_KEEP_ALIVE
-              value: "-1"
-            - name: OLLAMA_MAX_LOADED_MODELS
-              value: "1"
-            - name: OLLAMA_NUM_PARALLEL
-              value: "2"
-          ports:
-            - containerPort: 8080
-              protocol: TCP
-          readinessProbe:
-            exec:
-              command: ["/bin/sh", "-c", "ollama ps 2>/dev/null | grep -q qwen2.5"]
-            periodSeconds: 10
-          startupProbe:
-            exec:
-              command: ["/bin/sh", "-c", "ollama ps 2>/dev/null | grep -q qwen2.5"]
-            failureThreshold: 60
-            periodSeconds: 10
-          resources:
-            requests:
-              cpu: "4"
-              memory: 4Gi
-            limits:
-              cpu: "8"
-              memory: 8Gi
-          volumeMounts:
-            - mountPath: /root/.ollama
-              name: ollama-data
-      volumes:
-        - name: ollama-data
-          persistentVolumeClaim:
-            claimName: qwen-cpu-data
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: qwen-cpu
-  namespace: llm-serving
-  labels:
-    app: qwen-cpu
-    app.kubernetes.io/part-of: llm-serving
-spec:
-  selector:
-    app: qwen-cpu
-  ports:
-    - port: 80
-      targetPort: 8080
-      protocol: TCP
----
-# Small PVC for qwen2.5:3b model weights (~1.9GB).
-# Separate from llm-models PVC which is pinned to worker-1.
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: qwen-cpu-data
-  namespace: llm-serving
-spec:
-  accessModes:
-    - ReadWriteOnce
-  storageClassName: longhorn
-  resources:
-    requests:
-      storage: 5Gi

k8s/apps/messaging/management-service/templates/deployment.yaml

@@ -13,7 +13,6 @@ spec:
       labels:
         app: management-service
     spec:
-      serviceAccountName: kmsvc
       topologySpreadConstraints:
         - maxSkew: 1
           topologyKey: kubernetes.io/hostname

k8s/apps/messaging/management-service/values.yaml

@@ -2,7 +2,7 @@ namespace: sqs
 replicaCount: 3
 
 image:
-  repository: forgejo.riotpiao.com/rock/kmsvc-manage
+  repository: ghcr.io/riotpiaole/kmsvc-management-service
   tag: latest
   pullPolicy: Always
 

k8s/apps/messaging/memory-queues/Chart.yaml

@@ -1,6 +0,0 @@
-apiVersion: v2
-name: memory-queues
-description: Kafka queues (DLQ) for Poimen Memory service (Phase 6.6)
-type: application
-version: 0.1.0
-appVersion: "1.0"

k8s/apps/messaging/memory-queues/templates/queues.yaml

@@ -1,20 +0,0 @@
-{{- range .Values.queues }}
----
-apiVersion: kmsvc.io/v1alpha1
-kind: Queue
-metadata:
-  name: {{ .name }}
-  namespace: {{ $.Values.namespace }}
-  labels:
-    app: memory-service
-    queue: dlq
-spec:
-  name: {{ .name }}
-  description: {{ .description }}
-  partitions: {{ .partitions }}
-  replicationFactor: {{ .replicationFactor }}
-  config:
-    retention.ms: "{{ .config.retention.ms }}"
-    message.retention.seconds: "{{ .config.message.retention.seconds }}"
-    visibility.timeout.seconds: "{{ .config.visibility.timeout.seconds }}"
-{{- end }}

k8s/apps/messaging/memory-queues/values.yaml

@@ -1,25 +0,0 @@
-# Poimen Memory Service Kafka Queues (kmsvc)
-# Phase 6.6: DLQ topics for webhook + metrics failures
-
-queues:
-  # DLQ for extraction, webhook, and agent failures
-  - name: poimen-memory-dlq
-    description: "DLQ for extraction, webhook, and agent failures"
-    partitions: 3
-    replicationFactor: 1
-    config:
-      retention.ms: "1209600000"  # 14 days
-      message.retention.seconds: "1209600"
-      visibility.timeout.seconds: "300"
-
-  # DLQ for metrics persistence failures  
-  - name: poimen-memory-metric-dlq
-    description: "DLQ for metrics persistence failures"
-    partitions: 3
-    replicationFactor: 1
-    config:
-      retention.ms: "1209600000"  # 14 days
-      message.retention.seconds: "1209600"
-      visibility.timeout.seconds: "300"
-
-namespace: sqs

k8s/apps/messaging/queue-crd/values.yaml

@@ -1,7 +1,7 @@
 namespace: sqs
 
 image:
-  repository: forgejo.riotpiao.com/rock/kmsvc-manage
+  repository: ghcr.io/riotpiaole/kmsvc-management-service
   tag: latest
   pullPolicy: Always
 

k8s/apps/paperless/adapter-configmap.yaml

@@ -1,95 +0,0 @@
-# Overrides paperless-ngx's own paperless/adapter.py at the same import path
-# (mounted via subPath in deployment.yaml) - settings.py hardcodes
-# SOCIALACCOUNT_ADAPTER = "paperless.adapter.CustomSocialAccountAdapter", so
-# no Django setting needs to change, just the file content underneath it.
-#
-# Stock CustomSocialAccountAdapter.populate_user() is a stub ("kept in case
-# global default permissions are implemented in the future" - they aren't),
-# so every OIDC signup lands with zero permissions and 403s on every API
-# endpoint. This adds the actual mapping: Authentik's "permissions" claim
-# (via the permissions scope, requested in PAPERLESS_SOCIALACCOUNT_PROVIDERS,
-# computed server-side from group membership by authentik-provision.py) ->
-# "paperless:write" or "*" (homelab-admins) grants is_staff+is_superuser,
-# same convention already used for MinIO's policy claim and Grafana's
-# role_attribute_path. Checking the permission string rather than a literal
-# group name decouples "what grants access" from which group happens to
-# hold it - same pattern applies to every other service's Role/RoleBinding
-# in k8s/infra/rbac/.
-apiVersion: v1
-kind: ConfigMap
-metadata:
-  name: paperless-adapter
-data:
-  adapter.py: |
-    from urllib.parse import quote
-
-    from allauth.account.adapter import DefaultAccountAdapter
-    from allauth.core import context
-    from allauth.socialaccount.adapter import DefaultSocialAccountAdapter
-    from django.conf import settings
-    from django.forms import ValidationError
-    from django.urls import reverse
-
-    REQUIRED_PERMISSIONS = {"paperless:write", "*"}
-
-
-    class CustomAccountAdapter(DefaultAccountAdapter):
-        def is_open_for_signup(self, request):
-            allow_signups = super().is_open_for_signup(request)
-            return getattr(settings, "ACCOUNT_ALLOW_SIGNUPS", allow_signups)
-
-        def pre_authenticate(self, request, **credentials):
-            if settings.DISABLE_REGULAR_LOGIN:
-                raise ValidationError("Regular login is disabled")
-            return super().pre_authenticate(request, **credentials)
-
-        def is_safe_url(self, url):
-            from django.utils.http import url_has_allowed_host_and_scheme
-
-            allowed_hosts = {context.request.get_host()} | set(settings.ALLOWED_HOSTS)
-            if "*" in allowed_hosts:
-                allowed_hosts.remove("*")
-                allowed_hosts.add(context.request.get_host())
-                return url_has_allowed_host_and_scheme(url, allowed_hosts=allowed_hosts)
-            return url_has_allowed_host_and_scheme(url, allowed_hosts=allowed_hosts)
-
-        def get_reset_password_from_key_url(self, key):
-            if settings.PAPERLESS_URL is None:
-                return super().get_reset_password_from_key_url(key)
-            path = reverse(
-                "account_reset_password_from_key",
-                kwargs={"uidb36": "UID", "key": "KEY"},
-            )
-            path = path.replace("UID-KEY", quote(key))
-            return settings.PAPERLESS_URL + path
-
-
-    class CustomSocialAccountAdapter(DefaultSocialAccountAdapter):
-        def is_open_for_signup(self, request, sociallogin):
-            allow_signups = super().is_open_for_signup(request, sociallogin)
-            return getattr(settings, "SOCIALACCOUNT_ALLOW_SIGNUPS", allow_signups)
-
-        def get_connect_redirect_url(self, request, socialaccount):
-            return reverse("base")
-
-        def populate_user(self, request, sociallogin, data):
-            user = super().populate_user(request, sociallogin, data)
-            perms = set(sociallogin.account.extra_data.get("permissions") or [])
-            if perms & REQUIRED_PERMISSIONS:
-                user.is_staff = True
-                user.is_superuser = True
-            return user
-
-        def save_user(self, request, sociallogin, form=None):
-            # populate_user() sets the flags on the in-memory user, but
-            # allauth's default save_user() re-derives is_staff from
-            # ACCOUNT_DEFAULT_HTTP_PROTOCOL-independent defaults and can
-            # overwrite them on save - re-apply after super().save_user()
-            # persists the row, matching the permissions check above exactly.
-            user = super().save_user(request, sociallogin, form)
-            perms = set(sociallogin.account.extra_data.get("permissions") or [])
-            if perms & REQUIRED_PERMISSIONS and not (user.is_staff and user.is_superuser):
-                user.is_staff = True
-                user.is_superuser = True
-                user.save(update_fields=["is_staff", "is_superuser"])
-            return user

k8s/apps/paperless/configmap.yaml

@@ -15,8 +15,3 @@ data:
   # env vars can't be split across a ConfigMap + Secret for the same key, so
   # this whole value is sourced from the Secret in deployment.yaml instead.
   PAPERLESS_APPS: "allauth.socialaccount.providers.openid_connect"
-  # Authentik already verifies identity via OIDC - a second email-confirmation
-  # step has no SMTP configured to send it anyway, and paperless-ngx doesn't
-  # wire up allauth's confirm-email view, so signup 500s with NoReverseMatch
-  # on 'account_confirm_email' without this.
-  PAPERLESS_ACCOUNT_EMAIL_VERIFICATION: "none"

k8s/apps/paperless/deployment.yaml

@@ -24,18 +24,11 @@ spec:
       labels:
         app: paperless
     spec:
-      # Kubernetes injects legacy Docker-links env vars for every Service in
-      # this namespace (<SVC>_SERVICE_HOST, <SVC>_PORT, ...). The Service here
-      # is named "paperless", so that becomes PAPERLESS_PORT=tcp://<ip>:8000 -
-      # paperless-ngx's own entrypoint reads PAPERLESS_PORT for gunicorn's
-      # bind address, collides, and gunicorn crash-loops on "not a valid port
-      # number". Disable the injection instead of renaming the Service.
-      enableServiceLinks: false
       nodeSelector:
         kubernetes.io/hostname: talos-cp-3
       containers:
         - name: paperless
-          image: ghcr.io/paperless-ngx/paperless-ngx:2.20.15
+          image: ghcr.io/paperless-ngx/paperless-ngx:2.13
           ports:
             - containerPort: 8000
           envFrom:
@@ -82,13 +75,6 @@ spec:
               mountPath: /usr/src/paperless/data
             - name: consume
               mountPath: /usr/src/paperless/consume
-            # Overrides paperless-ngx's own adapter.py in place - settings.py
-            # hardcodes the import path, so no Django setting changes, just
-            # the file content underneath it (see adapter-configmap.yaml).
-            - name: adapter
-              mountPath: /usr/src/paperless/src/paperless/adapter.py
-              subPath: adapter.py
-              readOnly: true
       volumes:
         - name: media
           persistentVolumeClaim:
@@ -98,6 +84,3 @@ spec:
             claimName: paperless-data
         - name: consume
           emptyDir: {}
-        - name: adapter
-          configMap:
-            name: paperless-adapter

k8s/apps/paperless/kustomization.yaml

@@ -9,8 +9,6 @@ resources:
   - service.yaml
   - ingress.yaml
   - backup-cronjob.yaml
-  - adapter-configmap.yaml
-  - rbac.yaml
 # postgres: paperless-db CNPG Cluster, deployed by k8s/infra/databases (wave 2,
 # before this app at wave 8) - not duplicated here. Same for the paperless-oidc
 # and paperless-minio-creds Secrets, written by PostSync provisioning Jobs in

k8s/apps/paperless/pvc.yaml

@@ -1,11 +1,9 @@
 # Two volumes, deliberately separate storage classes:
 #
 # - media: the actual documents (originals + OCR'd archive PDFs + thumbnails).
-#   Lives on the cp-3 USB HDD, single replica (see
-#   k8s/infra/longhorn/longhorn-paperless-storageclass.yaml). Shares the disk
-#   with Immich's immich-media PVC (k8s/apps/immich/pvc.yaml, 2000Gi) - photo
-#   libraries grow much faster than scanned documents, so paperless gets the
-#   smaller 500Gi share.
+#   Grows to multi-TB, lives on the cp-3 USB HDD, single replica (see
+#   k8s/infra/longhorn/longhorn-paperless-storageclass.yaml). Sized 3500Gi to
+#   leave headroom on the 4TB disk rather than claiming it to 100%.
 # - data: the SQLite classification model + search index. Small (low GB),
 #   frequently rewritten, and disposable (rebuilds from the DB + media on
 #   next consume) - stays on the default 3-replica pool instead of the
@@ -20,7 +18,7 @@ spec:
   storageClassName: longhorn-paperless-media
   resources:
     requests:
-      storage: 500Gi
+      storage: 3500Gi
 ---
 apiVersion: v1
 kind: PersistentVolumeClaim

k8s/apps/paperless/rbac.yaml

@@ -1,35 +0,0 @@
-# Scoped operator access for paperless-admins: restart/config-edit rights on
-# just this service's own resources, nothing CNPG-managed (paperless-db-*)
-# or provisioning-managed (paperless-oidc, paperless-minio-creds). Inert
-# until kube-apiserver's OIDC wiring lands (--oidc-groups-claim=groups,
-# --oidc-groups-prefix=oidc:) - subject name below assumes that prefix.
-apiVersion: rbac.authorization.k8s.io/v1
-kind: Role
-metadata:
-  name: paperless-operator
-rules:
-  - apiGroups: ["apps"]
-    resources: ["deployments"]
-    resourceNames: ["paperless"]
-    verbs: ["get", "list", "watch", "update", "patch"]
-  - apiGroups: [""]
-    resources: ["configmaps"]
-    resourceNames: ["paperless-config"]
-    verbs: ["get", "list", "watch", "update", "patch"]
-  - apiGroups: [""]
-    resources: ["secrets"]
-    resourceNames: ["paperless-secrets"]
-    verbs: ["get", "list", "watch", "update", "patch"]
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: RoleBinding
-metadata:
-  name: paperless-admins-binding
-subjects:
-  - kind: Group
-    name: "oidc:paperless-admins"
-    apiGroup: rbac.authorization.k8s.io
-roleRef:
-  kind: Role
-  name: paperless-operator
-  apiGroup: rbac.authorization.k8s.io

k8s/apps/secret-rotation-controller/crd.yaml

@@ -1,144 +0,0 @@
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
-  name: secretrotations.homelab.riotpiao.com
-spec:
-  group: homelab.riotpiao.com
-  names:
-    kind: SecretRotation
-    plural: secretrotations
-  scope: Namespaced
-  versions:
-  - name: v1
-    served: true
-    storage: true
-    schema:
-      openAPIV3Schema:
-        type: object
-        properties:
-          metadata:
-            type: object
-          spec:
-            type: object
-            required:
-            - provider
-            - rotationInterval
-            properties:
-              # External system: authentik | forgejo | minio | vault
-              provider:
-                type: string
-                enum: [authentik, forgejo, minio, vault]
-              
-              # How often to rotate (hours)
-              rotationInterval:
-                type: integer
-                minimum: 24
-              
-              # Application ID in external system
-              appId:
-                type: string
-              
-              # k8s Secret to update (name, namespace, key)
-              secretRef:
-                type: object
-                required: [name, namespace]
-                properties:
-                  name:
-                    type: string
-                  namespace:
-                    type: string
-                  key:
-                    type: string
-                    description: "Secret key to update (e.g., MINIO_IDENTITY_OPENID_CLIENT_SECRET)"
-              
-              # Path to git file that holds the secret (for .enc.yaml files)
-              gitPath:
-                type: string
-                description: "Path in homelab repo to .enc.yaml file"
-              
-              # Ansible template values to substitute
-              templateValues:
-                type: object
-                additionalProperties:
-                  type: string
-          
-          status:
-            type: object
-            properties:
-              lastRotationTime:
-                type: string
-                format: date-time
-              nextRotationTime:
-                type: string
-                format: date-time
-              lastRotationStatus:
-                type: string
-                enum: [Success, Failed, Pending]
-              lastRotationError:
-                type: string
-              lastCommitHash:
-                type: string
-
----
-# Example usage:
-apiVersion: homelab.riotpiao.com/v1
-kind: SecretRotation
-metadata:
-  name: minio-oidc
-  namespace: secret-rotation
-spec:
-  provider: authentik
-  rotationInterval: 2160  # 90 days in hours
-  appId: minio
-  secretRef:
-    name: minio-oidc
-    namespace: storage
-    key: MINIO_IDENTITY_OPENID_CLIENT_SECRET
-  gitPath: k8s/argocd/secrets/minio-oidc.enc.yaml
-
----
-apiVersion: homelab.riotpiao.com/v1
-kind: SecretRotation
-metadata:
-  name: portfolio-agent-oidc
-  namespace: secret-rotation
-spec:
-  provider: authentik
-  rotationInterval: 2160
-  appId: portfolio-agent
-  secretRef:
-    name: portfolio-agent-oidc
-    namespace: portfolio
-    key: CLIENT_SECRET
-  gitPath: k8s/argocd/secrets/portfolio-agent-oidc.enc.yaml
-
----
-apiVersion: homelab.riotpiao.com/v1
-kind: SecretRotation
-metadata:
-  name: forgejo-registry-token
-  namespace: secret-rotation
-spec:
-  provider: forgejo
-  rotationInterval: 2160
-  appId: rock/riotpiao.com
-  secretRef:
-    name: forgejo-registry-secret
-    namespace: kube-system
-    key: REGISTRY_TOKEN
-  gitPath: k8s/argocd/secrets/forgejo-registry-secret.enc.yaml
-
----
-apiVersion: homelab.riotpiao.com/v1
-kind: SecretRotation
-metadata:
-  name: minio-root-credentials
-  namespace: secret-rotation
-spec:
-  provider: minio
-  rotationInterval: 4320  # 180 days in hours
-  appId: root
-  secretRef:
-    name: minio-creds
-    namespace: storage
-  gitPath: k8s/argocd/secrets/minio-secrets.enc.yaml

k8s/apps/secret-rotation-controller/deployment.yaml

@@ -1,92 +0,0 @@
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: secret-rotation-controller
-  namespace: secret-rotation
-spec:
-  replicas: 1
-  selector:
-    matchLabels:
-      app: secret-rotation-controller
-  template:
-    metadata:
-      labels:
-        app: secret-rotation-controller
-    spec:
-      serviceAccountName: secret-rotation-controller
-      containers:
-      - name: controller
-        image: secret-rotation-controller:latest
-        imagePullPolicy: IfNotPresent
-        env:
-        # SOPS reads age key from this file
-        - name: SOPS_AGE_KEY_FILE
-          value: /etc/sops/age/private-key.txt
-        
-        # Vault auth (token in projected volume)
-        - name: VAULT_ADDR
-          value: http://vault.vault.svc.cluster.local:8200
-        - name: VAULT_TOKEN_FILE
-          value: /var/run/secrets/vault/token
-        
-        # Authentik
-        - name: AUTHENTIK_URL
-          value: http://authentik-server.iam.svc.cluster.local
-        - name: AUTHENTIK_BOOTSTRAP_TOKEN
-          valueFrom:
-            secretKeyRef:
-              name: authentik-bootstrap
-              key: token
-        
-        # Git
-        - name: GIT_REPO
-          value: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-        - name: GIT_AUTHOR_EMAIL
-          value: [email protected]
-        - name: GIT_AUTHOR_NAME
-          value: Secret Rotation Controller
-        - name: FORGEJO_TOKEN
-          valueFrom:
-            secretKeyRef:
-              name: forgejo-registry-secret
-              key: REGISTRY_TOKEN
-        
-        volumeMounts:
-        # Age key from ExternalSecret (synced from Vault)
-        - name: age-key
-          mountPath: /etc/sops/age
-          readOnly: true
-        
-        # Vault auth token (projected)
-        - name: vault-token
-          mountPath: /var/run/secrets/vault
-          readOnly: true
-        
-        # Temp working dir
-        - name: tmp
-          mountPath: /tmp
-        
-        resources:
-          requests:
-            cpu: 100m
-            memory: 256Mi
-          limits:
-            cpu: 500m
-            memory: 512Mi
-      
-      volumes:
-      - name: age-key
-        secret:
-          secretName: sops-age-key
-          defaultMode: 0400
-      
-      - name: vault-token
-        projected:
-          sources:
-          - serviceAccountToken:
-              path: token
-              audience: vault
-              expirationSeconds: 3600
-      
-      - name: tmp
-        emptyDir: {}

k8s/apps/secret-rotation-controller/kustomization.yaml

@@ -1,15 +0,0 @@
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-
-namespace: secret-rotation
-
-resources:
-  - rbac.yaml
-  - crd.yaml
-  - external-secret.yaml
-  - deployment.yaml
-
-commonLabels:
-  app.kubernetes.io/name: secret-rotation-controller
-  app.kubernetes.io/component: automation
-  managed-by: argocd

k8s/apps/secret-rotation-controller/rbac.yaml

@@ -1,53 +0,0 @@
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: secret-rotation-controller
-  namespace: secret-rotation
-
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRole
-metadata:
-  name: secret-rotation-controller
-rules:
-# Read SecretRotation CRDs
-- apiGroups: ["homelab.riotpiao.com"]
-  resources: ["secretrotations"]
-  verbs: ["get", "list", "watch"]
-
-# Update status
-- apiGroups: ["homelab.riotpiao.com"]
-  resources: ["secretrotations/status"]
-  verbs: ["get", "patch", "update"]
-
-# Read k8s secrets that will be rotated
-- apiGroups: [""]
-  resources: ["secrets"]
-  verbs: ["get", "list"]
-
-# For recording events
-- apiGroups: [""]
-  resources: ["events"]
-  verbs: ["create", "patch"]
-
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRoleBinding
-metadata:
-  name: secret-rotation-controller
-roleRef:
-  apiGroup: rbac.authorization.k8s.io
-  kind: ClusterRole
-  name: secret-rotation-controller
-subjects:
-- kind: ServiceAccount
-  name: secret-rotation-controller
-  namespace: secret-rotation
-
----
-apiVersion: v1
-kind: Namespace
-metadata:
-  name: secret-rotation
-  labels:
-    kubernetes.io/metadata.name: secret-rotation

k8s/argocd/apps/-1-projects.yaml

@@ -18,7 +18,7 @@ spec:
       prune: true
       selfHeal: true
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/argocd/projects
   destination:

k8s/argocd/apps/00-secrets.yaml

@@ -17,7 +17,7 @@ spec:
     syncOptions:
       - CreateNamespace=true
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     # ksops decrypts every *.enc.yaml here at kustomize-build time (repo-server
     # runs `kustomize build --enable-alpha-plugins --enable-exec`). Replaces the

k8s/argocd/apps/00-substrate.yaml

@@ -21,7 +21,7 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/bootstrap/cert-manager/cert-manager-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:
@@ -93,7 +93,7 @@ spec:
   project: homelab
   revisionHistoryLimit: 3
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     # A real kustomization.yaml (resources: the 3 issuer/CA files) renders these
     # deterministically. The previous directory.include with bare filenames
@@ -127,7 +127,7 @@ spec:
   project: homelab
   revisionHistoryLimit: 3
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/bootstrap/ingress
   destination:
@@ -138,75 +138,3 @@ spec:
       selfHeal: true
     syncOptions:
       - CreateNamespace=true
----
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: cluster-maintenance
-  namespace: argocd
-  annotations:
-    argocd.argoproj.io/sync-wave: "0"
-spec:
-  project: homelab
-  source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-    targetRevision: main
-    path: k8s/infra/cluster-maintenance
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: kube-system
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true
----
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: kyverno
-  namespace: argocd
-  annotations:
-    argocd.argoproj.io/sync-wave: "0"
-spec:
-  project: homelab
-  source:
-    repoURL: https://kyverno.github.io/kyverno/
-    chart: kyverno
-    targetRevision: "1.14.0"
-    helm:
-      valueFiles:
-        - $values/k8s/bootstrap/kyverno/kyverno-values.yaml
-  sources:
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-      targetRevision: main
-      ref: values
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: kyverno
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true
-    syncOptions:
-      - CreateNamespace=true
----
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: kyverno-policies
-  namespace: argocd
-  annotations:
-    argocd.argoproj.io/sync-wave: "0"
-spec:
-  project: homelab
-  source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-    targetRevision: main
-    path: k8s/bootstrap/kyverno
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: kyverno
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true

k8s/argocd/apps/01-image-updater.yaml

@@ -1,33 +0,0 @@
-# ArgoCD Image Updater - auto-updates Application images from registry
-# Watches forgejo.riotpiao.com for new image tags and updates Applications
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: argocd-image-updater
-  namespace: argocd
-  finalizers:
-    - resources-finalizer.argocd.argoproj.io
-  annotations:
-    argocd.argoproj.io/sync-wave: "1"
-spec:
-  project: homelab
-  revisionHistoryLimit: 3
-  sources:
-    - repoURL: https://argoproj.github.io/argo-helm
-      chart: argocd-image-updater
-      targetRevision: "0.11.2"
-      helm:
-        valueFiles:
-          - $values/k8s/infra/argocd-image-updater/values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-      targetRevision: main
-      ref: values
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: argocd
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true
-    syncOptions:
-      - CreateNamespace=false

k8s/argocd/apps/08-secret-rotation.yaml

@@ -1,32 +0,0 @@
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: secret-rotation
-  namespace: argocd
-  labels:
-    app.kubernetes.io/name: secret-rotation
-spec:
-  project: homelab
-  
-  sources:
-  - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-    path: k8s/apps/secret-rotation-controller
-    targetRevision: main
-  
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: secret-rotation
-  
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true
-    syncOptions:
-    - CreateNamespace=true
-    - RespectIgnoreDifferences=true
-    retry:
-      limit: 5
-      backoff:
-        duration: 5s
-        factor: 2
-        maxDuration: 3m

k8s/argocd/apps/10-storage-observability.yaml

@@ -17,7 +17,7 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/infra/minio/minio-operator-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:
@@ -41,7 +41,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/infra/minio
   destination:
@@ -66,7 +66,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/infra/longhorn
   destination:
@@ -102,7 +102,7 @@ spec:
         skipCrds: true
         valueFiles:
           - $values/k8s/infra/monitoring/prometheus-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:
@@ -152,7 +152,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/infra/monitoring/crds
   destination:
@@ -183,7 +183,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/infra/monitoring
   destination:
@@ -213,7 +213,7 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/infra/monitoring/blackbox-exporter-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:
@@ -223,29 +223,3 @@ spec:
     automated:
       prune: true
       selfHeal: true
----
-# Distributed tracing: Tempo + OpenTelemetry Collector.
-# Receives traces from instrumented services, stores in local volume (72h retention).
-# Grafana datasource auto-configured, service graph + latency dashboards included.
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: tracing
-  namespace: argocd
-  annotations:
-    argocd.argoproj.io/sync-wave: "1"
-spec:
-  project: homelab
-  source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-    targetRevision: main
-    path: k8s/infra/tracing
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: tracing
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true
-    syncOptions:
-      - CreateNamespace=true

k8s/argocd/apps/20-logging.yaml

@@ -19,7 +19,7 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/infra/logging/loki-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:
@@ -53,7 +53,7 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/infra/logging/grafana-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:
@@ -87,7 +87,7 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/infra/logging/promtail-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:

k8s/argocd/apps/30-security.yaml

@@ -17,7 +17,7 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/infra/iam/vault-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:
@@ -46,7 +46,7 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/infra/iam/authentik-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:
@@ -68,7 +68,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/infra/iam
   destination:
@@ -109,7 +109,7 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/bootstrap/phase3-forgejo/forgejo-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:
@@ -152,17 +152,10 @@ metadata:
   namespace: argocd
   annotations:
     argocd.argoproj.io/sync-wave: "3"
-    argocd-image-updater.argoproj.io/image-list: runner=forgejo.riotpiao.com/rock/forgejo-runner-golang
-    argocd-image-updater.argoproj.io/runner.update-strategy: newest-build
-    argocd-image-updater.argoproj.io/runner.allow-tags: regexp:^[0-9a-f]{7}$|^latest$|^v[0-9]+$
-    argocd-image-updater.argoproj.io/runner.helm.image-name: runner.image.repository
-    argocd-image-updater.argoproj.io/runner.helm.image-tag: runner.image.tag
-    argocd-image-updater.argoproj.io/write-back-method: git
-    argocd-image-updater.argoproj.io/git-branch: main
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/infra/forgejo-runner
   destination:
@@ -180,17 +173,10 @@ metadata:
   namespace: argocd
   annotations:
     argocd.argoproj.io/sync-wave: "3"
-    argocd-image-updater.argoproj.io/image-list: runner=forgejo.riotpiao.com/rock/forgejo-runner-node
-    argocd-image-updater.argoproj.io/runner.update-strategy: newest-build
-    argocd-image-updater.argoproj.io/runner.allow-tags: regexp:^[0-9a-f]{7}$|^latest$|^v[0-9]+$
-    argocd-image-updater.argoproj.io/runner.helm.image-name: runner.image.repository
-    argocd-image-updater.argoproj.io/runner.helm.image-tag: runner.image.tag
-    argocd-image-updater.argoproj.io/write-back-method: git
-    argocd-image-updater.argoproj.io/git-branch: main
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/infra/forgejo-runner
     helm:
@@ -211,17 +197,10 @@ metadata:
   namespace: argocd
   annotations:
     argocd.argoproj.io/sync-wave: "3"
-    argocd-image-updater.argoproj.io/image-list: runner=forgejo.riotpiao.com/rock/forgejo-runner-rust
-    argocd-image-updater.argoproj.io/runner.update-strategy: newest-build
-    argocd-image-updater.argoproj.io/runner.allow-tags: regexp:^[0-9a-f]{7}$|^latest$|^v[0-9]+$
-    argocd-image-updater.argoproj.io/runner.helm.image-name: runner.image.repository
-    argocd-image-updater.argoproj.io/runner.helm.image-tag: runner.image.tag
-    argocd-image-updater.argoproj.io/write-back-method: git
-    argocd-image-updater.argoproj.io/git-branch: main
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/infra/forgejo-runner
     helm:

k8s/argocd/apps/40-data.yaml

@@ -14,7 +14,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/infra/databases
   destination:

k8s/argocd/apps/50-memory-queues.yaml

@@ -1,20 +0,0 @@
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: memory-queues
-  namespace: argocd
-  annotations:
-    argocd.argoproj.io/sync-wave: "7"
-spec:
-  project: homelab
-  source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-    targetRevision: main
-    path: k8s/apps/messaging/memory-queues
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: sqs
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true

k8s/argocd/apps/50-messaging.yaml

@@ -1,6 +1,6 @@
-# Wave 5 — Kafka (Strimzi operator + cluster CR), Redis infrastructure.
-# Strimzi/Redis are public Helm charts; kafka-cluster is a local chart.
-# queue-crd and management-service are managed by kmsvc-root (kmsvc-manage.git).
+# Wave 5 — Kafka (Strimzi operator + cluster CR), Redis, and the SQS-like
+# queue services. Strimzi/Redis are public Helm charts; kafka-cluster/queue-crd/
+# management-service are local charts (rendered from their own Chart.yaml).
 apiVersion: argoproj.io/v1alpha1
 kind: Application
 metadata:
@@ -68,7 +68,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/apps/messaging/kafka-cluster
   destination:
@@ -78,5 +78,45 @@ spec:
     automated:
       prune: true
       selfHeal: true
-# queue-crd and management-service moved to kmsvc-manage.git repo
-# Managed by kmsvc-root Application
+---
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: queue-crd
+  namespace: argocd
+  annotations:
+    argocd.argoproj.io/sync-wave: "6"
+spec:
+  project: homelab
+  source:
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
+    targetRevision: main
+    path: k8s/apps/messaging/queue-crd
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: sqs
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+---
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: management-service
+  namespace: argocd
+  annotations:
+    argocd.argoproj.io/sync-wave: "7"
+spec:
+  project: homelab
+  source:
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
+    targetRevision: main
+    path: k8s/apps/messaging/management-service
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: sqs
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true

k8s/argocd/apps/55-api-gateway.yaml

@@ -36,19 +36,14 @@ metadata:
     app.kubernetes.io/component: gateway
   annotations:
     argocd.argoproj.io/sync-wave: "7"
-    # ArgoCD Image Updater - auto-update on new image push
-    argocd-image-updater.argoproj.io/image-list: gw=forgejo.riotpiao.com/rock/api-gateway
-    argocd-image-updater.argoproj.io/gw.update-strategy: digest
-    argocd-image-updater.argoproj.io/gw.allow-tags: regexp:^latest$
-    argocd-image-updater.argoproj.io/write-back-method: argocd
 spec:
   project: homelab
   revisionHistoryLimit: 3
   sources:
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab-frontend.git
+    - repoURL: https://forgejo.riotpiao.com/rock/homelab-frontend.git
       targetRevision: main
       path: k8s
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       path: k8s/apps/api
   destination:

k8s/argocd/apps/56-llm-serving.yaml

@@ -20,7 +20,7 @@ spec:
   project: homelab
   revisionHistoryLimit: 3
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/apps/llm-serving
   destination:

k8s/argocd/apps/58-comfyui.yaml

@@ -1,32 +0,0 @@
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: comfyui
-  namespace: argocd
-  labels:
-    app.kubernetes.io/name: comfyui
-    app.kubernetes.io/component: image-generation
-  annotations:
-    argocd.argoproj.io/sync-wave: "8"
-spec:
-  project: homelab
-  revisionHistoryLimit: 3
-  source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-    targetRevision: main
-    path: k8s/apps/comfyui
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: comfyui
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true
-    syncOptions:
-      - CreateNamespace=true
-    retry:
-      limit: 5
-      backoff:
-        duration: 5s
-        factor: 2
-        maxDuration: 3m

k8s/argocd/apps/59-gotify.yaml

@@ -1,32 +0,0 @@
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: gotify
-  namespace: argocd
-  labels:
-    app.kubernetes.io/name: gotify
-    app.kubernetes.io/component: notifications
-  annotations:
-    argocd.argoproj.io/sync-wave: "8"
-spec:
-  project: homelab
-  revisionHistoryLimit: 3
-  source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-    targetRevision: main
-    path: k8s/apps/gotify
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: notifications
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true
-    syncOptions:
-      - CreateNamespace=true
-    retry:
-      limit: 5
-      backoff:
-        duration: 5s
-        factor: 2
-        maxDuration: 3m

k8s/argocd/apps/60-applications.yaml

@@ -19,10 +19,10 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/apps/temporal/temporal-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       path: k8s/apps/temporal
   destination:
@@ -51,7 +51,7 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/apps/portainer/portainer-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
   destination:
@@ -74,7 +74,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/apps/cloudflared
   destination:
@@ -97,7 +97,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/apps/agent-pod
   destination:
@@ -130,7 +130,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/apps/sms
   destination:
@@ -157,7 +157,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/apps/paperless
   destination:
@@ -170,38 +170,6 @@ spec:
     syncOptions:
       - CreateNamespace=true
 ---
-# Photo/video backup. Self-contained (unlike paperless, its CNPG Postgres
-# lives here too, not in k8s/infra/databases) - CreateNamespace=true creates
-# the namespace before any manifest in this Application applies, including
-# the Cluster CR, so no separate wave-2 pre-creation step is needed. Postgres
-# is pg18 (not this repo's usual 16.2) because CNPG's official pgvector
-# extension image only publishes pg18 builds - see k8s/apps/immich/db.yaml.
-# media PVC shares the cp-3 HDD 2TB/2TB with paperless-media. OIDC via
-# Authentik provisioned by k8s/infra/iam's PostSync job (immich entry in
-# SERVICES + immich_role scope mapping for admin-via-claim).
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: immich
-  namespace: argocd
-  annotations:
-    argocd.argoproj.io/sync-wave: "8"
-spec:
-  project: homelab
-  source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-    targetRevision: main
-    path: k8s/apps/immich
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: immich
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true
-    syncOptions:
-      - CreateNamespace=true
----
 # Consolidated: homarr + homarr-patches → homarr
 # Helm chart + values + PostSync hook patch (fix-probes-job.yaml)
 apiVersion: argoproj.io/v1alpha1
@@ -220,10 +188,10 @@ spec:
       helm:
         valueFiles:
           - $values/k8s/apps/homarr/homarr-values.yaml
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       ref: values
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    - repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
       targetRevision: main
       path: k8s/apps/homarr  # PostSync hook: fix-probes-job.yaml
   destination:
@@ -235,60 +203,3 @@ spec:
       selfHeal: true
     syncOptions:
       - CreateNamespace=true
----
-# Portfolio site at riotpiao.com - static Next.js site from rock/riotpiao.com repo.
-# Points directly to infra/portfolio/base (bypassing repo's own argocd-apps.yaml
-# which has wrong URLs). Image built by Forgejo Actions on rock/portfolio repo.
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: portfolio
-  namespace: argocd
-  annotations:
-    argocd.argoproj.io/sync-wave: "8"
-    # ArgoCD Image Updater - auto-update on new image push
-    argocd-image-updater.argoproj.io/image-list: app=forgejo.riotpiao.com/rock/portfolio
-    argocd-image-updater.argoproj.io/app.update-strategy: digest
-    argocd-image-updater.argoproj.io/app.allow-tags: regexp:^latest$
-    argocd-image-updater.argoproj.io/write-back-method: argocd
-spec:
-  project: homelab
-  source:
-    repoURL: https://forgejo.riotpiao.com/rock/riotpiao.com.git
-    targetRevision: main
-    path: infra/portfolio/base
-  destination:
-    server: https://kubernetes.default.svc
-    namespace: portfolio
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true
-    syncOptions:
-      - CreateNamespace=true
----
-# Wave 9 - per-service scoped RBAC (Role/RoleBinding), deliberately last so
-# every target namespace above already exists. Inert until kube-apiserver
-# gets --oidc-groups-claim=groups wired up (separate, not-yet-applied
-# terraform/talosctl change) - these grant nothing until then.
-apiVersion: argoproj.io/v1alpha1
-kind: Application
-metadata:
-  name: rbac
-  namespace: argocd
-  annotations:
-    argocd.argoproj.io/sync-wave: "9"
-spec:
-  project: homelab
-  source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-    targetRevision: main
-    path: k8s/infra/rbac
-  destination:
-    server: https://kubernetes.default.svc
-    # No namespace: cluster-scoped resources (ClusterRoleBinding, etc.)
-    # Namespace is set per-resource in kustomization
-  syncPolicy:
-    automated:
-      prune: true
-      selfHeal: true

k8s/argocd/apps/60-kmsvc-manage.yaml

@@ -12,7 +12,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/kmsvc-manage.git
+    repoURL: https://forgejo.riotpiao.com/rock/kmsvc-manage.git
     targetRevision: main
     path: k8s/argocd/apps
     directory:

k8s/argocd/apps/70-poimen.yaml

@@ -1,34 +1,27 @@
+# Poimen project collection — manages poimen-memory, poimen-workflows, and poiman
+# Each repo tracks its own main branch (no prod branch). Poiman is the primary
+# orchestrator with k8s/argocd/ containing the AppProject and deployment structure.
+#
+# CI: All three repos trigger on main branch pushes (no image builds yet).
+# Future: Add build workflows for poiman once container runtime needs are clear.
 apiVersion: argoproj.io/v1alpha1
 kind: Application
 metadata:
-  name: poimen
+  name: poimen-root
   namespace: argocd
+  labels:
+    app.kubernetes.io/name: poimen
+    app.kubernetes.io/component: orchestrator
   annotations:
     argocd.argoproj.io/sync-wave: "7"
-    # Image Updater: auto-update on new image push (SHA tag filter)
-    argocd-image-updater.argoproj.io/image-list: |
-      memory=forgejo.riotpiao.com/rock/poimen-memory
-      workflows=forgejo.riotpiao.com/rock/poimen-workflows
-      frontend=forgejo.riotpiao.com/rock/poimen-frontend
-    argocd-image-updater.argoproj.io/memory.update-strategy: digest
-    argocd-image-updater.argoproj.io/memory.allow-tags: regexp:^latest$
-    argocd-image-updater.argoproj.io/workflows.update-strategy: digest
-    argocd-image-updater.argoproj.io/workflows.allow-tags: regexp:^latest$
-    argocd-image-updater.argoproj.io/frontend.update-strategy: digest
-    argocd-image-updater.argoproj.io/frontend.allow-tags: regexp:^latest$
-    argocd-image-updater.argoproj.io/write-back-method: argocd
 spec:
   project: homelab
-  sources:
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/poimen-memory.git
-      targetRevision: main
-      path: k8s/argocd
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/poimen-workflows.git
-      targetRevision: main
-      path: k8s/argocd
-    - repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/poimen-frontend.git
-      targetRevision: main
-      path: k8s/argocd
+  source:
+    repoURL: https://forgejo.riotpiao.com/rock/poimen.git
+    targetRevision: main
+    path: k8s/argocd
+    directory:
+      recurse: false
   destination:
     server: https://kubernetes.default.svc
     namespace: poimen

k8s/argocd/projects/homelab-project.yaml

@@ -16,15 +16,9 @@ spec:
     - https://github.com/Riotpiaole/Poimen-memory.git
     - https://github.com/Riotpiaole/Poimen-workflows.git
     - https://github.com/Riotpiaole/poimen*.git
-    # In-cluster Forgejo repos — explicit allowlist (no wildcard)
-    - https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
-    - https://forgejo.riotpiao.com/riotpiao-poimen/homelab-frontend.git
-    - https://forgejo.riotpiao.com/riotpiao-poimen/kmsvc-manage.git
-    - https://forgejo.riotpiao.com/riotpiao-poimen/poimen.git
-    - https://forgejo.riotpiao.com/riotpiao-poimen/poimen-memory.git
-    - https://forgejo.riotpiao.com/riotpiao-poimen/poimen-workflows.git
-    - https://forgejo.riotpiao.com/riotpiao-poimen/poimen-frontend.git
-    - https://forgejo.riotpiao.com/rock/riotpiao.com.git
+    # In-cluster Forgejo wildcard — all rock/* repos can be onboarded without
+    # touching this AppProject. Enabled by Stage 1 (A1). Includes poimen-* repos.
+    - https://forgejo.riotpiao.com/rock/*
     # Public Helm chart repos referenced by k8s/argocd/apps/* and bootstrap/*
     - https://cloudnative-pg.github.io/charts
     - https://dl.gitea.com/charts/
@@ -43,8 +37,6 @@ spec:
     - https://charts.jetstack.io
     - https://kubernetes.github.io/ingress-nginx
     - https://stakater.github.io/stakater-charts
-    # ArgoCD ecosystem charts
-    - https://argoproj.github.io/argo-helm
   destinations:
     - server: https://kubernetes.default.svc
       namespace: "*"

k8s/argocd/root/homelab-root.yaml

@@ -12,7 +12,7 @@ metadata:
 spec:
   project: homelab
   source:
-    repoURL: https://forgejo.riotpiao.com/riotpiao-poimen/homelab.git
+    repoURL: https://github.com/Riotpiaole/riotpiao.homelab.com.git
     targetRevision: main
     path: k8s/argocd/apps
     directory:
@@ -26,4 +26,3 @@ spec:
       selfHeal: true
     syncOptions:
       - CreateNamespace=true
-      - ServerSideApply=true

k8s/argocd/secrets/gotify-admin-secrets.enc.yaml

@@ -1,24 +0,0 @@
-apiVersion: ENC[AES256_GCM,data:wR4=,iv:cRBzbvu0eUYCYeKeysua1/P3Meli/rQyj/mIV6VWnPM=,tag:oyTPA/mLYNUX+QDkYLlZyQ==,type:str]
-kind: ENC[AES256_GCM,data:bdKQdadW,iv:F3DQiBI9xhSx87jkS1Hyevo48uUCBjsJSjbScIBznKA=,tag:PFakzzBj5S9F65dxu0L2rg==,type:str]
-metadata:
-    name: ENC[AES256_GCM,data:XHFYrKClPo+IwRbM,iv:ZGuL+cN840Y1bOTC62NhDDcoZpTzDWQ4GfqPJX/QWmI=,tag:hlCVjzvfcxXGOASc3vda/Q==,type:str]
-    namespace: ENC[AES256_GCM,data:0BbhgZBog+1qY/iRJA==,iv:88tiSpEDqIokT5VP/d6bB2+aUyh1kZ7NEHwZWoJW3XU=,tag:CBR01jh2U9h7kNEcK1e1sA==,type:str]
-type: ENC[AES256_GCM,data:Mpv1V73w,iv:BW3r6RpLnwe3XDKwrZySyOjrWUnSwIG5JOoPLzP/5gM=,tag:oyJySL0haOei1m4i+1AJ1g==,type:str]
-stringData:
-    username: ENC[AES256_GCM,data:8xmxLGE=,iv:J/vEvXGoD+ka6FDgnSwDz0fs9IxuJIZW9r7Oyu2qxC8=,tag:dN9jd6NSavMN8+uzvemYWg==,type:str]
-    password: ENC[AES256_GCM,data:vB9PaaUiQZ8FY8pO0cq1fHLi7Gq5t4U=,iv:C0Y1m2SGYP3oTIFoau5JavVmLblj6te/SPMLodIwiZw=,tag:3Ip4YvR4WPzR0bBpTyjytA==,type:str]
-sops:
-    age:
-        - enc: |
-            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRT3JoTnhVVW5SUUJXbTdz
-            dUpLcmtBWWhyaGk3Y1hBM1ArcVI5eHREbmp3CiswUGVmd0NhejZwQ2UvNEdxS3ow
-            L1RweS9Kb2paeStLZ0tLSFdWbVZqQW8KLS0tIHJHT3hiMmxtM0Q1Ym5KOVpLVFpl
-            enR3NmdNdmdwVitTQVJlRHFWcjR0N2sKQw9ZZs+Ji/Zq/feO3qy4DwaCfWgDOQ/z
-            FVVhcCXweN58tb+9fzCJ+pNi/hSmvUkCMbb1+60qBvEehNzOoMRJ5g==
-            -----END AGE ENCRYPTED FILE-----
-          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
-    lastmodified: "2026-09-10T13:31:00Z"
-    mac: ENC[AES256_GCM,data:3HWV/NG0yTbsxY28u41zTRmAPc1kokGb4nCLmAsyq8/uvxcP+evDNyZXYpS93eVdPRfRZ1OqVBzyVGJEnj7JSXQVmlzor9JcWEL2fcpogoLVfJZKTRD6hk6optnBMjj84iQEEOx3A80JrDOdoXsH0pd9bJBABwoUOJwuufbXcIU=,iv:KLZsmAcapQgV08pFhGIvPt5ylsWg3xazAAjPuJYOaXA=,tag:1vKA3fISMIurHzxZ04F3lg==,type:str]
-    unencrypted_suffix: _unencrypted
-    version: 3.13.2

k8s/argocd/secrets/gotify-smtp-secrets.enc.yaml

@@ -1,30 +0,0 @@
-apiVersion: ENC[AES256_GCM,data:xUA=,iv:YH1WYnoXhxbeES3Dmo8ZrMDwFcwCykt6+Zn4YSgN4gc=,tag:t3bIWo/LetlQv4tJ0nJ2Sw==,type:str]
-kind: ENC[AES256_GCM,data:PJVzDTcT,iv:aRxvzLUuCzSDryOjA/v67csL6E2FcCpoK8aD7KNStno=,tag:cQgbowSTTV99nQ41yFOlsA==,type:str]
-metadata:
-    name: ENC[AES256_GCM,data:atkSiP0rjyuWFjs=,iv:l/48X9W9lrkAyw6Q/p39V775L6xbr0tOIFI4vjYrg+E=,tag:LZrQxN5h6T5tehFG8L9KFw==,type:str]
-    namespace: ENC[AES256_GCM,data:fbklNyooBeqsogKbeg==,iv:z2wLgl81q8xfzGGVQ9A4Pkst/zLmuZpyBzHrXzmVfm4=,tag:QKJik4IWc4nrOBINjGCxvQ==,type:str]
-type: ENC[AES256_GCM,data:bfAFBd/7,iv:9/W3X1KefqDAsurIatT6kBuEIQvWVRTq4G9A+9ZfYKs=,tag:Optlb+gZ2udbqIOerqV2zQ==,type:str]
-stringData:
-    #ENC[AES256_GCM,data:RZaw/FNQZOf0F3+XfFmGW+4gjKXNdjrAbMRjBt4hptwk,iv:IEd/+Bsv/h08nKak+bcovNiohC7nNjEQ+QdtIFMUGng=,tag:zsF5ZQ3+5JijF4MU+RhnyA==,type:comment]
-    #ENC[AES256_GCM,data:ux3ONHnuI0uFEihElbLI1rKlrN91Pn0iLLZfmez0+t5d8yhH6OD3L/3oi6yHbdgD8bV90kRK/lw=,iv:7Ed+W/TeOY9Af8bKO0Fu3XOuaK8b0Hw70rwobfmosxM=,tag:Z0RbIOjdDroK0drC6P+Y8A==,type:comment]
-    host: ENC[AES256_GCM,data:YKgas/8hT5ZOGIW7O0c=,iv:WtBz1MFrwd6EgDDS5fMqtcVGIZGNLgbXnmkqlNWoXy8=,tag:+H9ARGPi5XocgZYp4oA3wA==,type:str]
-    port: ENC[AES256_GCM,data:f0hx,iv:HkSnJcCyzyfBTA39BNw+muhHTyrNLzzGxZ7M3bcLVu4=,tag:4+xL1xNyYPaV2b0mwnbQVQ==,type:str]
-    from: ENC[AES256_GCM,data:lqYqXxJuI4CAWPlX+5cltCdxqA==,iv:ZgZYySJXvjuTNIkMaDxynUJ6PFdp4fibqlgoA1vNDKE=,tag:vb/XKl2N1Hl5u7ndZXQvbg==,type:str]
-    user: ENC[AES256_GCM,data:ajljGVbRO//G,iv:E4Q0LGDCd5wnHc+Oq2A9ZhWGbCmkKlhllt922UmppIg=,tag:Cf4ft1JKFi9PaJEfPfq92Q==,type:str]
-    password: ENC[AES256_GCM,data:YNbtoiWLsQGl,iv:FJ3y1lWkqKkM8GVr7FZljAXJ331GkDgy4TenAbdIY3w=,tag:KIvhv+RL49ZMwWhYrU0SPA==,type:str]
-    notify-email: ENC[AES256_GCM,data:s5wRkDuaiflCyu4IyAnaocVPA3I=,iv:dK5PsweYa3JotYcDcz74DPNDXmhW5ngNEbY6JuG1ctg=,tag:/Dt2UoCtfau6qv0G7VzmrQ==,type:str]
-sops:
-    age:
-        - enc: |
-            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBLQlJVekcwb01ESnhxSE9m
-            MzliVmlaYUttZVdiMDVtMlQ2L29LMXBWRzBZCkxRMi9RcE5LMzQ1MHFqUUxLaFZo
-            MUZ2UTg5eUNsMFVmenlhL2F3NDd0akkKLS0tIGFwbUUrSGFUTDFVOWlxM3V1bVdy
-            RUVVNDFMRHZCSWptdSt1bFVoN2JPWVkKXMlrJNPbm7Sj9vd/H3o6iqPRLy4LyjVe
-            xJyh9vn9RCwS9Jr6BN2Q9P2UZfTGwvIYc6zdmV+b2uRwvi4t4lMsBA==
-            -----END AGE ENCRYPTED FILE-----
-          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
-    lastmodified: "2026-09-10T13:31:00Z"
-    mac: ENC[AES256_GCM,data:g1YjOANlnvXErPqtDdDjRB9yTUiGEhX8QDkmf1aLfQBwoXbGUpoPIZibF8A4cAL1yUSZ2rD4l9TIe9nK8U3oF4WiKXf9u0bFK2g//bOv2A5JPewwmzoazkjIDXcJ/76nOQ5c+v8/r5tUG4c0SyzBH/gFP0D0+P5Yo3SEXjTxj2w=,iv:gW+AUwwAWkXZ+aLBXjjNDZw8VEraMpSE16RJGQc/L5g=,tag:DM/DUtil/7yLhfGYQXKFGA==,type:str]
-    unencrypted_suffix: _unencrypted
-    version: 3.13.2

k8s/argocd/secrets/gotify-tokens-secrets.enc.yaml

@@ -1,27 +0,0 @@
-apiVersion: ENC[AES256_GCM,data:YE0=,iv:s3yNqcBn7/DKUQNgbGGwVmlM1HzxYGLBKyB6Y2ii2WE=,tag:77KxAjOFU3G0fSfrgudKkg==,type:str]
-kind: ENC[AES256_GCM,data:PFW4VV9T,iv:n8gzMjE5C2TAfUTpp/8ex64B+hWUGG1K+2oQ4deN03Q=,tag:18Xqb6Di5izMHmzR5EU+QA==,type:str]
-metadata:
-    name: ENC[AES256_GCM,data:Xhg1fQrD4itrbvDW0g==,iv:/THWSXAThb9fj+mm3wcxqzSdzdt7nE06+xOpkCxyImE=,tag:UChokr06VkbDZcFZDcUY2A==,type:str]
-    namespace: ENC[AES256_GCM,data:r25U5MuuzDd8JJ2YjQ==,iv:uDUcS4ZTpZe8HmZMArzkdu7LV5GUoLSP2wIs5HB1gv0=,tag:fci7j30hoxnVKJwPFNfd1Q==,type:str]
-type: ENC[AES256_GCM,data:agEG8Fu8,iv:ckYX0buX8md1CWHXfxbAXQJLzoTxTJI16nlQ9LSzYi0=,tag:4tZWf3REbGOmmBiT14+dew==,type:str]
-stringData:
-    #ENC[AES256_GCM,data:ehjcsmz1R0i/n31f8M0IIUT4KDBwzz6f5ds=,iv:j5swFAKpC67ZvGioiCCC1D651LxUl9gME8GqK5Uc+ys=,tag:BUJ8k9LHs8NAPPZAwPuifA==,type:comment]
-    #ENC[AES256_GCM,data:ndAB00yun636VHDMonqTghO/jCWodNd/hmdbm1QmCvOAi4FvTLl8bY3wYR+ZtlkSQYvFNqH798MJixv5OZwxBj5H,iv:pS+7B60jaS5jhqDRw2LbBFv7mD1HO6+hjjv+iNpenXc=,tag:NU6+gxCHTGxqeDMfvkwJWw==,type:comment]
-    #ENC[AES256_GCM,data:T3mhSm/5q7JTSXNLrjhpP5GhqA7nXz8uAhJcEV1RDctAX0Dyfq8Qn4bNFO51ibwTETx4SnunPuFZVs++AvRnsA==,iv:oRGM8N4iEcwBrMzoEXQAeKqCKzUq+jXTMVq9W2l6oh4=,tag:GrrbC/tkWpA5kp5UYCyRBA==,type:comment]
-    app-token: ENC[AES256_GCM,data:jw+AqbsxyoYRvNrUDrq+GNq/TNfweFCs,iv:67vSSgMZCMV0GG37ZxUxHAWZqOOGMYCmo3J43WgLyNk=,tag:xlrnip4XomXBbMmooW9FKg==,type:str]
-    client-token: ENC[AES256_GCM,data:gsxeCqKh4e+DFjpn9jM5GfemAdBf8dSv,iv:l2bBMdxQUil8jU9XDjXGn3EtvFVrK5ibXDCeGaPbYTY=,tag:ELf5S6cdNQJ84Es5upu1IQ==,type:str]
-sops:
-    age:
-        - enc: |
-            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAza3YrRW1VTVNSOGlMK21a
-            bDhzQlEyZDNnZTZrSVNnTUEzU1hSdnM1ZFVzCjdBSUlmWG5EcnlzbSs5bXdDaGkx
-            ME1nMnRqQzF1Vkc3b3FXSUVHT1g0ZDgKLS0tIHpMUytEMjdLQ2E0Unp2di9KS3JQ
-            eHBraDk1clJyanhLY2dGM0tESmJIUFkKHTl3y9uQiEofOFD8j2vH3YK/CVzlq11w
-            GfShIji1yCvvowKGzYYhsQK0UM0FzhzBv0GFMYWQCBq8pGdoPVmO5g==
-            -----END AGE ENCRYPTED FILE-----
-          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
-    lastmodified: "2026-09-10T13:31:00Z"
-    mac: ENC[AES256_GCM,data:JhnO0V6cd2QVY/VhwHAJ5J75GeVlOVHBfVBTZstkj/IvpkAcwS/JE2b6jXiCwEC4n/vdA8DJ074noysUBRxh4Y7O4NKuvWcTniQKgqULNL1Hzgj3NdUkcQlWT+Z0HQ7FlvFH97VVXVfasU+CLHG48ShT2fDSj8JusEllb9CwSvg=,iv:PZo2krxvJc1TLJbvx+S9ClthoBe4w7WigUkq7dg0gNk=,tag:2IF5g/WTRP4XdnCZzDbiIA==,type:str]
-    unencrypted_suffix: _unencrypted
-    version: 3.13.2

k8s/argocd/secrets/kustomization.yaml

@@ -1,10 +1,8 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
-
-# Disable hash suffix for all generated secrets (stable names)
-generatorOptions:
-  disableNameSuffixHash: true
-
-# SOPS-encrypted secrets via ksops generator
+# All homelab SOPS-encrypted Secrets, decrypted in-line via the ksops generator.
+# Each *.enc.yaml carries its own metadata.namespace, so no namespace transformer
+# here (that would rewrite every Secret into one namespace). Renders exactly the
+# Secret objects — replaces the old argocd-cmp-cm SOPS plugin.
 generators:
   - secret-generator.yaml

k8s/argocd/secrets/portfolio-secrets.enc.env

@@ -1,2 +0,0 @@
-FORGEJO_TOKEN=273fdcffabbcbb5a191e8289c73d106063acefc6
-LLM_API_TOKEN=s3VksXyw2z3sGbegnwjMDFnJ6CtNRd1a5CcnE5A4ET77toCcykNdunk6Oa2J

k8s/argocd/secrets/portfolio-secrets.enc.yaml

@@ -1,23 +0,0 @@
-apiVersion: ENC[AES256_GCM,data:bnY=,iv:Fuc3aqncHQ+L16o7eLarPbOECD3o8Mk5c2r9pQBpy70=,tag:JPfEnbNb3wZXPdXafnJDqw==,type:str]
-kind: ENC[AES256_GCM,data:WFlmi4Yg,iv:Zq/KQbgNcBVoo8ZsQ2H79ygyc8Dtkgxh4fCpEExfwSg=,tag:cWHP6Y5V+nZP2tFMJrOB8A==,type:str]
-metadata:
-    name: ENC[AES256_GCM,data:iCXbhwvg3Zq6YL/4j0wAy7Y=,iv:8s+d/8lDVEL7bGdIF+GOtAxapKnmx8JTjLSO04hXF5I=,tag:LjzX40DjK+uRZPCXsMlmwQ==,type:str]
-    namespace: ENC[AES256_GCM,data:HRMdZdCbxORQ,iv:MvaIWoKWjJRA7/fce0KtXRkFH/7cn0OuIg2QwHEdQzM=,tag:NqztiGyfU3BaopWBKhx2eg==,type:str]
-type: ENC[AES256_GCM,data:myBW86Za,iv:3x9ys5UzVhAuX8gvZO67B1e+Orw4Aqasv/lHBgUV0b4=,tag:yl18P6SaxVLJidwmRtd4aQ==,type:str]
-stringData:
-    FORGEJO_TOKEN: ENC[AES256_GCM,data:SUoBpNKOItyNGY01EhKNlPH0fyN4N7g6bfU2jsGogpCMhP7NuRipoA==,iv:h77RtmYZjXxiHYw1pHynQHuVX1+yJDGHwsXLf+DbUYA=,tag:GHzoDNHP1GGqlN5eT/N7TQ==,type:str]
-sops:
-    age:
-        - enc: |
-            -----BEGIN AGE ENCRYPTED FILE-----
-            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBzTVBsekl3TGgzQVRMUU9m
-            cTduS2NoZW5uZFNNMG13cFY2cGVsTnlXaXhrCjJjbzhLdHZ4ZWpUV3J0cDQ0eVlM
-            WDNxdzVoQ2ZzcGJSbTU3RVorcnczNVkKLS0tIFdtQTE4Umk2TDBzUmdKOXNkbjFi
-            Vk5vK2VuUHVsb3FQL21vcGU1UW5CT1kKFM8vVjji3Cg9dvfTr4Hx7BJC8JH5ovef
-            Dj6zkofhsNWPgP9T+mnQakj+C0RKmHOMJqfWP7vwCBkZoNosIJVlMw==
-            -----END AGE ENCRYPTED FILE-----
-          recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
-    lastmodified: "2026-09-01T05:32:42Z"
-    mac: ENC[AES256_GCM,data:it24T9y9ixXo2aiL37k93vKFR+SRjjuI9DQdv0sWYtTogWnc7+uXBY4Zip/ouWyCse1muKKAGuek5c0XVrvSw4an9VkaXFczeunaZb6MOyVbVOkmJr+5xZFpZGjYcSkrhaWcVheedZ3iIFU5UWI7BBn/qQCf+HJ483cJqwtrV34=,iv:WprlWJdsMBNjqaA0O3ekfXMUpX5gC6OLYortQXYdTS4=,tag:rGqSSRTYTv2VR6AcRokO0A==,type:str]
-    unencrypted_suffix: _unencrypted
-    version: 3.13.2

k8s/argocd/secrets/secret-generator.yaml

@@ -26,7 +26,3 @@ files:
   - paperless-secrets.enc.yaml
   - vault-secrets.enc.yaml
   - vault-unseal-keys.enc.yaml
-  - portfolio-secrets.enc.yaml
-  - gotify-admin-secrets.enc.yaml
-  - gotify-tokens-secrets.enc.yaml
-  - gotify-smtp-secrets.enc.yaml

k8s/argocd/secrets/vault-unseal-keys.enc.yaml

@@ -5,9 +5,9 @@ metadata:
     namespace: ENC[AES256_GCM,data:wK6m,iv:KtA31Bo8aGE1HU8H9KWbMwt2NfywWfy77G/LaReaI1E=,tag:V9SSlPawRQw0x2utFNN1aw==,type:str]
 type: ENC[AES256_GCM,data:6UPZ1ZTR,iv:FxN1ebrlJ4IO3eDGEYSvktSpMgAebcl0DO1WHh5O0+0=,tag:2bZN4zdbHzx0oR+bTJPPJg==,type:str]
 stringData:
-    key1: ENC[AES256_GCM,data:ke5EfsOsZHBJyAQoFfwhhWCQJQgnwcrBqL4CzPIfkx7bi9S7kWgWySdxcXQ=,iv:eYKxG8k+hizp2t2i/YMR2lQNJQFV+A21YyWnDc+kJ9w=,tag:Wa1Lx3EKvfpYfnLrBOWSIg==,type:str]
-    key2: ENC[AES256_GCM,data:JxiXgLKvDe5oiHlwIL/Cj8txHc7fVQ5VzBcQMU/ro9TSOcTGSe7Z3omQzwY=,iv:x5arcY2dey+npMpUxjdUPV+t94LEaMOXq3iOar88eT8=,tag:E/Z5zEDy87B1IAWkdjdNEg==,type:str]
-    key3: ENC[AES256_GCM,data:AsxyyBTHzt+SLru6ayi1UbgPPFmdIvowIDdcfc7N03Xt8V/KaJW3kbXExG8=,iv:lA+bERIBq4+bxx08ZZjP5tMHC9JQlGPW1EMCwEixSeg=,tag:xkuTAU3KTidFuhNMXkVFhA==,type:str]
+    key1: ENC[AES256_GCM,data:dV2HOh7W1Pl0QDJaGvtEKpBppybQMiK5kxzyk05zkA352ZmJvE8Ppm5yWKk=,iv:grQ8v2o/LHpJnIZonjtgTHKcLUQIH8xl1FtWtEs4rEs=,tag:lANcccYT6Ouo/0IcoLw4uA==,type:str]
+    key2: ENC[AES256_GCM,data:YSdpYL8h56PfUrvMFhBXhmBG2en0toLKEpYlJqwjAk/vI0jm7gq/TqGn034=,iv:GxthkNLhm3qHxjkZetiYl58Qa/K7G2ib6E+LWK16H8Q=,tag:q79UUoh6uCcMZJ0U18Ireg==,type:str]
+    key3: ENC[AES256_GCM,data:sO7Lao9qkmIOdARL+6FBcLo+e4z8LzMEzNrwcy5hv9uRuxppwAXrP0MO1vw=,iv:wSWe9h1HovRhV5YCZrmvcB2VEfRYQ+gasGO3uh8IpmQ=,tag:Bc5yaROQDTz1AhVnFDm/BA==,type:str]
 sops:
     age:
         - enc: |
@@ -19,7 +19,7 @@ sops:
             7RltNZF2SCxjIv5C2pqf3CgmRBaQGWgMybRRH5gdB87PLBKkPL3+HQ==
             -----END AGE ENCRYPTED FILE-----
           recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
-    lastmodified: "2026-08-26T23:33:09Z"
-    mac: ENC[AES256_GCM,data:kr8XuOqXyhfrC84yBot5aoVQZZhQd9xYMUEs0XDwkmCgtG0iobADYH5XWPh72xSdWEtwxkZJL7r0UEyEhU9zjlUA5qHupOSBBt1SgZ+zstMvaOqUnlNn//p/DIJBpsiT/qmx64NpTLAiz6lm0796MozIMr8PTX+ubGLHi9tnUiY=,iv:jwP0MLCT7nG6m+nZeqNip9q3BcScpcDXmInnY95DicY=,tag:Cgg2IUZARkaK1gF2qUIthg==,type:str]
+    lastmodified: "2026-08-12T20:52:24Z"
+    mac: ENC[AES256_GCM,data:kLjoBWmJ2bZ13EbuzgppahHKmCY/xtCi1R7xCEUbCP4FQfiVaC4qAbVftE4lTNlXa2mqCKsGPYZ/A3HV/4+a+it/pg0a+rj7E7DszhieWbZufHMJs+w4/Le8l8wFFE5aNW0wdvGTJZ0n4HBOdAkE1qN9ReaiaQgr0rk5ggOPG9g=,iv:MiDVUgcoKrP/gj4keqomo6OB12dmz9VoesTiHQTnBFM=,tag:cLK67Pym4BuvXLF1RUA8Gw==,type:str]
     unencrypted_suffix: _unencrypted
     version: 3.13.2

k8s/bootstrap/ingress/ingress.yaml

@@ -323,4 +323,3 @@ spec:
                 name: homarr
                 port:
                   number: 7575
-

k8s/bootstrap/kyverno/kustomization.yaml

@@ -1,6 +0,0 @@
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
-namespace: kyverno
-
-resources:
-  - policies.yaml

k8s/bootstrap/kyverno/kyverno-values.yaml

@@ -1,42 +0,0 @@
-# Kyverno: Policy engine for Kubernetes image scanning, Pod security, and admission control
-# Scan all images, enforce baseline Pod Security Standard, prevent privilege escalation
-
-replicaCount: 1
-
-image:
-  registry: ghcr.io
-  repository: kyverno/kyverno
-  tag: "v1.14.0"
-
-config:
-  # Webhook timeout for policy evaluation. Increase if scanning takes longer.
-  webhookTimeoutSeconds: 30
-  # Failure policy: fail-open (audit/log) vs fail-closed (reject on error)
-  failurePolicy: fail
-  # Resource limits for webhook
-  webhookAnnotations:
-    rules: "allow"
-
-# Pod security via Kyverno instead of Pod Security Policies (deprecated)
-# Enforces baseline restrictions cluster-wide, with exceptions for privileged namespaces
-podSecurityContext:
-  runAsNonRoot: true
-  runAsUser: 1000
-
-rbac:
-  create: true
-
-resources:
-  requests:
-    memory: "256Mi"
-    cpu: "100m"
-  limits:
-    memory: "512Mi"
-    cpu: "500m"
-
-# Webhook configuration
-webhook:
-  timeoutSeconds: 30
-  # Failure policy: "Fail" (reject on error) or "Ignore" (audit-only)
-  # Set to "Ignore" for initial testing, then change to "Fail"
-  failurePolicy: ignore

k8s/bootstrap/kyverno/policies.yaml

@@ -1,204 +0,0 @@
-# Kyverno ClusterPolicies: Image scanning, Pod security, and admission control
----
-# Policy 1: Require non-root containers
-apiVersion: kyverno.io/v1
-kind: ClusterPolicy
-metadata:
-  name: require-non-root
-  namespace: kyverno
-spec:
-  validationFailureAction: audit  # audit first, then change to enforce
-  rules:
-    - name: check-runAsNonRoot
-      match:
-        any:
-          - resources:
-              kinds:
-                - Pod
-              selector:
-                matchLabels:
-                  pod-security.kubernetes.io/enforce: "!privileged"
-      validate:
-        message: "Container must not run as root"
-        pattern:
-          spec:
-            containers:
-              - securityContext:
-                  runAsNonRoot: true
----
-# Policy 2: Drop all Linux capabilities, add only required ones
-apiVersion: kyverno.io/v1
-kind: ClusterPolicy
-metadata:
-  name: require-dropped-caps
-  namespace: kyverno
-spec:
-  validationFailureAction: audit
-  rules:
-    - name: drop-all-capabilities
-      match:
-        any:
-          - resources:
-              kinds:
-                - Pod
-              selector:
-                matchLabels:
-                  pod-security.kubernetes.io/enforce: "!privileged"
-      validate:
-        message: "All Linux capabilities must be dropped"
-        pattern:
-          spec:
-            containers:
-              - securityContext:
-                  capabilities:
-                    drop:
-                      - ALL
----
-# Policy 3: Require image tags (no 'latest')
-apiVersion: kyverno.io/v1
-kind: ClusterPolicy
-metadata:
-  name: disallow-latest-tag
-  namespace: kyverno
-spec:
-  validationFailureAction: audit  # Change to enforce after testing
-  rules:
-    - name: disallow-latest
-      match:
-        any:
-          - resources:
-              kinds:
-                - Pod
-                - Deployment
-                - StatefulSet
-                - DaemonSet
-                - Job
-      validate:
-        message: "Image tag 'latest' is not allowed. Use explicit version tags."
-        pattern:
-          spec:
-            =(template):
-              spec:
-                containers:
-                  - image: "!*:latest"
-                =(initContainers):
-                  - image: "!*:latest"
----
-# Policy 4: Restrict images to trusted registries
-apiVersion: kyverno.io/v1
-kind: ClusterPolicy
-metadata:
-  name: restrict-registries
-  namespace: kyverno
-spec:
-  validationFailureAction: audit
-  rules:
-    - name: trusted-registries
-      match:
-        any:
-          - resources:
-              kinds:
-                - Pod
-                - Deployment
-                - StatefulSet
-                - DaemonSet
-                - Job
-              selector:
-                matchLabels:
-                  pod-security.kubernetes.io/enforce: "!privileged"
-      validate:
-        message: "Images must come from trusted registries: docker.io, ghcr.io, quay.io, k8s.gcr.io, registry.k8s.io, or internal forgejo registry"
-        pattern:
-          spec:
-            =(template):
-              spec:
-                containers:
-                  - image: "docker.io/* | ghcr.io/* | quay.io/* | k8s.gcr.io/* | registry.k8s.io/* | forgejo.riotpiao.com/* | *"
----
-# Policy 5: Require read-only root filesystem (audit only, exceptions for apps that need writes)
-apiVersion: kyverno.io/v1
-kind: ClusterPolicy
-metadata:
-  name: require-readonly-filesystem
-  namespace: kyverno
-spec:
-  validationFailureAction: audit
-  rules:
-    - name: check-readOnlyRootFilesystem
-      match:
-        any:
-          - resources:
-              kinds:
-                - Pod
-              selector:
-                matchLabels:
-                  pod-security.kubernetes.io/enforce: "!privileged"
-      validate:
-        message: "Root filesystem should be read-only for defense-in-depth"
-        pattern:
-          spec:
-            containers:
-              - securityContext:
-                  readOnlyRootFilesystem: true
----
-# Policy 6: Require resource requests and limits (prevent resource starvation)
-apiVersion: kyverno.io/v1
-kind: ClusterPolicy
-metadata:
-  name: require-resource-limits
-  namespace: kyverno
-spec:
-  validationFailureAction: audit
-  rules:
-    - name: check-resources
-      match:
-        any:
-          - resources:
-              kinds:
-                - Pod
-                - Deployment
-                - StatefulSet
-                - DaemonSet
-              excludeResources:
-                namespaceSelector:
-                  matchLabels:
-                    kubernetes.io/metadata.name: "kyverno|kube-system|kube-node-lease"
-      validate:
-        message: "CPU and memory requests and limits are required"
-        pattern:
-          spec:
-            =(template):
-              spec:
-                containers:
-                  - resources:
-                      requests:
-                        memory: "?*"
-                        cpu: "?*"
-                      limits:
-                        memory: "?*"
-                        cpu: "?*"
----
-# Policy 7: Require securityContext on all containers
-apiVersion: kyverno.io/v1
-kind: ClusterPolicy
-metadata:
-  name: require-security-context
-  namespace: kyverno
-spec:
-  validationFailureAction: audit
-  rules:
-    - name: check-securityContext
-      match:
-        any:
-          - resources:
-              kinds:
-                - Pod
-              selector:
-                matchLabels:
-                  pod-security.kubernetes.io/enforce: "!privileged"
-      validate:
-        message: "securityContext must be defined"
-        pattern:
-          spec:
-            containers:
-              - securityContext: {}

k8s/infra/argocd-image-updater/values.yaml

@@ -1,72 +0,0 @@
-# ArgoCD Image Updater configuration
-# Watches Forgejo registry and updates ArgoCD Applications with new image tags
-
-config:
-  # Registry configuration - Forgejo allows anonymous pulls
-  registries:
-    - name: forgejo
-      api_url: https://forgejo.riotpiao.com
-      prefix: forgejo.riotpiao.com
-      default: true
-      insecure: false
-
-  # Log level
-  logLevel: debug
-
-  # ArgoCD API server
-  argocd:
-    grpcWeb: true
-    serverAddress: argocd-server.argocd.svc.cluster.local
-    insecure: true
-    plaintext: true
-
-  # Git write-back configuration (for multi-source Applications)
-  git:
-    # Commit author for image updates
-    user:
-      name: "ArgoCD Image Updater"
-      email: "[email protected]"
-    # Use SSH keys from ArgoCD's known hosts + credentials
-    # Image Updater inherits ArgoCD's git credentials (mounted via ArgoCD secret)
-
-# Mount ArgoCD's git credentials for write-back
-extraVolumes:
-  - name: argocd-ssh-known-hosts-cm
-    configMap:
-      name: argocd-ssh-known-hosts-cm
-      defaultMode: 0644
-  - name: argocd-gpg-keys-cm
-    configMap:
-      name: argocd-gpg-keys-cm
-      optional: true
-      defaultMode: 0644
-  - name: argocd-gpg-pubring
-    configMap:
-      name: argocd-gpg-pubring-cm
-      optional: true
-      defaultMode: 0644
-
-extraVolumeMounts:
-  - name: argocd-ssh-known-hosts-cm
-    mountPath: /etc/ssh/ssh_known_hosts.d/argocd-ssh-known-hosts
-    subPath: ssh_known_hosts
-  - name: argocd-gpg-keys-cm
-    mountPath: /etc/gpg/source
-  - name: argocd-gpg-pubring
-    mountPath: /etc/gpg/pubring
-
-# Extra environment variables
-extraEnv:
-  - name: ARGOCD_GRPC_WEB
-    value: "true"
-  - name: GIT_SSH_KNOWN_HOSTS_CONFIG_MAP_ENABLED
-    value: "true"
-
-# Resources
-resources:
-  requests:
-    cpu: 50m
-    memory: 64Mi
-  limits:
-    cpu: 200m
-    memory: 128Mi

k8s/infra/cluster-maintenance/stale-job-cleanup.yaml

@@ -1,137 +0,0 @@
-# Cluster-wide cleanup of stale failed/completed Jobs and Pods.
-# Runs daily at 04:00 UTC. Deletes:
-#   - Failed Jobs older than 24h (any namespace)
-#   - Completed Jobs older than 72h with no owning CronJob
-#   - Orphan pods in Error/Failed/Evicted state older than 1h
-#
-# CronJob-owned Jobs are managed by failedJobsHistoryLimit/successfulJobsHistoryLimit,
-# but standalone Jobs (helm hooks, one-off runs, longhorn maintenance) have no TTL
-# and linger forever.
-
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: stale-job-cleanup
-  namespace: kube-system
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRole
-metadata:
-  name: stale-job-cleanup
-rules:
-  - apiGroups: ["batch"]
-    resources: ["jobs"]
-    verbs: ["get", "list", "delete"]
-  - apiGroups: [""]
-    resources: ["pods"]
-    verbs: ["get", "list", "delete"]
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRoleBinding
-metadata:
-  name: stale-job-cleanup
-subjects:
-  - kind: ServiceAccount
-    name: stale-job-cleanup
-    namespace: kube-system
-roleRef:
-  kind: ClusterRole
-  name: stale-job-cleanup
-  apiGroup: rbac.authorization.k8s.io
----
-apiVersion: batch/v1
-kind: CronJob
-metadata:
-  name: stale-job-cleanup
-  namespace: kube-system
-  labels:
-    app: stale-job-cleanup
-spec:
-  schedule: "0 4 * * *"
-  concurrencyPolicy: Forbid
-  successfulJobsHistoryLimit: 3
-  failedJobsHistoryLimit: 3
-  jobTemplate:
-    spec:
-      ttlSecondsAfterFinished: 86400  # self-cleanup after 24h
-      backoffLimit: 1
-      activeDeadlineSeconds: 300
-      template:
-        spec:
-          serviceAccountName: stale-job-cleanup
-          restartPolicy: Never
-          tolerations:
-            - key: node-role.kubernetes.io/control-plane
-              operator: Exists
-              effect: NoSchedule
-          containers:
-            - name: cleanup
-              image: alpine/k8s:1.31.0
-              command:
-                - sh
-                - -c
-                - |
-                  set -e
-                  NOW=$(date +%s)
-
-                  echo "=== Cleaning failed Jobs older than 24h ==="
-                  kubectl get jobs --all-namespaces -o json | \
-                    jq -r '.items[] |
-                      select(.status.conditions[]?.type == "Failed") |
-                      select(.status.completionTime or .status.startTime) |
-                      "\(.metadata.namespace) \(.metadata.name) \(.status.startTime // .status.completionTime // .metadata.creationTimestamp)"' | \
-                  while read -r NS NAME TS; do
-                    JOB_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0)
-                    AGE_H=$(( (NOW - JOB_EPOCH) / 3600 ))
-                    if [ "$AGE_H" -ge 24 ]; then
-                      echo "[delete] $NS/$NAME (failed ${AGE_H}h ago)"
-                      kubectl delete job "$NAME" -n "$NS" --cascade=foreground 2>/dev/null || true
-                    fi
-                  done
-
-                  echo ""
-                  echo "=== Cleaning completed standalone Jobs older than 72h ==="
-                  kubectl get jobs --all-namespaces -o json | \
-                    jq -r '.items[] |
-                      select(.status.succeeded >= 1) |
-                      select((.metadata.ownerReferences // []) | length == 0) |
-                      "\(.metadata.namespace) \(.metadata.name) \(.status.completionTime // .metadata.creationTimestamp)"' | \
-                  while read -r NS NAME TS; do
-                    JOB_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0)
-                    AGE_H=$(( (NOW - JOB_EPOCH) / 3600 ))
-                    if [ "$AGE_H" -ge 72 ]; then
-                      echo "[delete] $NS/$NAME (completed ${AGE_H}h ago, no owner)"
-                      kubectl delete job "$NAME" -n "$NS" --cascade=foreground 2>/dev/null || true
-                    fi
-                  done
-
-                  echo ""
-                  echo "=== Cleaning orphan Error/Failed/Evicted pods older than 1h ==="
-                  # Evicted pods show as Failed with reason Evicted
-                  kubectl get pods --all-namespaces -o json | \
-                    jq -r '.items[] |
-                      select(
-                        .status.phase == "Failed" or
-                        (.status.reason // "") == "Evicted" or
-                        (.status.containerStatuses // [] | any(.state.terminated.reason == "Error"))
-                      ) |
-                      select((.metadata.ownerReferences // []) | all(.kind != "Job")) |
-                      "\(.metadata.namespace) \(.metadata.name) \(.metadata.creationTimestamp)"' | \
-                  while read -r NS NAME TS; do
-                    POD_EPOCH=$(date -d "$TS" +%s 2>/dev/null || echo 0)
-                    AGE_H=$(( (NOW - POD_EPOCH) / 3600 ))
-                    if [ "$AGE_H" -ge 1 ]; then
-                      echo "[delete] $NS/$NAME (error/evicted ${AGE_H}h ago)"
-                      kubectl delete pod "$NAME" -n "$NS" --force 2>/dev/null || true
-                    fi
-                  done
-
-                  echo ""
-                  echo "Cleanup complete"
-              resources:
-                requests:
-                  cpu: 50m
-                  memory: 64Mi
-                limits:
-                  cpu: 250m
-                  memory: 128Mi

k8s/infra/databases/kustomization.yaml

@@ -10,4 +10,3 @@ resources:
   - temporal-db.yaml
   - memory-db.yaml
   - paperless-db.yaml
-  - obsidian-vault-pvc.yaml

k8s/infra/databases/namespaces.yaml

@@ -16,11 +16,3 @@ apiVersion: v1
 kind: Namespace
 metadata:
   name: paperless
----
-# Needed here (not just immich's own CreateNamespace=true at wave 8) because
-# k8s/infra/iam's PostSync job (wave 3) has a RoleBinding targeting this
-# namespace - same ordering reason as paperless above.
-apiVersion: v1
-kind: Namespace
-metadata:
-  name: immich

k8s/infra/databases/obsidian-vault-pvc.yaml

@@ -1,18 +0,0 @@
----
-# Obsidian vault PVC — shared storage for REST API + UI pods
-# ReadWriteMany so both obsidian-server and obsidian-ui can mount simultaneously
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
-  name: obsidian-vault
-  namespace: poimen
-  labels:
-    app.kubernetes.io/name: obsidian-server
-    app.kubernetes.io/part-of: poimen-memory
-spec:
-  accessModes:
-    - ReadWriteMany
-  storageClassName: longhorn
-  resources:
-    requests:
-      storage: 10Gi

k8s/infra/databases/paperless-db.yaml

@@ -11,7 +11,7 @@ metadata:
   annotations:
     argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
 spec:
-  instances: 3
+  instances: 2
   imageName: ghcr.io/cloudnative-pg/postgresql:16.2
   bootstrap:
     initdb:

k8s/infra/forgejo-runner/templates/configmap.yaml

@@ -36,4 +36,3 @@ data:
       valid_volumes:
         - /docker-certs/client
       network: host
-      docker_host: automount

k8s/infra/forgejo-runner/templates/deployment.yaml

@@ -34,11 +34,7 @@ spec:
           command: ["sh", "-c"]
           args:
             - |
-              # Always re-register to keep labels in sync with values.yaml.
-              # Without this, changing a runner label requires manually deleting
-              # the PVC or .runner file — not GitOps-friendly.
-              rm -f /data/.runner
-              forgejo-runner register --no-interactive \
+              test -f /data/.runner || forgejo-runner register --no-interactive \
                 --instance {{ .Values.runner.forgejoUrl }} \
                 --token $(RUNNER_TOKEN) \
                 --name {{ .Values.runner.name }} \
@@ -60,18 +56,20 @@ spec:
       containers:
         - name: runner
           image: {{ .Values.runner.image.repository }}:{{ .Values.runner.image.tag }}
-          command: ["sh", "-c", "while ! wget -q -O- http://localhost:2375/_ping >/dev/null 2>&1; do echo 'waiting for dind...'; sleep 2; done; echo 'dind ready'; forgejo-runner daemon --config /etc/forgejo-runner/config.yaml"]
+          command: ["sh", "-c", "forgejo-runner daemon --config /etc/forgejo-runner/config.yaml"]
           workingDir: /data
           env:
             - name: DOCKER_HOST
-              value: tcp://localhost:2375
+              value: tcp://localhost:2376
+            - name: DOCKER_TLS_VERIFY
+              value: "1"
+            - name: DOCKER_CERT_PATH
+              value: /docker-certs/client
           volumeMounts:
             - name: runner-data
               mountPath: /data
             - name: docker-certs
               mountPath: /docker-certs
-            - name: docker-sock
-              mountPath: /run
             - name: homelab-ca
               mountPath: /etc/ssl/certs/homelab-ca.pem
               subPath: ca.crt
@@ -87,12 +85,10 @@ spec:
             privileged: true    # required for DinD; cicd namespace is labelled privileged
           env:
             - name: DOCKER_TLS_CERTDIR
-              value: ""
+              value: /docker-certs
           volumeMounts:
             - name: docker-certs
               mountPath: /docker-certs
-            - name: docker-sock
-              mountPath: /run
             - name: dind-storage
               mountPath: /var/lib/docker
             - name: homelab-ca
@@ -117,8 +113,6 @@ spec:
             claimName: {{ .Release.Name }}-dind
         - name: docker-certs
           emptyDir: {}    # DinD regenerates mTLS certs on each start
-        - name: docker-sock
-          emptyDir: {}    # Shared docker socket between dind and runner
         - name: homelab-ca
           # homelab-ca is a ConfigMap (public CA trust bundle), not a Secret.
           # The volumeMounts use subPath: ca.crt to project the single cert file.

k8s/infra/forgejo-runner/templates/gc-cronjob.yaml

@@ -1,152 +0,0 @@
-{{- if .Values.gc.enabled }}
-# Garbage-collects DinD Docker images/volumes/build-cache and actcache across
-# ALL forgejo-runner pods. Prevents PVC fill-up that breaks CI runs.
-# Only rendered once (enable in default values.yaml, disable in per-runner overrides).
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  name: runner-gc
-  namespace: {{ .Release.Namespace }}
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: Role
-metadata:
-  name: runner-gc
-  namespace: {{ .Release.Namespace }}
-rules:
-  - apiGroups: [""]
-    resources: ["pods"]
-    verbs: ["get", "list"]
-  - apiGroups: [""]
-    resources: ["pods/exec"]
-    verbs: ["create"]
----
-apiVersion: rbac.authorization.k8s.io/v1
-kind: RoleBinding
-metadata:
-  name: runner-gc
-  namespace: {{ .Release.Namespace }}
-subjects:
-  - kind: ServiceAccount
-    name: runner-gc
-    namespace: {{ .Release.Namespace }}
-roleRef:
-  kind: Role
-  name: runner-gc
-  apiGroup: rbac.authorization.k8s.io
----
-apiVersion: batch/v1
-kind: CronJob
-metadata:
-  name: forgejo-runner-gc
-  namespace: {{ .Release.Namespace }}
-  labels:
-    app: forgejo-runner-gc
-spec:
-  schedule: {{ .Values.gc.schedule | quote }}
-  concurrencyPolicy: Forbid
-  successfulJobsHistoryLimit: 3
-  failedJobsHistoryLimit: 3
-  jobTemplate:
-    spec:
-      backoffLimit: 1
-      activeDeadlineSeconds: 900
-      template:
-        spec:
-          serviceAccountName: runner-gc
-          restartPolicy: Never
-          tolerations:
-            - key: node-role.kubernetes.io/control-plane
-              operator: Exists
-              effect: NoSchedule
-          containers:
-            - name: gc
-              image: {{ .Values.gc.image }}
-              command:
-                - sh
-                - -c
-                - |
-                  set -e
-
-                  # Iterate all forgejo-runner pods (golang, rust, node)
-                  PODS=$(kubectl -n {{ .Release.Namespace }} get pod \
-                    -l app -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.labels.app}{"\n"}{end}' \
-                    | grep 'forgejo-runner-' | awk '{print $1}')
-
-                  if [ -z "$PODS" ]; then
-                    echo "no forgejo-runner pods found, skipping"
-                    exit 0
-                  fi
-
-                  for POD in $PODS; do
-                    echo "===== $POD ====="
-
-                    # 1. Docker image prune (DinD sidecar)
-                    echo "[docker] before:"
-                    kubectl -n {{ .Release.Namespace }} exec "$POD" -c dind -- docker system df 2>/dev/null || true
-
-                    echo "[docker] pruning non-latest images older than {{ .Values.gc.pruneAge }}..."
-                    # Keep :latest tagged images, delete all others older than pruneAge.
-                    # docker image prune can't filter by tag, so we list and selectively rmi.
-                    kubectl -n {{ .Release.Namespace }} exec "$POD" -c dind -- \
-                      sh -c '
-                        # Remove dangling (untagged) images older than {{ .Values.gc.pruneAge }}
-                        docker image prune -f --filter "until={{ .Values.gc.pruneAge }}" 2>/dev/null
-
-                        # Remove tagged non-latest images older than {{ .Values.gc.pruneAge }}
-                        CUTOFF=$(date -d "-{{ .Values.gc.pruneAgeHours }} hours" +%s 2>/dev/null || date -v-{{ .Values.gc.pruneAgeHours }}H +%s)
-                        docker images --format "{{"{{"}} .Repository {{"}}"}}:{{"{{"}} .Tag {{"}}"}} {{"{{"}} .CreatedAt {{"}}"}}" | while read -r IMAGE_TAG CREATED_REST; do
-                          TAG=$(echo "$IMAGE_TAG" | rev | cut -d: -f1 | rev)
-                          # Skip latest-tagged images
-                          if [ "$TAG" = "latest" ]; then
-                            echo "[keep] $IMAGE_TAG (latest)"
-                            continue
-                          fi
-                          # Check image age via inspect
-                          CREATED_TS=$(docker inspect --format="{{"{{"}} .Created {{"}}"}}" "$IMAGE_TAG" 2>/dev/null | head -1)
-                          if [ -z "$CREATED_TS" ]; then continue; fi
-                          IMAGE_EPOCH=$(date -d "$CREATED_TS" +%s 2>/dev/null || date -jf "%Y-%m-%dT%H:%M:%S" "$(echo $CREATED_TS | cut -dT -f1-2 | cut -d. -f1)" +%s 2>/dev/null || echo 0)
-                          if [ "$IMAGE_EPOCH" -lt "$CUTOFF" ] 2>/dev/null; then
-                            echo "[delete] $IMAGE_TAG (older than {{ .Values.gc.pruneAge }})"
-                            docker rmi -f "$IMAGE_TAG" 2>/dev/null || true
-                          else
-                            echo "[keep] $IMAGE_TAG (recent)"
-                          fi
-                        done
-                      ' 2>/dev/null || true
-
-                    echo "[docker] pruning build cache unused >{{ .Values.gc.pruneAge }}..."
-                    kubectl -n {{ .Release.Namespace }} exec "$POD" -c dind -- \
-                      docker builder prune -af --filter "until={{ .Values.gc.pruneAge }}" 2>/dev/null || true
-
-                    echo "[docker] pruning dangling volumes..."
-                    kubectl -n {{ .Release.Namespace }} exec "$POD" -c dind -- \
-                      docker volume prune -af 2>/dev/null || true
-
-                    echo "[docker] after:"
-                    kubectl -n {{ .Release.Namespace }} exec "$POD" -c dind -- docker system df 2>/dev/null || true
-
-                    # 2. Actcache cleanup (runner container)
-                    echo "[actcache] cleaning incomplete and stale cache entries..."
-                    kubectl -n {{ .Release.Namespace }} exec "$POD" -c runner -- \
-                      sh -c '
-                        # Delete incomplete/partial cache uploads immediately (tmp dirs)
-                        find /data/.cache/actcache/cache -name "tmp" -type d -exec rm -rf {} + 2>/dev/null || true
-                        # Delete cache entries not accessed in last {{ .Values.gc.actcacheMaxAgeDays }} day(s)
-                        find /data/.cache/actcache/cache -type f -mtime +{{ .Values.gc.actcacheMaxAgeDays }} -delete 2>/dev/null || true
-                        # Clean up empty directories
-                        find /data/.cache/actcache/cache -type d -empty -delete 2>/dev/null || true
-                        echo "actcache size: $(du -sh /data/.cache/actcache/cache 2>/dev/null | cut -f1)"
-                      ' || true
-
-                    echo ""
-                  done
-                  echo "GC complete"
-              resources:
-                requests:
-                  cpu: 50m
-                  memory: 64Mi
-                limits:
-                  cpu: 250m
-                  memory: 128Mi
-{{- end }}

k8s/infra/forgejo-runner/values-node.yaml

@@ -2,15 +2,8 @@
 # runner instance. Only runner.name and runner.labels differ -- everything
 # else (image, dind, persistence, tolerations, nodeSelector) is shared.
 #
-# Label image: node:22-bookworm — Debian, root, apt-get, Node.js, npm, git.
-# Install docker in workflow steps as needed.
+# node:22-bookworm ships Node natively, so unlike the golang/rust instances,
+# jobs on this runner need no "install node" step before actions/checkout.
 runner:
-  image:
-    repository: code.forgejo.org/forgejo/runner
-    tag: "6"
   name: node-runner
   labels: "node:docker://node:22-bookworm"
-
-# GC CronJob renders only from the default (golang) values to avoid duplicates
-gc:
-  enabled: false

k8s/infra/forgejo-runner/values-rust.yaml

@@ -2,17 +2,9 @@
 # runner instance. Only runner.name and runner.labels differ -- everything
 # else (image, dind, persistence, tolerations, nodeSelector) is shared.
 #
-# Label image: rust:1-bookworm — Debian, root, apt-get, Rust, cargo, git.
-# Install Node.js/docker in workflow steps as needed.
+# rust:1.83-bookworm -- verified this tag exists (docker manifest inspect)
+# before pinning it, per this repo's convention of not trusting a tag exists
+# without checking.
 runner:
-  image:
-    repository: code.forgejo.org/forgejo/runner
-    tag: "6"
   name: rust-runner
-  labels: "rust:docker://rust:1-bookworm"
-
-
-
-# GC CronJob renders only from the default (golang) values to avoid duplicates
-gc:
-  enabled: false
+  labels: "rust:docker://rust:1.83-bookworm"

k8s/infra/forgejo-runner/values.yaml

@@ -1,13 +1,20 @@
 runner:
   image:
     repository: code.forgejo.org/forgejo/runner
-    tag: "6"
+    tag: "6"          # pin exact release before apply
   name: golang-runner
-  # Label image is what workflow steps run in (NOT the runner daemon image).
-  # golang:1.26-bookworm: Debian, root, apt-get, Go, git.
-  # TODO: Switch to custom image once build-runner-images.yml pushes images
-  labels: "golang:docker://golang:1.26-bookworm"
+  # Default image is only used when a job's `container:` doesn't override it
+  # (both ci.yaml and build.yaml in homelab-frontend do). Retired the old
+  # "docker" label entirely; every repo this runner serves is Go, so this
+  # instance carries the golang toolchain and its own dind sidecar builds and
+  # pushes that repo's images too -- there is no separate generic runner
+  # anymore.
+  labels: "golang:docker://golang:1.25-bookworm"
+  # In-cluster Service (:3000) — direct, avoids the ingress/public-hostname hop
+  # (the public URL is :443 which forgejo doesn't serve; runner got i/o timeout).
   forgejoUrl: http://forgejo-gitea-http.cicd.svc.cluster.local:3000
+  # tokenSecret: name of the K8s Secret that holds the runner registration token
+  # created automatically by the helmfile presync hook (see helmfile.yaml.gotmpl)
   tokenSecret: runner-token
   resources:
     requests:
@@ -32,7 +39,7 @@ dind:
 persistence:
   reg:
     storageClass: longhorn  # Unified StorageClass (3 replicas)
-    size: 20Gi              # .runner registration file + action tool cache + actcache artifacts
+    size: 1Gi               # .runner registration file + config — survives pod restarts
   dind:
     storageClass: longhorn  # Unified StorageClass (3 replicas)
     size: 30Gi              # docker layer cache — keeps rebuilds fast across restarts
@@ -42,18 +49,7 @@ tolerations:
     operator: Exists
     effect: NoSchedule
 
-# Pin to az-b (talos-cp-2) — more Longhorn storage than az-a (worker-1 over-provisioned).
-# RWO PVCs will recreate on talos-cp-2 when nodeSelector changes.
+# Pin to az-a (talos-cp-1) — sole Longhorn node; RWO PVCs (reg/dind cache) only
+# attach there.
 nodeSelector:
-  topology.kubernetes.io/zone: az-b
-
-# GC CronJob — prunes Docker images/volumes/build-cache and actcache across
-# ALL forgejo-runner pods. Only enable in default values (golang instance);
-# disable in per-runner overrides so it renders once.
-gc:
-  enabled: true
-  schedule: "*/30 * * * *"     # every 30 minutes
-  image: alpine/k8s:1.31.0
-  pruneAge: "30m"              # Docker artifacts unused longer than this get pruned
-  pruneAgeHours: 0.5            # Same as pruneAge but numeric for date arithmetic in shell
-  actcacheMaxAgeDays: 1        # actcache files older than N days (aggressive for heavy Rust cargo builds)
+  topology.kubernetes.io/zone: az-a

k8s/infra/iam/authentik-provision-job.yaml

@@ -0,0 +1,181 @@
+# Authentik OAuth provisioning — PostSync hook, reruns on every ArgoCD sync
+# (hook-delete-policy: BeforeHookCreation deletes the previous run's Job before
+# creating a new one, so this stays reconciled the same way the rest of the
+# cluster does — no separate manual bootstrap step like setup_talos_iam.sh /
+# provision_oidc.py, which never got migrated off the old helmfile workflow).
+#
+# What it does (see scripts/authentik-provision.py docstring): creates the
+# "groups" scope mapping, homelab-admins / grafana-admins groups, the "rock"
+# admin user, OAuth2 providers + Applications for grafana/minio/forgejo/argocd,
+# and binds homelab-admins to all of them. The script is generated into the
+# authentik-provision-script ConfigMap by kustomize configMapGenerator (see
+# kustomization.yaml), not embedded here.
+#
+# RBAC: this Job only touches Secrets (get existing client secrets, create new
+# ones for forgejo/argocd/rock) across the namespaces those services live in.
+# It never touches any other resource type.
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: authentik-provisioner
+  namespace: iam
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: authentik-provisioner
+rules:
+  - apiGroups: [""]
+    resources: ["secrets"]
+    verbs: ["get", "list", "create", "update", "patch"]
+---
+# One RoleBinding per namespace the script touches (least-privilege: Secrets
+# only, and only in these 5 namespaces — not a cluster-wide ClusterRoleBinding).
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: iam
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: cicd
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: argocd
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: logging
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: storage
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: paperless
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: authentik-provision
+  namespace: iam
+  annotations:
+    argocd.argoproj.io/hook: PostSync
+    argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
+spec:
+  ttlSecondsAfterFinished: 600
+  backoffLimit: 3
+  template:
+    spec:
+      serviceAccountName: authentik-provisioner
+      restartPolicy: Never
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 1000
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        - name: provision
+          image: python:3.12-alpine
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+          env:
+            - name: AUTHENTIK_BOOTSTRAP_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: authentik-secrets
+                  key: AUTHENTIK_BOOTSTRAP_TOKEN
+          volumeMounts:
+            - name: script
+              mountPath: /script
+          command:
+            - /bin/sh
+            - -c
+            - |
+              set -e
+              echo "waiting for authentik-server..."
+              until wget -q -O /dev/null http://authentik-server.iam.svc.cluster.local/-/health/ready/ 2>/dev/null; do
+                sleep 5
+              done
+              echo "installing kubectl (via python urllib - no apk/curl: this"
+              echo "container runs as non-root UID 1000 and can't write to"
+              echo "apk's directories or /usr/local/bin, both root-owned in"
+              echo "the python:3.12-alpine image; /tmp is world-writable)..."
+              python3 -c "
+              import urllib.request, os, stat
+              kver = urllib.request.urlopen('https://dl.k8s.io/release/stable.txt').read().decode().strip()
+              url = f'https://dl.k8s.io/release/{kver}/bin/linux/amd64/kubectl'
+              urllib.request.urlretrieve(url, '/tmp/kubectl')
+              st = os.stat('/tmp/kubectl')
+              os.chmod('/tmp/kubectl', st.st_mode | stat.S_IEXEC)
+              "
+              export PATH="/tmp:$PATH"
+              echo "running provisioning script..."
+              python3 /script/authentik-provision.py
+      volumes:
+        - name: script
+          configMap:
+            name: authentik-provision-script

k8s/infra/iam/kustomization.yaml

@@ -1,12 +1,35 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
-
+# NOTE: no top-level `namespace:` transformer here (removed) - it used to
+# force-rewrite metadata.namespace to "iam" on every resource in this
+# kustomization, which was harmless while every manifest here only ever
+# targeted the iam namespace itself. authentik-provision-job.yaml's
+# RoleBindings deliberately target cicd/argocd/logging/storage (least-
+# privilege access for the authentik-provisioner ServiceAccount to touch
+# Secrets in those namespaces) - the namespace transformer would have
+# silently rewritten all of them back to iam, breaking the RBAC. Every
+# manifest in this directory already sets its own explicit
+# metadata.namespace, so dropping the transformer changes nothing for the
+# existing resources/.
 resources:
+  - authentik-provision-job.yaml
   - rbac-dashboard-rolebinding.yaml
 
-# IAM provisioning is manual-only (security-sensitive).
-# Script: scripts/iam/authentik-provision.py
-# Run:
-#   export AUTHENTIK_BOOTSTRAP_TOKEN=$(kubectl -n iam get secret authentik-secrets \
-#     -o jsonpath='{.data.AUTHENTIK_BOOTSTRAP_TOKEN}' | base64 -d)
-#   python3 scripts/iam/authentik-provision.py
+# Provisioning/verification python lives in scripts/*.py (real files, linted +
+# diff-friendly) and is generated into ConfigMaps here rather than embedded in
+# the job YAML. disableNameSuffixHash keeps the names stable so the Jobs'
+# configMap volume refs and PostSync hook-delete semantics keep working; each
+# hook Job is recreated per sync so it always mounts the latest script.
+configMapGenerator:
+  - name: authentik-provision-script
+    namespace: iam
+    files:
+      - authentik-provision.py=scripts/authentik-provision.py
+
+generatorOptions:
+  disableNameSuffixHash: true
+# authentik-migrations-job.yaml removed — redundant + broken. The authentik
+# `server` entrypoint runs migrations itself; this standalone job lacked the
+# authentik-secrets envFrom (Secret key missing) and always failed.
+# SOPS secrets (*.enc.yaml) handled by ArgoCD SOPS plugin at sync time
+# authentik/vault deployed via ArgoCD Helm source

k8s/infra/iam/scripts/authentik-provision.py

@@ -0,0 +1,437 @@
+#!/usr/bin/env python3
+"""
+Authentik OAuth provisioning - idempotent, safe to re-run (ArgoCD PostSync hook).
+
+Creates/updates, in order:
+  1. A custom "groups" OAuth2 scope mapping (Authentik ships openid/email/profile
+     by default but NOT groups - required for ArgoCD RBAC group mapping and
+     Grafana's role_attribute_path, both of which read a `groups` claim).
+  2. Groups: homelab-admins (is_superuser=true), grafana-admins.
+  3. User "rock": created if missing, always (re-)synced into both groups above.
+     Password is generated once and only written to the k8s Secret
+     rock-credentials (iam ns) the first time the user is created - re-runs
+     never rotate an existing password.
+  4. OAuth2/OIDC providers + Applications for: grafana, minio, forgejo, argocd.
+     Client secrets are read from existing k8s Secrets (grafana-oidc, minio-oidc)
+     if present, or generated once and written out (forgejo-oidc, oidc-secret)
+     the first time.
+  5. PolicyBinding of homelab-admins -> every Application above, so "rock" (and
+     anyone else in that group) has guaranteed access regardless of each app's
+     default visibility.
+
+Talks to Authentik over the in-cluster Service (authentik-server.iam.svc:80),
+authenticating with the bootstrap token. Everything is done with GET-then-
+create-or-patch so this can be re-run on every ArgoCD sync without duplicating
+or clobbering objects (PostSync hook, not a one-shot Job with hook-delete).
+
+kubectl is used only to read/write the small set of Secrets this script
+touches - it shells out rather than using the Python k8s client to keep the
+container image to stdlib Python + the kubectl binary, no pip installs.
+"""
+import json
+import os
+import secrets
+import string
+import subprocess
+import sys
+import urllib.error
+import urllib.request
+
+AUTHENTIK_URL = "http://authentik-server.iam.svc.cluster.local"
+TOKEN = os.environ["AUTHENTIK_BOOTSTRAP_TOKEN"]
+
+
+def api(method, path, data=None):
+    url = f"{AUTHENTIK_URL}{path}"
+    body = json.dumps(data).encode() if data is not None else None
+    req = urllib.request.Request(
+        url,
+        data=body,
+        method=method,
+        headers={
+            "Authorization": f"Bearer {TOKEN}",
+            "Content-Type": "application/json",
+        },
+    )
+    try:
+        with urllib.request.urlopen(req, timeout=30) as resp:
+            raw = resp.read()
+            return resp.status, (json.loads(raw) if raw else {})
+    except urllib.error.HTTPError as e:
+        raw = e.read()
+        try:
+            parsed = json.loads(raw) if raw else {}
+        except json.JSONDecodeError:
+            parsed = {"raw": raw.decode(errors="replace")}
+        return e.code, parsed
+
+
+def die(msg):
+    print(f"FATAL: {msg}", file=sys.stderr)
+    sys.exit(1)
+
+
+def gen_secret(n=40):
+    alphabet = string.ascii_letters + string.digits
+    return "".join(secrets.choice(alphabet) for _ in range(n))
+
+
+def kubectl_get_secret_key(namespace, name, key):
+    """Returns decoded value, or None if the secret/key doesn't exist."""
+    p = subprocess.run(
+        ["kubectl", "-n", namespace, "get", "secret", name, "-o", f"jsonpath={{.data.{key}}}"],
+        capture_output=True, text=True,
+    )
+    if p.returncode != 0 or not p.stdout.strip():
+        return None
+    import base64
+    return base64.b64decode(p.stdout).decode()
+
+
+def kubectl_create_secret(namespace, name, literals: dict, labels: dict = None):
+    """Idempotent: create-or-update via dry-run|apply, same pattern used
+    elsewhere in this repo (setup_vault.sh, apply-vault-secrets.sh)."""
+    args = ["kubectl", "-n", namespace, "create", "secret", "generic", name]
+    for k, v in literals.items():
+        args += [f"--from-literal={k}={v}"]
+    args += ["--dry-run=client", "-o", "yaml"]
+    render = subprocess.run(args, capture_output=True, text=True)
+    if render.returncode != 0:
+        die(f"rendering secret {namespace}/{name}: {render.stderr}")
+    apply = subprocess.run(["kubectl", "apply", "-f", "-"], input=render.stdout,
+                            capture_output=True, text=True)
+    if apply.returncode != 0:
+        die(f"applying secret {namespace}/{name}: {apply.stderr}")
+    print(f"  secret {namespace}/{name}: {apply.stdout.strip()}")
+    if labels:
+        # argocd's `$secret:key` substitution only reads Secrets carrying
+        # app.kubernetes.io/part-of: argocd — without it OIDC login fails with
+        # oauth2 "invalid_client" (empty client_secret sent to the IdP).
+        label_args = ["kubectl", "-n", namespace, "label", "secret", name,
+                      "--overwrite"] + [f"{k}={v}" for k, v in labels.items()]
+        subprocess.run(label_args, capture_output=True, text=True)
+
+
+def get_or_create(list_path, create_path, query, payload, patch_existing=None):
+    status, res = api("GET", f"{list_path}?{query}")
+    if status != 200:
+        die(f"GET {list_path}?{query} -> {status} {res}")
+    results = res.get("results", [])
+    if results:
+        obj = results[0]
+        if patch_existing:
+            status, obj2 = api("PATCH", f"{create_path}{obj['pk']}/", patch_existing)
+            if status not in (200, 201):
+                die(f"PATCH {create_path}{obj['pk']}/ -> {status} {obj2}")
+            return obj2
+        return obj
+    status, obj = api("POST", create_path, payload)
+    if status not in (200, 201):
+        die(f"POST {create_path} -> {status} {obj}")
+    return obj
+
+
+# -----------------------------------------------------------------------------
+print("[1/5] Ensuring custom 'groups' scope mapping exists...")
+groups_mapping = get_or_create(
+    "/api/v3/propertymappings/provider/scope/",
+    "/api/v3/propertymappings/provider/scope/",
+    "scope_name=groups",
+    {
+        "name": "homelab: groups claim",
+        "scope_name": "groups",
+        # request.user.ak_groups is deprecated in authentik 2026.x (logs a
+        # deprecation warning on every token issue) -> use request.user.groups.
+        "expression": (
+            "return {\"groups\": [group.name for group in request.user.groups.all()]}"
+        ),
+    },
+    # Force the expression onto the already-created mapping on re-run.
+    patch_existing={
+        "expression": (
+            "return {\"groups\": [group.name for group in request.user.groups.all()]}"
+        ),
+    },
+)
+GROUPS_MAPPING_PK = groups_mapping["pk"]
+
+# MinIO maps OIDC users to a MinIO policy via a "policy" claim
+# (MINIO_IDENTITY_OPENID_CLAIM_NAME=policy). Emit consoleAdmin (full admin) for
+# homelab-admins members, readonly for everyone else. Without this claim MinIO
+# assigns no policy and OIDC users get no access.
+_POLICY_EXPR = (
+    "return {\"policy\": \"consoleAdmin\" "
+    "if request.user.ak_groups.filter(name=\"homelab-admins\").exists() "
+    "else \"readonly\"}"
+)
+policy_mapping = get_or_create(
+    "/api/v3/propertymappings/provider/scope/",
+    "/api/v3/propertymappings/provider/scope/",
+    "scope_name=minio",
+    {
+        "name": "homelab: minio policy claim",
+        "scope_name": "minio",
+        "expression": _POLICY_EXPR,
+    },
+    patch_existing={"expression": _POLICY_EXPR},
+)
+POLICY_MAPPING_PK = policy_mapping["pk"]
+
+# Fetch the standard openid/email/profile mapping pks (shipped by default).
+status, res = api("GET", "/api/v3/propertymappings/provider/scope/")
+by_scope = {m["scope_name"]: m["pk"] for m in res["results"]}
+SCOPE_PKS = [by_scope["openid"], by_scope["email"], by_scope["profile"], GROUPS_MAPPING_PK]
+
+status, res = api("GET", "/api/v3/flows/instances/?slug=default-provider-authorization-implicit-consent")
+AUTHORIZATION_FLOW_PK = res["results"][0]["pk"]
+status, res = api("GET", "/api/v3/flows/instances/?slug=default-provider-invalidation-flow")
+INVALIDATION_FLOW_PK = res["results"][0]["pk"]
+status, res = api("GET", "/api/v3/crypto/certificatekeypairs/?has_key=true")
+SIGNING_KEY_PK = res["results"][0]["pk"]
+
+# -----------------------------------------------------------------------------
+print("[2/5] Ensuring groups homelab-admins / grafana-admins exist...")
+homelab_admins = get_or_create(
+    "/api/v3/core/groups/", "/api/v3/core/groups/",
+    "name=homelab-admins",
+    {"name": "homelab-admins", "is_superuser": True},
+)
+grafana_admins = get_or_create(
+    "/api/v3/core/groups/", "/api/v3/core/groups/",
+    "name=grafana-admins",
+    {"name": "grafana-admins", "is_superuser": False},
+)
+
+# -----------------------------------------------------------------------------
+print("[3/5] Ensuring user 'rock' exists with admin group membership...")
+status, res = api("GET", "/api/v3/core/users/?username=rock")
+rock_password = None
+if res.get("results"):
+    rock = res["results"][0]
+    status, rock = api("PATCH", f"/api/v3/core/users/{rock['pk']}/", {
+        "groups": [homelab_admins["pk"], grafana_admins["pk"]],
+        "is_active": True,
+        # email is REQUIRED: Grafana's OIDC login reads the email claim from
+        # userinfo; an empty email makes Grafana fall back to a GitHub-style
+        # <userinfo>/emails call, which Authentik 404s -> login fails entirely.
+        "email": "[email protected]",
+    })
+    if status not in (200, 201):
+        die(f"PATCH user rock -> {status} {rock}")
+    print("  rock already exists, group membership synced (password unchanged)")
+else:
+    rock_password = gen_secret(24)
+    status, rock = api("POST", "/api/v3/core/users/", {
+        "username": "rock",
+        "name": "Rock",
+        "is_active": True,
+        # Required for Grafana OIDC (see PATCH branch above).
+        "email": "[email protected]",
+        "groups": [homelab_admins["pk"], grafana_admins["pk"]],
+        "path": "users",
+        "type": "internal",
+    })
+    if status not in (200, 201):
+        die(f"POST user rock -> {status} {rock}")
+    status, pw_res = api("POST", f"/api/v3/core/users/{rock['pk']}/set_password/",
+                          {"password": rock_password})
+    if status not in (200, 204):
+        die(f"set_password for rock -> {status} {pw_res}")
+    kubectl_create_secret("iam", "rock-credentials", {
+        "username": "rock",
+        "password": rock_password,
+    })
+    print("  rock created, credentials stored in iam/rock-credentials")
+
+# -----------------------------------------------------------------------------
+print("[4/5] Ensuring OAuth2 providers + applications for grafana/minio/forgejo/argocd...")
+
+SERVICES = {
+    "grafana": {
+        "client_secret_source": ("logging", "grafana-oidc", "GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET"),
+        "redirect_uris": ["https://grafana.riotpiao.com/login/generic_oauth"],
+        "launch_url": "https://grafana.riotpiao.com",
+        "display_name": "Grafana",
+    },
+    "minio": {
+        "client_secret_source": ("storage", "minio-oidc", "MINIO_IDENTITY_OPENID_CLIENT_SECRET"),
+        "redirect_uris": ["https://minio.riotpiao.com/oauth_callback"],
+        "launch_url": "https://minio.riotpiao.com",
+        "display_name": "MinIO",
+    },
+    "forgejo": {
+        # No secret exists yet for forgejo - generate + store on first run.
+        "client_secret_source": ("cicd", "forgejo-oidc", "CLIENT_SECRET"),
+        "generate_if_missing": True,
+        "redirect_uris": [
+            "https://forgejo.riotpiao.com/user/oauth2/authentik/callback",
+            "https://forgejo.riotpiao.com/user/oauth2/openidconnect/callback",
+        ],
+        "launch_url": "https://forgejo.riotpiao.com",
+        "display_name": "Forgejo",
+    },
+    "argocd": {
+        # oidc-secret uses hyphenated keys (client-id/client-secret) per
+        # argocd-values.yaml's `$oidc-secret:client-id` / `:client-secret` refs.
+        "client_secret_source": ("argocd", "oidc-secret", "client-secret"),
+        "generate_if_missing": True,
+        "extra_secret_literals": {"client-id": "argocd"},
+        # argocd only reads $secret refs from Secrets labelled part-of: argocd.
+        "secret_labels": {"app.kubernetes.io/part-of": "argocd"},
+        "redirect_uris": ["https://argocd.riotpiao.com/auth/callback"],
+        "launch_url": "https://argocd.riotpiao.com",
+        "display_name": "Argo CD",
+    },
+    "homarr": {
+        "client_secret_source": ("dashboard", "homarr-oidc", "client-secret"),
+        "generate_if_missing": True,
+        "extra_secret_literals": {"client-id": "homarr"},
+        "redirect_uris": ["https://homarr.riotpiao.com/api/auth/callback/oidc"],
+        "launch_url": "https://homarr.riotpiao.com",
+        "display_name": "Homarr",
+    },
+    "paperless": {
+        # No secret exists yet for paperless - generate + store on first run.
+        # django-allauth's generic openid_connect provider callback path is
+        # /accounts/oidc/<provider_id>/login/callback/ - provider_id "authentik"
+        # is set in PAPERLESS_SOCIALACCOUNT_PROVIDERS (see configmap.yaml).
+        "client_secret_source": ("paperless", "paperless-oidc", "CLIENT_SECRET"),
+        "generate_if_missing": True,
+        "redirect_uris": ["https://paperless.riotpiao.com/accounts/oidc/authentik/login/callback/"],
+        "launch_url": "https://paperless.riotpiao.com",
+        "display_name": "Paperless-ngx",
+    },
+}
+
+app_pks_for_binding = []
+
+for name, cfg in SERVICES.items():
+    # MinIO also needs the "policy" claim (via the minio scope mapping) so its
+    # MINIO_IDENTITY_OPENID_CLAIM_NAME=policy maps homelab-admins -> consoleAdmin.
+    provider_mappings = SCOPE_PKS + ([POLICY_MAPPING_PK] if name == "minio" else [])
+    ns, secret_name, key = cfg["client_secret_source"]
+    client_secret = kubectl_get_secret_key(ns, secret_name, key)
+    if client_secret is None:
+        if not cfg.get("generate_if_missing"):
+            print(f"  WARNING: {ns}/{secret_name} key {key} not found and "
+                  f"generate_if_missing not set for '{name}' - skipping provider/app")
+            continue
+        client_secret = gen_secret(40)
+        literals = {key: client_secret}
+        literals.update(cfg.get("extra_secret_literals", {}))
+        kubectl_create_secret(ns, secret_name, literals,
+                              labels=cfg.get("secret_labels"))
+        print(f"  {name}: generated new client secret -> {ns}/{secret_name}")
+    else:
+        print(f"  {name}: using existing client secret from {ns}/{secret_name}")
+
+    if name == "paperless":
+        # paperless-ngx's django-allauth OIDC config takes client_id/secret
+        # bundled inside one JSON blob (PAPERLESS_SOCIALACCOUNT_PROVIDERS), not
+        # discrete env vars - compose it here and store it alongside
+        # CLIENT_SECRET so the Deployment can source it directly via
+        # secretKeyRef, no shell wrapper needed. Runs every time (not just on
+        # generate), so it stays in sync if the client_secret is ever rotated
+        # by hand.
+        providers_json = json.dumps({
+            "openid_connect": {
+                "APPS": [{
+                    "provider_id": "authentik",
+                    "name": "Authentik",
+                    "client_id": "paperless",
+                    "secret": client_secret,
+                    "settings": {
+                        "server_url": "https://authentik.riotpiao.com/application/o/paperless/.well-known/openid-configuration",
+                    },
+                }],
+            },
+        })
+        kubectl_create_secret("paperless", "paperless-oidc", {
+            "CLIENT_SECRET": client_secret,
+            "SOCIALACCOUNT_PROVIDERS_JSON": providers_json,
+        })
+
+    provider = get_or_create(
+        "/api/v3/providers/oauth2/", "/api/v3/providers/oauth2/",
+        f"name={name}",
+        {
+            "name": name,
+            "client_id": name,
+            "client_secret": client_secret,
+            "client_type": "confidential",
+            "authorization_flow": AUTHORIZATION_FLOW_PK,
+            "invalidation_flow": INVALIDATION_FLOW_PK,
+            "signing_key": SIGNING_KEY_PK,
+            "property_mappings": provider_mappings,
+            "sub_mode": "hashed_user_id",
+            "include_claims_in_id_token": True,
+            # authentik 2026.x requires grant_types to be set explicitly; the
+            # API defaults it to [] when omitted, which makes /authorize reject
+            # every login with "Invalid grant_type for provider" ->
+            # invalid_request. authorization_code = the web SSO flow all these
+            # apps use; refresh_token = long-lived sessions (offline_access).
+            "grant_types": ["authorization_code", "refresh_token"],
+            "redirect_uris": [
+                {"matching_mode": "strict", "url": u} for u in cfg["redirect_uris"]
+            ],
+        },
+        # Keep the redirect_uris/mappings/grant_types in sync on re-run, but
+        # never touch client_secret again once created (that's the source of
+        # truth in the k8s Secret, and re-sending it here is harmless anyway).
+        patch_existing={
+            "property_mappings": provider_mappings,
+            "grant_types": ["authorization_code", "refresh_token"],
+            "redirect_uris": [
+                {"matching_mode": "strict", "url": u} for u in cfg["redirect_uris"]
+            ],
+        },
+    )
+
+    # superuser_full_list=true is REQUIRED on the LIST: the applications list
+    # applies access-policy filtering to the results array (these apps are bound
+    # to homelab-admins, and the bootstrap-token user akadmin is not a member),
+    # so without it the GET returns an empty results list even though the app
+    # exists -> fall through to POST -> 400 "already exists".
+    #
+    # We deliberately do NOT patch_existing here: the application DETAIL endpoint
+    # (PATCH /applications/{pk}/) enforces the same access policy and does NOT
+    # honor superuser_full_list, so PATCH-by-pk returns 404 for akadmin once the
+    # homelab-admins binding exists. That 404 aborted the loop before later
+    # providers got their grant_types. slug/provider/launch_url are set at
+    # creation and are stable (provider is get_or_create'd by name, stable pk),
+    # so find-or-create is sufficient.
+    application = get_or_create(
+        "/api/v3/core/applications/", "/api/v3/core/applications/",
+        f"slug={name}&superuser_full_list=true",
+        {
+            "name": cfg["display_name"],
+            "slug": name,
+            "provider": provider["pk"],
+            "meta_launch_url": cfg["launch_url"],
+        },
+    )
+    app_pks_for_binding.append((name, application["pk"]))
+    print(f"  {name}: provider pk={provider['pk']} application pk={application['pk']}")
+
+# -----------------------------------------------------------------------------
+print("[5/5] Binding homelab-admins to every application (guaranteed access for rock)...")
+for name, app_pk in app_pks_for_binding:
+    get_or_create(
+        "/api/v3/policies/bindings/", "/api/v3/policies/bindings/",
+        f"target={app_pk}&group={homelab_admins['pk']}",
+        {
+            "target": app_pk,
+            "group": homelab_admins["pk"],
+            "order": 0,
+            "enabled": True,
+        },
+    )
+    print(f"  {name}: homelab-admins bound")
+
+print("\nDone. Summary:")
+print("  groups:  homelab-admins (superuser), grafana-admins")
+print("  user:    rock -> homelab-admins + grafana-admins")
+print(f"  apps:    {', '.join(n for n, _ in app_pks_for_binding)}")
+if rock_password:
+    print("  NOTE: rock's password was generated this run - see")
+    print("  kubectl -n iam get secret rock-credentials -o jsonpath='{.data.password}' | base64 -d")

k8s/infra/logging/grafana-values.yaml

@@ -44,10 +44,6 @@ grafana.ini:
   server:
     root_url: https://grafana.riotpiao.com
 
-  # Allow embedding dashboards in iframes (NextJS integration)
-  security:
-    allow_embedding: true
-
   # No anonymous read access — every user must log in via Authentik SSO.
   auth.anonymous:
     enabled: false
@@ -68,14 +64,15 @@ grafana.ini:
   # doesn't return localhost redirects in its token responses.
   #
   # role_attribute_path: JMESPath expression evaluated against the userinfo
-  # response. akadmin gets GrafanaAdmin (server admin, can impersonate);
-  # homelab-admins members get Admin (org admin); everyone else Viewer.
+  # response. Members of the 'grafana-admins' Authentik group get Admin role;
+  # everyone else gets Viewer. The group name must match exactly what Authentik
+  # sends in the 'groups' claim.
   auth.generic_oauth:
     enabled: true
     name: Authentik
     allow_sign_up: true
     client_id: grafana
-    scopes: openid email profile groups
+    scopes: openid email profile
     auth_url: https://authentik.riotpiao.com/application/o/authorize/
     token_url: https://authentik.riotpiao.com/application/o/token/
     api_url: https://authentik.riotpiao.com/application/o/userinfo/
@@ -86,8 +83,7 @@ grafana.ini:
     email_attribute_path: email
     login_attribute_path: preferred_username
     name_attribute_path: name
-    role_attribute_path: "preferred_username == 'akadmin' && 'GrafanaAdmin' || contains(groups[*], 'homelab-admins') && 'Admin' || 'Viewer'"
-    allow_assign_grafana_admin: true
+    role_attribute_path: "contains(groups[*], 'grafana-admins') && 'Admin' || 'Viewer'"
     use_pkce: false
     use_refresh_token: false
     skip_org_role_sync: false

k8s/infra/longhorn/expand-replicas-job.yaml

@@ -0,0 +1,96 @@
+# PostSync hook Job that expands all Longhorn volumes to 3 replicas.
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: longhorn-expand-replicas
+  namespace: longhorn-system
+  annotations:
+    argocd.argoproj.io/hook: PostSync
+    argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
+spec:
+  backoffLimit: 3
+  template:
+    metadata:
+      name: longhorn-expand-replicas
+    spec:
+      restartPolicy: Never
+      serviceAccountName: longhorn-expand-replicas
+      containers:
+      - name: expand
+        image: bitnami/kubectl:latest
+        command:
+        - /bin/bash
+        - -c
+        - |
+          set -euo pipefail
+          
+          echo "Waiting for all 3 Longhorn nodes to be Ready..."
+          for i in {1..30}; do
+            READY_COUNT=$(kubectl -n longhorn-system get nodes.longhorn.io -o json | \
+              jq -r '[.items[] | select(.status.conditions[] | select(.type=="Ready" and .status=="True"))] | length')
+            
+            if [ "$READY_COUNT" -ge 3 ]; then
+              echo "✓ All 3 nodes Ready"
+              break
+            fi
+            
+            echo "  $READY_COUNT/3 nodes ready, waiting..."
+            sleep 10
+            
+            if [ $i -eq 30 ]; then
+              echo "✗ Timeout waiting for 3 nodes"
+              exit 1
+            fi
+          done
+          
+          echo
+          echo "Expanding volumes with < 3 replicas..."
+          
+          VOLUMES=$(kubectl -n longhorn-system get volumes.longhorn.io -o json | \
+            jq -r '.items[] | select(.spec.numberOfReplicas < 3) | .metadata.name')
+          
+          if [ -z "$VOLUMES" ]; then
+            echo "  No volumes need expansion"
+          else
+            echo "$VOLUMES" | while read -r vol; do
+              CURRENT=$(kubectl -n longhorn-system get volume "$vol" -o jsonpath='{.spec.numberOfReplicas}')
+              echo "  $vol: $CURRENT → 3 replicas"
+              kubectl -n longhorn-system patch volume "$vol" --type merge \
+                -p '{"spec":{"numberOfReplicas":3}}'
+            done
+          fi
+          
+          echo
+          echo "Done. Final replica counts:"
+          kubectl -n longhorn-system get volumes.longhorn.io \
+            -o custom-columns='NAME:.metadata.name,REPLICAS:.spec.numberOfReplicas' | head -20
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: longhorn-expand-replicas
+  namespace: longhorn-system
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: longhorn-expand-replicas
+  namespace: longhorn-system
+rules:
+- apiGroups: ["longhorn.io"]
+  resources: ["volumes", "nodes"]
+  verbs: ["get", "list", "patch"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: longhorn-expand-replicas
+  namespace: longhorn-system
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: longhorn-expand-replicas
+subjects:
+- kind: ServiceAccount
+  name: longhorn-expand-replicas
+  namespace: longhorn-system

k8s/infra/longhorn/kustomization.yaml

@@ -8,11 +8,11 @@ resources:
   - longhorn-servicemonitor.yaml
   - longhorn-taint-toleration.yaml
   - longhorn-nodes.yaml
-  - longhorn-tag-disks-job.yaml
+  - expand-replicas-job.yaml
   - patch-csi-tolerations-job.yaml
   - longhorn-add-disks-job.yaml  # Add extra disks to talos-cp-2
 # Longhorn deployed via bootstrap script or Helm.
 # These manifests configure it: unified StorageClass (default, 3 replicas),
 # Prometheus ServiceMonitor, taint toleration for control-plane nodes, explicit
 # Node CRDs for cp-2/cp-3, CSI plugin tolerations, and a PostSync hook Job
-# that configures storage for the cluster.
+# that ensures all existing volumes have 3 replicas.

k8s/infra/longhorn/longhorn-cnpg-storageclass.yaml

@@ -27,7 +27,6 @@ provisioner: driver.longhorn.io
 allowVolumeExpansion: true
 parameters:
   numberOfReplicas: "3"
-  diskSelector: "storage"           # Only schedule on CP node disks, not GPU worker
   staleReplicaTimeout: "30"
   fromBackup: ""
   dataLocality: "best-effort"

k8s/infra/longhorn/longhorn-nodes.yaml

@@ -1,14 +1,16 @@
-# Longhorn Node CRDs for cp-2, cp-3, and worker-1.
-# cp-2/cp-3 have the control-plane taint, so Longhorn doesn't auto-discover them.
-# Explicit Node CRDs + the taint-toleration setting enable storage across all 3 CP nodes.
+# Longhorn Node CRDs for cp-2 and cp-3.
+# These nodes have the control-plane taint, so Longhorn doesn't auto-discover them.
+# Explicit Node CRDs + the taint-toleration setting enable storage across all 3 nodes.
 #
-# `spec.disks` is deliberately absent for CP nodes. Longhorn owns disk identity:
-# it names the entry itself and writes `storageReserved`, `diskType` and
-# `evictionRequested` into it. Disk tags are applied via kubectl patch (see
-# longhorn-tag-disks-job.yaml) since disk names are runtime-discovered.
+# `spec.disks` is deliberately absent. Longhorn owns disk identity: it names the
+# entry itself (`default-disk-080400000000`, not `default-disk`) and writes
+# `storageReserved`, `diskType` and `evictionRequested` into it. Declaring a
+# `default-disk` key here never matched the live one, so the Application sat
+# OutOfSync and selfHeal kept trying to add a SECOND disk record pointing at the
+# same /var/lib/longhorn path — which is worse than the drift it was fixing.
 #
-# worker-1 has scheduling disabled so only StorageClasses with explicit
-# diskSelector (e.g. `llm` for longhorn-llm-local) can use it.
+# What these objects are actually for is `allowScheduling: true` on tainted
+# control-plane nodes. That is all they need to declare.
 ---
 apiVersion: longhorn.io/v1beta2
 kind: Node
@@ -29,13 +31,3 @@ spec:
   name: talos-cp-3
   allowScheduling: true
   tags: []
----
-apiVersion: longhorn.io/v1beta2
-kind: Node
-metadata:
-  name: worker-1
-  namespace: longhorn-system
-spec:
-  name: worker-1
-  allowScheduling: false
-  tags: []

k8s/infra/longhorn/longhorn-paperless-storageclass.yaml

@@ -10,23 +10,13 @@ apiVersion: storage.k8s.io/v1
 kind: StorageClass
 metadata:
   name: longhorn-paperless-media
-  annotations:
-    # StorageClass.parameters is immutable - any future edit here needs
-    # delete+recreate, not patch. Same fix as longhorn-cnpg-storageclass.yaml;
-    # safe since a StorageClass is only consulted at provisioning time.
-    argocd.argoproj.io/sync-options: Replace=true,Force=true
 provisioner: driver.longhorn.io
 allowVolumeExpansion: true
 reclaimPolicy: Retain
 volumeBindingMode: WaitForFirstConsumer
 parameters:
   numberOfReplicas: "1"
-  # diskSelector alone is sufficient: only cp-3 has a disk tagged
-  # paperless-media, so placement is already pinned. A nodeSelector value
-  # here must be a Longhorn *node* tag (set via nodes.longhorn.io spec.tags),
-  # not a Kubernetes hostname - "talos-cp-3" was never a node tag, which made
-  # every provision attempt fail with "specified node tag talos-cp-3 does
-  # not exist".
   diskSelector: "paperless-media"
+  nodeSelector: "talos-cp-3"
   staleReplicaTimeout: "30"
   fsType: "ext4"