docs: Add JWT auth rollout status

README.md

@@ -42,6 +42,86 @@ All logs + metrics centralized in Grafana for debugging
 - **Secrets at rest** — Vault + encrypted etcd; credentials never in logs or ConfigMaps
 - **Infrastructure-as-code** — Every service deployed via Helmfile; one `helmfile apply` recovers from total failure
 
+## ArgoCD — GitOps Deployment Flow
+
+**ArgoCD** pulls infrastructure changes from git and syncs the cluster automatically. 
+No manual `kubectl apply` — push to git, ArgoCD detects the change, and deploys within ~3 minutes.
+
+```
+Developer pushes to git
+       ↓
+ArgoCD detects change (every 3 min or webhook)
+       ↓
+Syncs manifests to cluster
+       ↓
+Workloads reconcile automatically
+```
+
+Applications are deployed in waves (numbered 00, 10, 20, 30, ...) to respect dependencies — 
+storage deploys before databases, databases before applications.
+
+### Tracked Git Repositories
+
+ArgoCD monitors these repos for changes:
+
+| Repository | Purpose |
+|------------|----------|
+| `https://github.com/Riotpiaole/riotpiao.homelab.com` | Main infrastructure repo (all manifests in `k8s/argocd/apps/`) |
+| `https://forgejo.riotpiao.com/rock/*` | Any `rock/*` repo in in-cluster Forgejo (apps + configs) |
+| `https://github.com/Riotpiaole/Poimen-*` | External Poimen services (memory, workflows) |
+
+To deploy a new application: create a git repo, add an Application manifest to the homelab repo's 
+`k8s/argocd/apps/`, commit + push, and ArgoCD syncs within 3 minutes.
+
+## Management Planes — Talos vs Kubernetes
+
+This cluster has **two separate management planes**, each with different workflows:
+
+| Plane | What it manages | Workflow | Tool |
+|-------|-----------------|----------|------|
+| **Talos (OS)** | Node configuration, kernel params, networking, CoreDNS, machine state | Edit `terraform/` → `terraform apply` → `make apply-cp` | `terraform` + `talosctl` |
+| **Kubernetes (workloads)** | All pods, services, deployments, ingresses, databases | Edit `k8s/argocd/apps/` → `git push` → ArgoCD syncs | `git` + ArgoCD |
+
+**Critical distinction:**
+- **Kubernetes resources** (`k8s/**`) flow through **git → ArgoCD** — never use `kubectl apply`
+- **Talos machine config** (`terraform/**`) uses **local `terraform apply`** (sanctioned exception — CI can't hold node credentials)
+
+Example: To add a CoreDNS hostname rewrite, you edit `terraform/files/coredns/Corefile`, then:
+```bash
+cd terraform && terraform apply -var-file=terraform.tfvars.local
+cd .. && make apply-cp     # talosctl apply-config to all 3 control planes
+```
+
+But to add a new Kubernetes Deployment or update an Ingress, you only `git push` — **never `kubectl apply`**.
+
+### CoreDNS ConfigMap Ownership — Critical
+
+⚠️ **Warning:** The `coredns` ConfigMap in `kube-system` namespace is **owned by Talos**, not ArgoCD or kubectl. 
+It is rendered from `terraform/files/coredns/Corefile` into Talos's machine config at bootstrap time.
+
+**Do not `kubectl apply` or `kubectl edit` this ConfigMap directly.** Doing so transfers field ownership to kubectl's 
+client-side-apply mechanism, and Talos's inline-manifest controller will silently no-op on every future reconcile 
+(server-side-apply conflict, no error surfaced).
+
+**To update CoreDNS (e.g., add a hostname rewrite):**
+1. Edit `terraform/files/coredns/Corefile`
+2. Commit + push
+3. Run `cd terraform && terraform apply -var-file=terraform.tfvars.local`
+4. Run `make apply-cp` to push config to all control planes
+5. CoreDNS picks up changes via its `reload` plugin — no pod restart needed
+
+**If you accidentally edited the ConfigMap directly and broke Talos's ownership:**
+```bash
+kubectl delete configmap coredns -n kube-system
+# Wait ~30s for Talos's k8s.ManifestApplyController to recreate it
+kubectl get configmap coredns -n kube-system -w
+```
+
+Or as a stopgap, apply the correct content yourself:
+```bash
+kubectl apply --server-side -f <(terraform output coredns_config)
+```
+
 ## Quick Start — Deploying the Cluster
 
 ### 1. Bootstrap Talos Nodes