rock
b8f934806e
comfyui served nothing and its oauth2-proxy could not start: the Secret carried only client-secret while the Deployment asked for client-id, cookie-secret and client-secret. Rather than repair a service nobody used, remove it from every layer. Gone: - k8s/apps/comfyui/* and its ArgoCD Application - the cloudflared tunnel rule for comfyui.riotpiao.com - the CoreDNS rewrite in the terraform Corefile (the tracked source) - the Cloudflare DNS record (terraform tfvars) - the ZITADEL OIDC app and its two rotation tables - the KeyRotation object Not gone by this commit: the comfyui namespace and its 50Gi PVC. ArgoCD does not own either, so a person deletes both by hand after the prune. The local cluster-config/talos-*.yaml copies are gitignored; they carry the same removal and need an apply-config to reach the live CoreDNS.
51 lines
2.2 KiB
Markdown
51 lines
2.2 KiB
Markdown
# Credential rotation
|
|
|
|
One `KeyRotation` object describes one credential. The controller creates the value,
|
|
writes the Secret, proves the new value works, then retires the old one.
|
|
|
|
## Why the state order is the safety property
|
|
|
|
```
|
|
Prepare -> Publish -> Verify -> Retire
|
|
```
|
|
|
|
`Retire` runs only for a provider whose old credential still works after the publish
|
|
step, and only after `Verify` passes. A failure before that point leaves the working
|
|
credential untouched. The hand-run rotation that broke Grafana did the opposite: it
|
|
regenerated the secret and never re-pointed the consumer.
|
|
|
|
## Two strategies, decided by the provider, not by preference
|
|
|
|
| Provider | Overlap in ZITADEL | Strategy | Retire |
|
|
|---|---|---|---|
|
|
| `zitadel-machine-key` | many keys per service user | `overlap` | after the new key obtains a token |
|
|
| `zitadel-oidc-client` | one secret per app | `atomic` | nothing to retire |
|
|
|
|
## The single master credential
|
|
|
|
The controller authenticates with one machine key, mounted from Secret
|
|
`iam/key-rotation-controller`. That key is the only credential the cluster cannot
|
|
rotate by itself, so it is the only one that stays encrypted in git.
|
|
|
|
Every other credential is created by the controller, which means the 34 `.enc.yaml`
|
|
files under `k8s/argocd/secrets/` are migration debt, not a design. The retirement
|
|
target is one root-of-trust file per thing that cannot rotate itself:
|
|
|
|
| Survives (root of trust) | Why it cannot rotate |
|
|
|---|---|
|
|
| ZITADEL instance masterkey | encrypts the whole ZITADEL database |
|
|
| Vault unseal keys | unsealing needs a human-held key |
|
|
| Talos repair config | recovery path, must work when the cluster does not |
|
|
| homelab CA | signs the wildcard certificate |
|
|
| controller bootstrap key | chicken and egg: it creates every other key |
|
|
| etcd defrag credentials | needed by a job that runs outside the controller |
|
|
|
|
Everything else — Grafana, Argo CD, Forgejo, MinIO, Paperless, Immich,
|
|
api-gw, Gotify — is a KeyRotation object, and its secret leaves git.
|
|
|
|
## Status
|
|
|
|
The CRD is installed. Eight `KeyRotation` objects exist for the consumers above and
|
|
all are `suspend: true`: the controller image does not exist yet, so no process acts
|
|
on them. poimen-memory is excluded from this migration on purpose.
|