51 lines
2.2 KiB
Markdown
51 lines
2.2 KiB
Markdown
|
# Credential rotation
|
||
|
|
|
||
|
|
One `KeyRotation` object describes one credential. The controller creates the value,
|
||
|
|
writes the Secret, proves the new value works, then retires the old one.
|
||
|
|
|
||
|
|
## Why the state order is the safety property
|
||
|
|
|
||
|
|
```
|
||
|
|
Prepare -> Publish -> Verify -> Retire
|
||
|
|
```
|
||
|
|
|
||
|
|
`Retire` runs only for a provider whose old credential still works after the publish
|
||
|
|
step, and only after `Verify` passes. A failure before that point leaves the working
|
||
|
|
credential untouched. The hand-run rotation that broke Grafana did the opposite: it
|
||
|
|
regenerated the secret and never re-pointed the consumer.
|
||
|
|
|
||
|
|
## Two strategies, decided by the provider, not by preference
|
||
|
|
|
||
|
|
| Provider | Overlap in ZITADEL | Strategy | Retire |
|
||
|
|
|---|---|---|---|
|
||
|
|
| `zitadel-machine-key` | many keys per service user | `overlap` | after the new key obtains a token |
|
||
|
|
| `zitadel-oidc-client` | one secret per app | `atomic` | nothing to retire |
|
||
|
|
|
||
|
|
## The single master credential
|
||
|
|
|
||
|
|
The controller authenticates with one machine key, mounted from Secret
|
||
|
|
`iam/key-rotation-controller`. That key is the only credential the cluster cannot
|
||
|
|
rotate by itself, so it is the only one that stays encrypted in git.
|
||
|
|
|
||
|
|
Every other credential is created by the controller, which means the 34 `.enc.yaml`
|
||
|
|
files under `k8s/argocd/secrets/` are migration debt, not a design. The retirement
|
||
|
|
target is one root-of-trust file per thing that cannot rotate itself:
|
||
|
|
|
||
|
|
| Survives (root of trust) | Why it cannot rotate |
|
||
|
|
|---|---|
|
||
|
|
| ZITADEL instance masterkey | encrypts the whole ZITADEL database |
|
||
|
|
| Vault unseal keys | unsealing needs a human-held key |
|
||
|
|
| Talos repair config | recovery path, must work when the cluster does not |
|
||
|
|
| homelab CA | signs the wildcard certificate |
|
||
|
|
| controller bootstrap key | chicken and egg: it creates every other key |
|
||
|
|
| etcd defrag credentials | needed by a job that runs outside the controller |
|
||
|
|
|
||
|
|
Everything else — Grafana, Argo CD, Forgejo, MinIO, Paperless, Immich, ComfyUI,
|
||
|
|
api-gw, Gotify — is a KeyRotation object, and its secret leaves git.
|
||
|
|
|
||
|
|
## Status
|
||
|
|
|
||
|
|
The CRD is installed. Eight `KeyRotation` objects exist for the consumers above and
|
||
|
|
all are `suspend: true`: the controller image does not exist yet, so no process acts
|
||
|
|
on them. poimen-memory is excluded from this migration on purpose.
|