rock
c2071da9d8
Eight objects cover grafana, argocd, comfyui, minio, paperless, immich, forgejo and api-gw. Each names its ZITADEL app, the Secret it writes, the consumer to roll and whether the consumer reloads hot or needs a restart. All are suspended: the controller image does not exist yet, and an unsuspended object would claim an intent that no process acts on. The README records the retirement target for the 34 .enc.yaml files: six root-of-trust secrets that cannot rotate themselves survive, everything else becomes a KeyRotation and leaves git.
51 lines
2.2 KiB
Markdown
51 lines
2.2 KiB
Markdown
# Credential rotation
|
|
|
|
One `KeyRotation` object describes one credential. The controller creates the value,
|
|
writes the Secret, proves the new value works, then retires the old one.
|
|
|
|
## Why the state order is the safety property
|
|
|
|
```
|
|
Prepare -> Publish -> Verify -> Retire
|
|
```
|
|
|
|
`Retire` runs only for a provider whose old credential still works after the publish
|
|
step, and only after `Verify` passes. A failure before that point leaves the working
|
|
credential untouched. The hand-run rotation that broke Grafana did the opposite: it
|
|
regenerated the secret and never re-pointed the consumer.
|
|
|
|
## Two strategies, decided by the provider, not by preference
|
|
|
|
| Provider | Overlap in ZITADEL | Strategy | Retire |
|
|
|---|---|---|---|
|
|
| `zitadel-machine-key` | many keys per service user | `overlap` | after the new key obtains a token |
|
|
| `zitadel-oidc-client` | one secret per app | `atomic` | nothing to retire |
|
|
|
|
## The single master credential
|
|
|
|
The controller authenticates with one machine key, mounted from Secret
|
|
`iam/key-rotation-controller`. That key is the only credential the cluster cannot
|
|
rotate by itself, so it is the only one that stays encrypted in git.
|
|
|
|
Every other credential is created by the controller, which means the 34 `.enc.yaml`
|
|
files under `k8s/argocd/secrets/` are migration debt, not a design. The retirement
|
|
target is one root-of-trust file per thing that cannot rotate itself:
|
|
|
|
| Survives (root of trust) | Why it cannot rotate |
|
|
|---|---|
|
|
| ZITADEL instance masterkey | encrypts the whole ZITADEL database |
|
|
| Vault unseal keys | unsealing needs a human-held key |
|
|
| Talos repair config | recovery path, must work when the cluster does not |
|
|
| homelab CA | signs the wildcard certificate |
|
|
| controller bootstrap key | chicken and egg: it creates every other key |
|
|
| etcd defrag credentials | needed by a job that runs outside the controller |
|
|
|
|
Everything else — Grafana, Argo CD, Forgejo, MinIO, Paperless, Immich, ComfyUI,
|
|
api-gw, Gotify — is a KeyRotation object, and its secret leaves git.
|
|
|
|
## Status
|
|
|
|
The CRD is installed. Eight `KeyRotation` objects exist for the consumers above and
|
|
all are `suspend: true`: the controller image does not exist yet, so no process acts
|
|
on them. poimen-memory is excluded from this migration on purpose.
|