harness
1ce9f86635
etcd's 2 GiB quota was reached on 2026-10-07 (backend 2.1 GiB, ~73 days of churn). etcd raised NOSPACE and refused all writes, so kubelets could not post node status: three nodes went NotReady and the tunnel origin pods on them disappeared. Recovery was compact + defrag + alarm disarm per member. The apiserver compacts every 5m, but compaction never shrinks the backend file - only a defrag returns space, and nothing scheduled it. This adds: - CronJob etcd-defrag (kube-system, Sundays 03:00): compact to the current revision, defrag each member sequentially, disarm NOSPACE, print before/after. - Secret etcd-defrag-creds (SOPS/ksops, k8s/argocd/secrets): etcd CA plus a CN=etcd-defrag client cert signed by the cluster's etcd CA (expires 2036-10). Applied by the existing apps: cluster-maintenance (directory) and 00-secrets (ksops generator).
38 lines
1.0 KiB
YAML
38 lines
1.0 KiB
YAML
apiVersion: viaduct.ai/v1
|
|
kind: ksops
|
|
metadata:
|
|
name: sops-secret-generator
|
|
annotations:
|
|
config.kubernetes.io/function: |
|
|
exec:
|
|
path: ksops
|
|
files:
|
|
- agent-pod-models.enc.yaml
|
|
- agent-pod-ssh-key.enc.yaml
|
|
- authentik-secrets.enc.yaml
|
|
- cloudflare-secrets.enc.yaml
|
|
- forgejo-registry-pat.enc.yaml
|
|
- forgejo-registry-pat-api.enc.yaml
|
|
- forgejo-registry-pull.enc.yaml
|
|
- forgejo-runner-token.enc.yaml
|
|
- forgejo-secrets.enc.yaml
|
|
- grafana-oidc-secrets.enc.yaml
|
|
- grafana-secrets.enc.yaml
|
|
- homarr-auth-oidc.enc.yaml
|
|
- homarr-db-encryption.enc.yaml
|
|
- homarr-secrets.enc.yaml
|
|
- homelab-ca-secrets.enc.yaml
|
|
- loki-secrets.enc.yaml
|
|
- minio-secrets.enc.yaml
|
|
- paperless-secrets.enc.yaml
|
|
- vault-secrets.enc.yaml
|
|
- vault-unseal-keys.enc.yaml
|
|
- portfolio-secrets.enc.yaml
|
|
- gotify-admin-secrets.enc.yaml
|
|
- gotify-tokens-secrets.enc.yaml
|
|
- gotify-smtp-secrets.enc.yaml
|
|
- forgejo-smtp-secrets.enc.yaml
|
|
- paperless-ai-secrets.enc.yaml
|
|
- pi-sandbox-secrets.enc.yaml
|
|
- etcd-defrag-creds.enc.yaml
|