feat(iam): wire Zitadel into namespace iam under ArgoCD
Coexistence wiring for the Authentik replacement: ZITADEL gets its own CNPG cluster, SOPS masterkey, Helm values and ArgoCD Application in namespace iam. Authentik keeps serving; no consumer is moved and no Authentik object changes. - k8s/infra/databases/zitadel-db.yaml: CNPG Postgres 16.2, 3 instances - k8s/infra/iam/zitadel-values.yaml: chart 10.4.0 pinned, DSN from the CNPG secret, masterkey from a Secret, metrics ServiceMonitor, second-sync caveat - k8s/argocd/apps/30-security.yaml: Application named zitadel, wave 3, automated sync deliberately absent for the first sync (chart PreSync hook re-run risk) - k8s/argocd/secrets/zitadel-masterkey.enc.yaml: 32-byte masterkey, SOPS - k8s/infra/monitoring/alerts/svc-zitadel-rules.yaml: target-down, login-UI, crash-loop and PreSync-job-failed rules - terraform/files/coredns/Corefile: in-cluster rewrite for the zitadel host Not applied. The cluster is mid-incident: vault-0 and authentik-server are 0/1, authentik-db-2 has 14 restarts, prometheus-0 is 1/2 CrashLoopBackOff and kube-state-metrics has 1925 restarts. Two manual steps still gate the first sync: the Cloudflare Tunnel public hostname plus a proxied CNAME, and terraform apply + make apply-cp for the CoreDNS rewrite (Talos-owned). The offline masterkey copy is at ~/zitadel-masterkey.offline (mode 600). Move it to sealed storage and delete it before the first sync. Losing it makes every encrypted ZITADEL value unreadable.
This commit is contained in:
@@ -0,0 +1,68 @@
|
||||
# ZITADEL alert parity with the Authentik rules it will replace.
|
||||
#
|
||||
# Only two signals are ported at this point. The metrics endpoint
|
||||
# (/debug/metrics, scraped by the chart's own ServiceMonitor in ns iam) exposes
|
||||
# no request or error counters that this cluster has verified, so no error-rate
|
||||
# rule is written here. Writing one against a guessed series name would produce
|
||||
# a permanently-firing or permanently-silent alert — both worse than none.
|
||||
# AuthentikOutpostDown is deliberately not ported: it monitors an outpost that
|
||||
# carries zero providers and is deleted with the migration, not migrated.
|
||||
apiVersion: monitoring.coreos.com/v1
|
||||
kind: PrometheusRule
|
||||
metadata:
|
||||
name: zitadel-rules
|
||||
namespace: iam
|
||||
spec:
|
||||
groups:
|
||||
- name: zitadel.rules
|
||||
interval: 30s
|
||||
rules:
|
||||
- alert: ZitadelDown
|
||||
# absent() is required: equality alone cannot fire when the series is
|
||||
# gone (ServiceMonitor deleted, Service selector changed, pods scaled
|
||||
# to zero). Those are exactly the outages a target-down rule exists for.
|
||||
expr: absent(up{job="zitadel"}) or up{job="zitadel"} == 0
|
||||
for: 10m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: "ZITADEL API target is down"
|
||||
description: "Prometheus cannot scrape the ZITADEL API for 10 minutes. Token issuance and every OIDC login depend on it. Check the zitadel pods and the zitadel-db cluster in namespace iam. Value: {{ $value }}"
|
||||
|
||||
- alert: ZitadelLoginUIDown
|
||||
expr: absent(kube_deployment_status_replicas_available{namespace="iam",deployment="zitadel-login"}) or kube_deployment_status_replicas_available{namespace="iam",deployment="zitadel-login"} == 0
|
||||
for: 10m
|
||||
labels:
|
||||
severity: warning
|
||||
annotations:
|
||||
summary: "ZITADEL login UI has no available replica"
|
||||
description: "The login UI serves /ui/v2/login. With zero available replicas a browser login completes at the IdP but cannot render. Value: {{ $value }}"
|
||||
|
||||
- alert: ZitadelRestarting
|
||||
# pod!~"zitadel-db-.*" matters: the CNPG database pods share the
|
||||
# `zitadel` name prefix, and a database restart must not page as an
|
||||
# IdP crash loop.
|
||||
expr: absent(kube_pod_container_status_restarts_total{namespace="iam",pod=~"zitadel.*",pod!~"zitadel-db-.*"}) or increase(kube_pod_container_status_restarts_total{namespace="iam",pod=~"zitadel.*",pod!~"zitadel-db-.*"}[1h]) > 3
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
annotations:
|
||||
summary: "ZITADEL pod restarts more than 3 times per hour"
|
||||
description: "A crash loop on the IdP is a cluster-wide login outage. Check the pod logs and the masterkey/database secret references. Value: {{ $value }}"
|
||||
|
||||
# The chart's zitadel-init and zitadel-setup Jobs are PreSync hooks with
|
||||
# restartPolicy: Never, so their restart counter stays 0. Without this
|
||||
# rule a failed schema init or first-instance setup reaches nobody but
|
||||
# the person who runs `kubectl -n iam get job`.
|
||||
# No absent() guard here on purpose: the chart ships only
|
||||
# `before-hook-creation`, so a successful hook Job is replaced at the
|
||||
# next PreSync. Once the jobs are retired, absent() would fire on a
|
||||
# healthy cluster.
|
||||
- alert: ZitadelSetupJobFailed
|
||||
expr: kube_job_status_failed{namespace="iam",job_name=~"zitadel-(init|setup).*"} > 0
|
||||
for: 5m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: "ZITADEL init/setup job failed"
|
||||
description: "A PreSync job failed, so ArgoCD aborted the sync before the Deployment was applied. Read the job pod logs, then delete the Job object to let the next sync retry. Value: {{ $value }}"
|
||||
Reference in New Issue
Block a user