feat(iam): automate Authentik OAuth provisioning + create admin user rock

Adds k8s/security/iam/authentik-provision-job.yaml - a PostSync hook Job
(reruns every ArgoCD sync via hook-delete-policy: BeforeHookCreation) that
replaces the never-migrated setup_talos_iam.sh / provision_oidc.py workflow
(both referenced helmfile + a Python script that no longer exists in this
repo - OAuth was never actually provisioned since the ArgoCD migration).

Idempotently creates:
  - Custom 'groups' OAuth2 scope mapping (Authentik doesn't ship one by
    default; required for ArgoCD's RBAC groups claim and Grafana's
    role_attribute_path, both of which read a groups claim from the token).
  - Groups: homelab-admins (is_superuser), grafana-admins.
  - User 'rock', member of both groups above - gets full Authentik superuser
    access, ArgoCD role:admin via the existing
     RBAC policy in argocd-values.yaml, and
    Grafana Admin role via role_attribute_path. Password generated once,
    stored in iam/rock-credentials (never rotated on re-run).
  - OAuth2 providers + Applications for grafana, minio, forgejo, argocd.
    Client secrets read from existing Secrets (grafana-oidc, minio-oidc) or
    generated once and written out (forgejo-oidc, argocd's oidc-secret).
  - PolicyBinding of homelab-admins -> every Application, guaranteeing rock
    access regardless of each app's default visibility.

Also fixes forgejo-values.yaml: oauth2.CLIENT_ID was set but CLIENT_SECRET
was missing entirely (oauth2 login could never have worked). Added via
extraEnv -> GITEA__oauth2__CLIENT_SECRET sourced from the new forgejo-oidc
Secret, since the oauth2: values map can't reference a Secret inline.

RBAC: dedicated ServiceAccount + ClusterRole (secrets get/list/create/update/
patch only) bound via namespace-scoped RoleBindings in iam/cicd/argocd/
logging/storage - the only 5 namespaces this job ever touches, and the only
resource type it ever touches.

NOTE: MinIO's OIDC env vars were removed from minio-tenant.yaml earlier
(blocked IAM init because the provider/app didn't exist yet -> 404 on
discovery). Now that this job creates them, re-adding MinIO's OIDC config is
a safe follow-up in a separate change.

k8s/security/ci-cd/forgejo-values.yaml

@@ -52,6 +52,12 @@ gitea:
       OPENID_CONNECT_DISCOVERY_URL: https://authentik.riotpiao.com/application/o/forgejo/.well-known/openid-configuration
       CLIENT_ID: forgejo
       AUTO_DISCOVER_URL: https://authentik.riotpiao.com/application/o/forgejo/.well-known/openid-configuration
+      # CLIENT_SECRET was missing entirely before - oauth2 login could never
+      # have worked. Value comes from the forgejo-oidc Secret (created by the
+      # authentik-provision PostSync hook, see k8s/security/iam/authentik-
+      # provision-job.yaml) via extraEnv below, since this oauth2: map is
+      # rendered directly into plain env vars and can't reference a Secret
+      # inline the way envFromSecret/extraEnv can.
     cache:
       ADAPTER: redis
       HOST: redis://forgejo-redis.cicd.svc:6379/0
@@ -139,6 +145,17 @@ tolerations:
 # trusting both the standard Mozilla bundle and our homelab CA.
 # Required for OIDC: Forgejo fetches Authentik's discovery endpoint which
 # presents a cert signed by homelab-ca.
+# CLIENT_SECRET for the oauth2 block above - gitea's own config map (oauth2:)
+# can only hold plain values, so the Secret-backed one is injected as
+# GITEA__oauth2__CLIENT_SECRET, following the chart's standard
+# GITEA__<section>__<KEY> env-var-to-ini-config convention.
+extraEnv:
+  - name: GITEA__oauth2__CLIENT_SECRET
+    valueFrom:
+      secretKeyRef:
+        name: forgejo-oidc
+        key: CLIENT_SECRET
+
 extraVolumes:
   - name: homelab-ca
     configMap:

k8s/security/iam/authentik-provision-job.yaml

@@ -0,0 +1,492 @@
+# Authentik OAuth provisioning — PostSync hook, reruns on every ArgoCD sync
+# (hook-delete-policy: BeforeHookCreation deletes the previous run's Job before
+# creating a new one, so this stays reconciled the same way the rest of the
+# cluster does — no separate manual bootstrap step like setup_talos_iam.sh /
+# provision_oidc.py, which never got migrated off the old helmfile workflow).
+#
+# What it does (see the embedded script's docstring below): creates the
+# "groups" scope mapping, homelab-admins / grafana-admins groups, the "rock"
+# admin user, OAuth2 providers + Applications for grafana/minio/forgejo/argocd,
+# and binds homelab-admins to all of them.
+#
+# RBAC: this Job only touches Secrets (get existing client secrets, create new
+# ones for forgejo/argocd/rock) across the namespaces those services live in.
+# It never touches any other resource type.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: authentik-provision-script
+  namespace: iam
+data:
+  authentik-provision.py: |
+    #!/usr/bin/env python3
+    """
+    Authentik OAuth provisioning - idempotent, safe to re-run (ArgoCD PostSync hook).
+    
+    Creates/updates, in order:
+      1. A custom "groups" OAuth2 scope mapping (Authentik ships openid/email/profile
+         by default but NOT groups - required for ArgoCD RBAC group mapping and
+         Grafana's role_attribute_path, both of which read a `groups` claim).
+      2. Groups: homelab-admins (is_superuser=true), grafana-admins.
+      3. User "rock": created if missing, always (re-)synced into both groups above.
+         Password is generated once and only written to the k8s Secret
+         rock-credentials (iam ns) the first time the user is created - re-runs
+         never rotate an existing password.
+      4. OAuth2/OIDC providers + Applications for: grafana, minio, forgejo, argocd.
+         Client secrets are read from existing k8s Secrets (grafana-oidc, minio-oidc)
+         if present, or generated once and written out (forgejo-oidc, oidc-secret)
+         the first time.
+      5. PolicyBinding of homelab-admins -> every Application above, so "rock" (and
+         anyone else in that group) has guaranteed access regardless of each app's
+         default visibility.
+    
+    Talks to Authentik over the in-cluster Service (authentik-server.iam.svc:80),
+    authenticating with the bootstrap token. Everything is done with GET-then-
+    create-or-patch so this can be re-run on every ArgoCD sync without duplicating
+    or clobbering objects (PostSync hook, not a one-shot Job with hook-delete).
+    
+    kubectl is used only to read/write the small set of Secrets this script
+    touches - it shells out rather than using the Python k8s client to keep the
+    container image to stdlib Python + the kubectl binary, no pip installs.
+    """
+    import json
+    import os
+    import secrets
+    import string
+    import subprocess
+    import sys
+    import urllib.error
+    import urllib.request
+    
+    AUTHENTIK_URL = "http://authentik-server.iam.svc.cluster.local"
+    TOKEN = os.environ["AUTHENTIK_BOOTSTRAP_TOKEN"]
+    
+    
+    def api(method, path, data=None):
+        url = f"{AUTHENTIK_URL}{path}"
+        body = json.dumps(data).encode() if data is not None else None
+        req = urllib.request.Request(
+            url,
+            data=body,
+            method=method,
+            headers={
+                "Authorization": f"Bearer {TOKEN}",
+                "Content-Type": "application/json",
+            },
+        )
+        try:
+            with urllib.request.urlopen(req, timeout=30) as resp:
+                raw = resp.read()
+                return resp.status, (json.loads(raw) if raw else {})
+        except urllib.error.HTTPError as e:
+            raw = e.read()
+            try:
+                parsed = json.loads(raw) if raw else {}
+            except json.JSONDecodeError:
+                parsed = {"raw": raw.decode(errors="replace")}
+            return e.code, parsed
+    
+    
+    def die(msg):
+        print(f"FATAL: {msg}", file=sys.stderr)
+        sys.exit(1)
+    
+    
+    def gen_secret(n=40):
+        alphabet = string.ascii_letters + string.digits
+        return "".join(secrets.choice(alphabet) for _ in range(n))
+    
+    
+    def kubectl_get_secret_key(namespace, name, key):
+        """Returns decoded value, or None if the secret/key doesn't exist."""
+        p = subprocess.run(
+            ["kubectl", "-n", namespace, "get", "secret", name, "-o", f"jsonpath={{.data.{key}}}"],
+            capture_output=True, text=True,
+        )
+        if p.returncode != 0 or not p.stdout.strip():
+            return None
+        import base64
+        return base64.b64decode(p.stdout).decode()
+    
+    
+    def kubectl_create_secret(namespace, name, literals: dict):
+        """Idempotent: create-or-update via dry-run|apply, same pattern used
+        elsewhere in this repo (setup_vault.sh, apply-vault-secrets.sh)."""
+        args = ["kubectl", "-n", namespace, "create", "secret", "generic", name]
+        for k, v in literals.items():
+            args += [f"--from-literal={k}={v}"]
+        args += ["--dry-run=client", "-o", "yaml"]
+        render = subprocess.run(args, capture_output=True, text=True)
+        if render.returncode != 0:
+            die(f"rendering secret {namespace}/{name}: {render.stderr}")
+        apply = subprocess.run(["kubectl", "apply", "-f", "-"], input=render.stdout,
+                                capture_output=True, text=True)
+        if apply.returncode != 0:
+            die(f"applying secret {namespace}/{name}: {apply.stderr}")
+        print(f"  secret {namespace}/{name}: {apply.stdout.strip()}")
+    
+    
+    def get_or_create(list_path, create_path, query, payload, patch_existing=None):
+        status, res = api("GET", f"{list_path}?{query}")
+        if status != 200:
+            die(f"GET {list_path}?{query} -> {status} {res}")
+        results = res.get("results", [])
+        if results:
+            obj = results[0]
+            if patch_existing:
+                status, obj2 = api("PATCH", f"{create_path}{obj['pk']}/", patch_existing)
+                if status not in (200, 201):
+                    die(f"PATCH {create_path}{obj['pk']}/ -> {status} {obj2}")
+                return obj2
+            return obj
+        status, obj = api("POST", create_path, payload)
+        if status not in (200, 201):
+            die(f"POST {create_path} -> {status} {obj}")
+        return obj
+    
+    
+    # -----------------------------------------------------------------------------
+    print("[1/5] Ensuring custom 'groups' scope mapping exists...")
+    groups_mapping = get_or_create(
+        "/api/v3/propertymappings/provider/scope/",
+        "/api/v3/propertymappings/provider/scope/",
+        "scope_name=groups",
+        {
+            "name": "homelab: groups claim",
+            "scope_name": "groups",
+            "expression": (
+                "return {\"groups\": [group.name for group in request.user.ak_groups.all()]}"
+            ),
+        },
+    )
+    GROUPS_MAPPING_PK = groups_mapping["pk"]
+    
+    # Fetch the standard openid/email/profile mapping pks (shipped by default).
+    status, res = api("GET", "/api/v3/propertymappings/provider/scope/")
+    by_scope = {m["scope_name"]: m["pk"] for m in res["results"]}
+    SCOPE_PKS = [by_scope["openid"], by_scope["email"], by_scope["profile"], GROUPS_MAPPING_PK]
+    
+    status, res = api("GET", "/api/v3/flows/instances/?slug=default-provider-authorization-implicit-consent")
+    AUTHORIZATION_FLOW_PK = res["results"][0]["pk"]
+    status, res = api("GET", "/api/v3/flows/instances/?slug=default-provider-invalidation-flow")
+    INVALIDATION_FLOW_PK = res["results"][0]["pk"]
+    status, res = api("GET", "/api/v3/crypto/certificatekeypairs/?has_key=true")
+    SIGNING_KEY_PK = res["results"][0]["pk"]
+    
+    # -----------------------------------------------------------------------------
+    print("[2/5] Ensuring groups homelab-admins / grafana-admins exist...")
+    homelab_admins = get_or_create(
+        "/api/v3/core/groups/", "/api/v3/core/groups/",
+        "name=homelab-admins",
+        {"name": "homelab-admins", "is_superuser": True},
+    )
+    grafana_admins = get_or_create(
+        "/api/v3/core/groups/", "/api/v3/core/groups/",
+        "name=grafana-admins",
+        {"name": "grafana-admins", "is_superuser": False},
+    )
+    
+    # -----------------------------------------------------------------------------
+    print("[3/5] Ensuring user 'rock' exists with admin group membership...")
+    status, res = api("GET", "/api/v3/core/users/?username=rock")
+    rock_password = None
+    if res.get("results"):
+        rock = res["results"][0]
+        status, rock = api("PATCH", f"/api/v3/core/users/{rock['pk']}/", {
+            "groups": [homelab_admins["pk"], grafana_admins["pk"]],
+            "is_active": True,
+        })
+        if status not in (200, 201):
+            die(f"PATCH user rock -> {status} {rock}")
+        print("  rock already exists, group membership synced (password unchanged)")
+    else:
+        rock_password = gen_secret(24)
+        status, rock = api("POST", "/api/v3/core/users/", {
+            "username": "rock",
+            "name": "Rock",
+            "is_active": True,
+            "groups": [homelab_admins["pk"], grafana_admins["pk"]],
+            "path": "users",
+            "type": "internal",
+        })
+        if status not in (200, 201):
+            die(f"POST user rock -> {status} {rock}")
+        status, pw_res = api("POST", f"/api/v3/core/users/{rock['pk']}/set_password/",
+                              {"password": rock_password})
+        if status not in (200, 204):
+            die(f"set_password for rock -> {status} {pw_res}")
+        kubectl_create_secret("iam", "rock-credentials", {
+            "username": "rock",
+            "password": rock_password,
+        })
+        print("  rock created, credentials stored in iam/rock-credentials")
+    
+    # -----------------------------------------------------------------------------
+    print("[4/5] Ensuring OAuth2 providers + applications for grafana/minio/forgejo/argocd...")
+    
+    SERVICES = {
+        "grafana": {
+            "client_secret_source": ("logging", "grafana-oidc", "GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET"),
+            "redirect_uris": ["https://grafana.riotpiao.com/login/generic_oauth"],
+            "launch_url": "https://grafana.riotpiao.com",
+            "display_name": "Grafana",
+        },
+        "minio": {
+            "client_secret_source": ("storage", "minio-oidc", "MINIO_IDENTITY_OPENID_CLIENT_SECRET"),
+            "redirect_uris": ["https://minio.riotpiao.com/oauth_callback"],
+            "launch_url": "https://minio.riotpiao.com",
+            "display_name": "MinIO",
+        },
+        "forgejo": {
+            # No secret exists yet for forgejo - generate + store on first run.
+            "client_secret_source": ("cicd", "forgejo-oidc", "CLIENT_SECRET"),
+            "generate_if_missing": True,
+            "redirect_uris": [
+                "https://forgejo.riotpiao.com/user/oauth2/authentik/callback",
+                "https://forgejo.riotpiao.com/user/oauth2/openidconnect/callback",
+            ],
+            "launch_url": "https://forgejo.riotpiao.com",
+            "display_name": "Forgejo",
+        },
+        "argocd": {
+            # oidc-secret uses hyphenated keys (client-id/client-secret) per
+            # argocd-values.yaml's `$oidc-secret:client-id` / `:client-secret` refs.
+            "client_secret_source": ("argocd", "oidc-secret", "client-secret"),
+            "generate_if_missing": True,
+            "extra_secret_literals": {"client-id": "argocd"},
+            "redirect_uris": ["https://argocd.riotpiao.com/auth/callback"],
+            "launch_url": "https://argocd.riotpiao.com",
+            "display_name": "Argo CD",
+        },
+    }
+    
+    app_pks_for_binding = []
+    
+    for name, cfg in SERVICES.items():
+        ns, secret_name, key = cfg["client_secret_source"]
+        client_secret = kubectl_get_secret_key(ns, secret_name, key)
+        if client_secret is None:
+            if not cfg.get("generate_if_missing"):
+                print(f"  WARNING: {ns}/{secret_name} key {key} not found and "
+                      f"generate_if_missing not set for '{name}' - skipping provider/app")
+                continue
+            client_secret = gen_secret(40)
+            literals = {key: client_secret}
+            literals.update(cfg.get("extra_secret_literals", {}))
+            kubectl_create_secret(ns, secret_name, literals)
+            print(f"  {name}: generated new client secret -> {ns}/{secret_name}")
+        else:
+            print(f"  {name}: using existing client secret from {ns}/{secret_name}")
+    
+        provider = get_or_create(
+            "/api/v3/providers/oauth2/", "/api/v3/providers/oauth2/",
+            f"name={name}",
+            {
+                "name": name,
+                "client_id": name,
+                "client_secret": client_secret,
+                "client_type": "confidential",
+                "authorization_flow": AUTHORIZATION_FLOW_PK,
+                "invalidation_flow": INVALIDATION_FLOW_PK,
+                "signing_key": SIGNING_KEY_PK,
+                "property_mappings": SCOPE_PKS,
+                "sub_mode": "hashed_user_id",
+                "include_claims_in_id_token": True,
+                "redirect_uris": [
+                    {"matching_mode": "strict", "url": u} for u in cfg["redirect_uris"]
+                ],
+            },
+            # Keep the redirect_uris/mappings in sync on re-run, but never touch
+            # client_secret again once created (that's the source of truth in the
+            # k8s Secret, and re-sending it here is harmless/idempotent anyway).
+            patch_existing={
+                "property_mappings": SCOPE_PKS,
+                "redirect_uris": [
+                    {"matching_mode": "strict", "url": u} for u in cfg["redirect_uris"]
+                ],
+            },
+        )
+    
+        application = get_or_create(
+            "/api/v3/core/applications/", "/api/v3/core/applications/",
+            f"slug={name}",
+            {
+                "name": cfg["display_name"],
+                "slug": name,
+                "provider": provider["pk"],
+                "meta_launch_url": cfg["launch_url"],
+            },
+            patch_existing={
+                "provider": provider["pk"],
+                "meta_launch_url": cfg["launch_url"],
+            },
+        )
+        app_pks_for_binding.append((name, application["pk"]))
+        print(f"  {name}: provider pk={provider['pk']} application pk={application['pk']}")
+    
+    # -----------------------------------------------------------------------------
+    print("[5/5] Binding homelab-admins to every application (guaranteed access for rock)...")
+    for name, app_pk in app_pks_for_binding:
+        get_or_create(
+            "/api/v3/policies/bindings/", "/api/v3/policies/bindings/",
+            f"target={app_pk}&group={homelab_admins['pk']}",
+            {
+                "target": app_pk,
+                "group": homelab_admins["pk"],
+                "order": 0,
+                "enabled": True,
+            },
+        )
+        print(f"  {name}: homelab-admins bound")
+    
+    print("\nDone. Summary:")
+    print("  groups:  homelab-admins (superuser), grafana-admins")
+    print("  user:    rock -> homelab-admins + grafana-admins")
+    print(f"  apps:    {', '.join(n for n, _ in app_pks_for_binding)}")
+    if rock_password:
+        print("  NOTE: rock's password was generated this run - see")
+        print("  kubectl -n iam get secret rock-credentials -o jsonpath='{.data.password}' | base64 -d")
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: authentik-provisioner
+  namespace: iam
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: authentik-provisioner
+rules:
+  - apiGroups: [""]
+    resources: ["secrets"]
+    verbs: ["get", "list", "create", "update", "patch"]
+---
+# One RoleBinding per namespace the script touches (least-privilege: Secrets
+# only, and only in these 5 namespaces — not a cluster-wide ClusterRoleBinding).
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: iam
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: cicd
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: argocd
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: logging
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: authentik-provisioner
+  namespace: storage
+subjects:
+  - kind: ServiceAccount
+    name: authentik-provisioner
+    namespace: iam
+roleRef:
+  kind: ClusterRole
+  name: authentik-provisioner
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: authentik-provision
+  namespace: iam
+  annotations:
+    argocd.argoproj.io/hook: PostSync
+    argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
+spec:
+  ttlSecondsAfterFinished: 600
+  backoffLimit: 3
+  template:
+    spec:
+      serviceAccountName: authentik-provisioner
+      restartPolicy: Never
+      securityContext:
+        runAsNonRoot: true
+        runAsUser: 1000
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+        - name: provision
+          image: python:3.12-alpine
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop: ["ALL"]
+          env:
+            - name: AUTHENTIK_BOOTSTRAP_TOKEN
+              valueFrom:
+                secretKeyRef:
+                  name: authentik-secrets
+                  key: AUTHENTIK_BOOTSTRAP_TOKEN
+          volumeMounts:
+            - name: script
+              mountPath: /script
+          command:
+            - /bin/sh
+            - -c
+            - |
+              set -e
+              echo "waiting for authentik-server..."
+              until wget -q -O /dev/null http://authentik-server.iam.svc.cluster.local/-/health/ready/ 2>/dev/null; do
+                sleep 5
+              done
+              echo "installing kubectl..."
+              apk add --no-cache curl >/dev/null
+              KVER=$(curl -sL https://dl.k8s.io/release/stable.txt)
+              curl -sLo /usr/local/bin/kubectl "https://dl.k8s.io/release/${KVER}/bin/linux/amd64/kubectl"
+              chmod +x /usr/local/bin/kubectl
+              echo "running provisioning script..."
+              python3 /script/authentik-provision.py
+      volumes:
+        - name: script
+          configMap:
+            name: authentik-provision-script