rock/homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
1518ebc2dd58ba2563810a7825594f1eb94ea7cf
homelab/k8s_authentik-migration.md
T

14 KiB
Raw Blame History

Kubernetes RBAC → Authentik Migration Plan

Status: Draft — awaiting decisions in Open Decisions Date: 2026-08-19 Scope: Move human identity + authorization for the cluster and its apps onto Authentik groups.

1. Finding: there is nothing to literally import

The original ask was "import the existing RBAC from the cluster into Authentik". Inventory of the live cluster says that set is empty.

1.1 Cluster RBAC audit (live, 2026-08-19)

Every ClusterRoleBinding / RoleBinding carrying a User or Group subject:

Binding Role Subjects
cluster-admin ClusterRole/cluster-admin Group:system:masters
system-bootstrap-approve-node-client-csr ClusterRole/system:certificates.k8s.io:certificatesigningrequests:nodeclient Group:system:bootstrappers:nodes
system-bootstrap-node-bootstrapper ClusterRole/system:node-bootstrapper Group:system:bootstrappers:nodes, Group:system:nodes
system-bootstrap-node-renewal ClusterRole/system:certificates.k8s.io:certificatesigningrequests:selfnodeclient Group:system:nodes
kube-system/system::extension-apiserver-authentication-reader Role/extension-apiserver-authentication-reader User:system:kube-controller-manager, User:system:kube-scheduler
kube-system/system::leader-locking-kube-controller-manager Role/… User:system:kube-controller-manager
kube-system/system::leader-locking-kube-scheduler Role/… User:system:kube-scheduler

All of these are Talos/control-plane machinery. Every other binding in the cluster targets a ServiceAccount (controllers: Argo CD, cert-manager, Cilium, Longhorn, MinIO operator, Prometheus, …).

There are zero human users in Kubernetes RBAC. The only human access path today is the Talos-issued admin certificate in cluster-config/kubeconfig, which lands in system:masters — a single shared, unattributable, non-revocable god credential.

The kube-apiserver has no OIDC configured. apiServer: in terraform/templates/controlplane.tftpl:110 contains only certSANs, image, admissionControl and auditPolicy.

1.2 What human RBAC does exist (app level, scattered)

Location Rule
k8s/bootstrap/phase4-argocd/argocd-values.yaml:163-169 policy.default: role:readonly; g, admin/rock/cicd, role:admin; g, homelab-admins, role:admin
k8s/infra/logging/grafana-values.yaml:86 role_attribute_path: contains(groups[*], 'grafana-admins') && 'Admin' || 'Viewer'
k8s/infra/iam/scripts/authentik-provision.py (minio scope mapping) homelab-admins → consoleAdmin, everyone else → readonly
Authentik, provisioned Groups homelab-admins (superuser), grafana-admins. Single user: rock, member of both.

1.3 Known drift

project-usage/authentik-oidc.md documents three groups — homelab-admins, homelab-devops, homelab-viewers. Only homelab-admins and grafana-admins are actually created by the provisioner. The documentation is wrong today; Phase 1 fixes it.

1.4 Restated goal

Not an import. This is: define the group taxonomy in Authentik, make Authentik the sole issuer of human identity, then bind those groups to Kubernetes roles and app roles declaratively through GitOps.

2. Constraints

  • GitOps only. No local terraform apply, no manual kubectl apply, no Authentik console clicking for anything that must survive a rebuild. Changes land in git; Forgejo Actions and Argo CD do the applying.
  • Authentik config stays Argo CD-managed via the authentik-provision.py PostSync hook. No Terraform Authentik provider. A prior attempt at one is abandoned at .claude/worktrees/debug-auth-issue/terraform/authentik-config.tf — do not revive it.
  • Terraform is limited to Talos machine config (Phase 3 only), applied through the pipeline.
  • The Talos admin certificate stays as break-glass. It is unaffected by OIDC and remains the recovery path when Authentik is unavailable.

3. Phases

Each phase is independently shippable and independently revertible.

Phase 1 — Group taxonomy

Extend the provisioner's group step ([2/5] in authentik-provision.py) from two groups to four:

Group is_superuser Intent
homelab-admins true Full cluster + full app admin
homelab-devops false Deploy and operate workloads; no IAM, no cluster config
homelab-viewers false Read-only everywhere
grafana-admins false Grafana Admin role specifically

Files: k8s/infra/iam/scripts/authentik-provision.py, project-usage/authentik-oidc.md (fix §1.3 drift).

Verify

kubectl -n iam exec deploy/authentik-server -- \
  curl -sH "Authorization: Bearer $TOKEN" localhost:9000/api/v3/core/groups/ \
  | jq -r '.results[].name'
# expect: homelab-admins, homelab-devops, homelab-viewers, grafana-admins
# and: rock still a member of homelab-admins + grafana-admins

Rollback: revert the commit. Extra groups with no bindings grant nothing.

Phase 2 — Authentik OAuth2 provider for Kubernetes

Add a kubernetes entry to the SERVICES dict in authentik-provision.py, with two departures from the existing web-app entries:

  • client_type: public — kubectl is a public client and cannot hold a secret. No k8s Secret is created for it, so the client_secret_source machinery is skipped for this service.
  • grant_types: ["authorization_code", "refresh_token", "urn:ietf:params:oauth:grant-type:device_code"] plus PKCE, to support both browser and headless login.
  • redirect_uris: http://localhost:8000, http://localhost:18000 (kubelogin's loopback listeners).

Reuses the existing groups property mapping created in step [1/5]. No new claim work.

Verify

curl -s https://authentik.riotpiao.com/application/o/kubernetes/.well-known/openid-configuration | jq .

# device-code login, then decode:
kubectl oidc-login get-token --oidc-issuer-url=... --oidc-client-id=kubernetes --oidc-extra-scope=groups
# id_token payload must contain: preferred_username, groups[]

Rollback: delete provider + application in Authentik; nothing else consumes them yet.

Phase 3 — Enable OIDC on kube-apiserver (Terraform, pipeline-applied)

Add to apiServer.extraArgs in terraform/templates/controlplane.tftpl:

    extraArgs:
      oidc-issuer-url: https://authentik.riotpiao.com/application/o/kubernetes/
      oidc-client-id: kubernetes
      oidc-username-claim: preferred_username
      oidc-username-prefix: "authentik:"
      oidc-groups-claim: groups
      oidc-groups-prefix: "authentik:"

Security — the prefixes are mandatory, not cosmetic. Without oidc-groups-prefix, anyone able to create or rename an Authentik group to system:masters obtains cluster-admin on the cluster. Without oidc-username-prefix, an Authentik username can impersonate a built-in identity such as system:kube-controller-manager. Both prefixes must be present in the same change that enables OIDC. Do not ship this phase partially.

Operational notes:

  • Apply is a rolling control-plane machine-config change: one node at a time, wait for Ready and for etcd quorum before proceeding to the next.
  • The Talos admin certificate authenticates via client cert, not OIDC, and is unaffected. It is the recovery path if the issuer URL is wrong or Authentik is down.
  • The issuer must be reachable from the control-plane nodes and its TLS chain must be trusted by them. Confirm this before applying, because a bad issuer URL means the apiserver logs OIDC discovery failures on every start.

Verify (per node, after each rolls)

kubectl auth whoami --token="$ID_TOKEN"
# expect Username: authentik:rock
#        Groups:   authentik:homelab-admins, authentik:grafana-admins, system:authenticated

kubectl --kubeconfig cluster-config/kubeconfig get nodes   # cert path still works

Rollback: revert the template change, re-run the pipeline, roll the control plane back. Cert-based access is never interrupted, so this rollback is safe at any point.

Phase 4 — Bind Authentik groups to Kubernetes roles

New manifest k8s/infra/iam/rbac-oidc-bindings.yaml, referenced from k8s/infra/iam/kustomization.yaml:

Subject Role Kind
Group:authentik:homelab-admins ClusterRole/cluster-admin ClusterRoleBinding
Group:authentik:homelab-devops ClusterRole/edit see Open Decisions #2
Group:authentik:homelab-viewers ClusterRole/view ClusterRoleBinding

admin, edit and view are the built-in aggregated ClusterRoles and already exist in the cluster — no custom roles needed.

Verify

kubectl auth can-i '*' '*' --all-namespaces --as=probe --as-group=authentik:homelab-admins   # yes
kubectl auth can-i delete pods -A          --as=probe --as-group=authentik:homelab-viewers   # no
kubectl auth can-i get pods -n apps        --as=probe --as-group=authentik:homelab-viewers   # yes
kubectl auth can-i get secrets -n iam      --as=probe --as-group=authentik:homelab-devops    # no

Rollback: revert the commit; Argo CD prunes the bindings.

Phase 5 — Client wiring and documentation

  • Ship a kubeconfig template using the kubelogin (kubectl oidc-login) exec credential plugin — no embedded certs, no long-lived token on disk.
  • Document login, token cache location, and the break-glass cert path in project-usage/authentik-oidc.md.
  • Same edit corrects the stale group list from §1.3.

Verify: on a clean machine with no certificates, kubectl get nodes triggers a browser login and succeeds; kubectl auth whoami reports the authentik:-prefixed identity.

Phase 6 — Converge app RBAC on the same four groups

Target Change
Argo CD policy.csv add g, homelab-devops, role:admin (or a scoped custom role) and g, homelab-viewers, role:readonly
Grafana role_attribute_path three-tier: grafana-admins/homelab-admins → Admin, homelab-devops → Editor, else Viewer
MinIO policy claim expression homelab-admins → consoleAdmin, homelab-devops → readwrite, else readonly
Argo CD local accounts retire accounts.rock once SSO admin is proven. Keep accounts.cicd — the pipeline needs a non-SSO apiKey.

Verify: end-to-end browser login per app as a test user who is only in homelab-devops, confirming the expected role in each of Argo CD, Grafana and MinIO. The e2e suite under tests/e2e/tests/authentik.spec.ts is the place to encode this.

Rollback: per-app revert; each app's mapping is independent.

4. Risks

Risk Severity Mitigation
Missing oidc-groups-prefix → group-name privilege escalation to cluster-admin Critical Prefixes ship in the same commit as the OIDC flags; Phase 4 verification asserts the prefixed form
Bad issuer URL / untrusted TLS bricks apiserver OIDC Medium Cert-based admin access is unaffected; roll one control-plane node at a time and check apiserver logs before continuing
Authentik outage blocks all human cluster access Medium Talos admin cert is the documented break-glass path and never depends on Authentik
Argo CD cicd account removed by accident → pipeline loses cluster access Medium Phase 6 explicitly retires only accounts.rock
Group renamed in Authentik silently revokes cluster access Low Group names are asserted by the provisioner on every Argo CD sync, so drift self-heals

5. Open Decisions

  1. Is kubectl-via-OIDC actually wanted? Phases 3–5 only pay for themselves if humans besides the shared admin cert need API access. If the answer is no, the plan collapses to Phases 1 and 6: no Terraform change, no control-plane roll, app-level SSO consolidation only.

  2. homelab-devops blast radius — cluster-wide ClusterRoleBinding to edit, or namespaced RoleBindings limited to application namespaces (excluding iam, kube-system, argocd)? Namespaced is the tighter default; cluster-wide is less to maintain.

  3. Username claim — preferred_username (human-readable in audit logs, mutable) versus sub (stable, opaque). Recommendation: preferred_username, given a single-operator homelab where audit readability beats rename-safety.

6. Reference

Reference in New Issue View Git Blame Copy Permalink