271 lines
14 KiB
Markdown
271 lines
14 KiB
Markdown
# Kubernetes RBAC → Authentik Migration Plan
|
||
|
||
**Status:** Draft — awaiting decisions in [Open Decisions](#open-decisions)
|
||
**Date:** 2026-08-19
|
||
**Scope:** Move human identity + authorization for the cluster and its apps onto Authentik groups.
|
||
|
||
---
|
||
|
||
## 1. Finding: there is nothing to literally import
|
||
|
||
The original ask was "import the existing RBAC from the cluster into Authentik". Inventory of the live
|
||
cluster says that set is empty.
|
||
|
||
### 1.1 Cluster RBAC audit (live, 2026-08-19)
|
||
|
||
Every `ClusterRoleBinding` / `RoleBinding` carrying a `User` or `Group` subject:
|
||
|
||
| Binding | Role | Subjects |
|
||
|---|---|---|
|
||
| `cluster-admin` | `ClusterRole/cluster-admin` | `Group:system:masters` |
|
||
| `system-bootstrap-approve-node-client-csr` | `ClusterRole/system:certificates.k8s.io:certificatesigningrequests:nodeclient` | `Group:system:bootstrappers:nodes` |
|
||
| `system-bootstrap-node-bootstrapper` | `ClusterRole/system:node-bootstrapper` | `Group:system:bootstrappers:nodes`, `Group:system:nodes` |
|
||
| `system-bootstrap-node-renewal` | `ClusterRole/system:certificates.k8s.io:certificatesigningrequests:selfnodeclient` | `Group:system:nodes` |
|
||
| `kube-system/system::extension-apiserver-authentication-reader` | `Role/extension-apiserver-authentication-reader` | `User:system:kube-controller-manager`, `User:system:kube-scheduler` |
|
||
| `kube-system/system::leader-locking-kube-controller-manager` | `Role/…` | `User:system:kube-controller-manager` |
|
||
| `kube-system/system::leader-locking-kube-scheduler` | `Role/…` | `User:system:kube-scheduler` |
|
||
|
||
All of these are Talos/control-plane machinery. Every other binding in the cluster targets a
|
||
`ServiceAccount` (controllers: Argo CD, cert-manager, Cilium, Longhorn, MinIO operator, Prometheus, …).
|
||
|
||
**There are zero human users in Kubernetes RBAC.** The only human access path today is the Talos-issued
|
||
admin certificate in `cluster-config/kubeconfig`, which lands in `system:masters` — a single shared,
|
||
unattributable, non-revocable god credential.
|
||
|
||
The kube-apiserver has no OIDC configured. `apiServer:` in
|
||
[`terraform/templates/controlplane.tftpl:110`](terraform/templates/controlplane.tftpl#L110) contains only
|
||
`certSANs`, `image`, `admissionControl` and `auditPolicy`.
|
||
|
||
### 1.2 What human RBAC *does* exist (app level, scattered)
|
||
|
||
| Location | Rule |
|
||
|---|---|
|
||
| [`k8s/bootstrap/phase4-argocd/argocd-values.yaml:163-169`](k8s/bootstrap/phase4-argocd/argocd-values.yaml#L163-L169) | `policy.default: role:readonly`; `g, admin/rock/cicd, role:admin`; `g, homelab-admins, role:admin` |
|
||
| [`k8s/infra/logging/grafana-values.yaml:86`](k8s/infra/logging/grafana-values.yaml#L86) | `role_attribute_path: contains(groups[*], 'grafana-admins') && 'Admin' \|\| 'Viewer'` |
|
||
| [`k8s/infra/iam/scripts/authentik-provision.py`](k8s/infra/iam/scripts/authentik-provision.py) (`minio` scope mapping) | `homelab-admins → consoleAdmin`, everyone else → `readonly` |
|
||
| Authentik, provisioned | Groups `homelab-admins` (superuser), `grafana-admins`. Single user: `rock`, member of both. |
|
||
|
||
### 1.3 Known drift
|
||
|
||
[`project-usage/authentik-oidc.md`](project-usage/authentik-oidc.md) documents three groups —
|
||
`homelab-admins`, `homelab-devops`, `homelab-viewers`. Only `homelab-admins` and `grafana-admins` are
|
||
actually created by the provisioner. The documentation is wrong today; Phase 1 fixes it.
|
||
|
||
### 1.4 Restated goal
|
||
|
||
Not an import. This is: **define the group taxonomy in Authentik, make Authentik the sole issuer of human
|
||
identity, then bind those groups to Kubernetes roles and app roles declaratively through GitOps.**
|
||
|
||
---
|
||
|
||
## 2. Constraints
|
||
|
||
- **GitOps only.** No local `terraform apply`, no manual `kubectl apply`, no Authentik console clicking for
|
||
anything that must survive a rebuild. Changes land in git; Forgejo Actions and Argo CD do the applying.
|
||
- **Authentik config stays Argo CD-managed** via the `authentik-provision.py` PostSync hook. No Terraform
|
||
Authentik provider. A prior attempt at one is abandoned at
|
||
`.claude/worktrees/debug-auth-issue/terraform/authentik-config.tf` — do not revive it.
|
||
- **Terraform is limited to Talos machine config** (Phase 3 only), applied through the pipeline.
|
||
- **The Talos admin certificate stays as break-glass.** It is unaffected by OIDC and remains the recovery
|
||
path when Authentik is unavailable.
|
||
|
||
---
|
||
|
||
## 3. Phases
|
||
|
||
Each phase is independently shippable and independently revertible.
|
||
|
||
### Phase 1 — Group taxonomy
|
||
|
||
Extend the provisioner's group step (`[2/5]` in `authentik-provision.py`) from two groups to four:
|
||
|
||
| Group | `is_superuser` | Intent |
|
||
|---|---|---|
|
||
| `homelab-admins` | `true` | Full cluster + full app admin |
|
||
| `homelab-devops` | `false` | Deploy and operate workloads; no IAM, no cluster config |
|
||
| `homelab-viewers` | `false` | Read-only everywhere |
|
||
| `grafana-admins` | `false` | Grafana `Admin` role specifically |
|
||
|
||
Files: `k8s/infra/iam/scripts/authentik-provision.py`, `project-usage/authentik-oidc.md` (fix §1.3 drift).
|
||
|
||
**Verify**
|
||
```bash
|
||
kubectl -n iam exec deploy/authentik-server -- \
|
||
curl -sH "Authorization: Bearer $TOKEN" localhost:9000/api/v3/core/groups/ \
|
||
| jq -r '.results[].name'
|
||
# expect: homelab-admins, homelab-devops, homelab-viewers, grafana-admins
|
||
# and: rock still a member of homelab-admins + grafana-admins
|
||
```
|
||
|
||
**Rollback:** revert the commit. Extra groups with no bindings grant nothing.
|
||
|
||
---
|
||
|
||
### Phase 2 — Authentik OAuth2 provider for Kubernetes
|
||
|
||
Add a `kubernetes` entry to the `SERVICES` dict in `authentik-provision.py`, with two departures from the
|
||
existing web-app entries:
|
||
|
||
- `client_type: public` — kubectl is a public client and cannot hold a secret. No k8s Secret is created for
|
||
it, so the `client_secret_source` machinery is skipped for this service.
|
||
- `grant_types: ["authorization_code", "refresh_token", "urn:ietf:params:oauth:grant-type:device_code"]`
|
||
plus PKCE, to support both browser and headless login.
|
||
- `redirect_uris`: `http://localhost:8000`, `http://localhost:18000` (kubelogin's loopback listeners).
|
||
|
||
Reuses the existing `groups` property mapping created in step `[1/5]`. No new claim work.
|
||
|
||
**Verify**
|
||
```bash
|
||
curl -s https://authentik.riotpiao.com/application/o/kubernetes/.well-known/openid-configuration | jq .
|
||
|
||
# device-code login, then decode:
|
||
kubectl oidc-login get-token --oidc-issuer-url=... --oidc-client-id=kubernetes --oidc-extra-scope=groups
|
||
# id_token payload must contain: preferred_username, groups[]
|
||
```
|
||
|
||
**Rollback:** delete provider + application in Authentik; nothing else consumes them yet.
|
||
|
||
---
|
||
|
||
### Phase 3 — Enable OIDC on kube-apiserver (Terraform, pipeline-applied)
|
||
|
||
Add to `apiServer.extraArgs` in
|
||
[`terraform/templates/controlplane.tftpl`](terraform/templates/controlplane.tftpl#L110):
|
||
|
||
```yaml
|
||
extraArgs:
|
||
oidc-issuer-url: https://authentik.riotpiao.com/application/o/kubernetes/
|
||
oidc-client-id: kubernetes
|
||
oidc-username-claim: preferred_username
|
||
oidc-username-prefix: "authentik:"
|
||
oidc-groups-claim: groups
|
||
oidc-groups-prefix: "authentik:"
|
||
```
|
||
|
||
> **Security — the prefixes are mandatory, not cosmetic.**
|
||
> Without `oidc-groups-prefix`, anyone able to create or rename an Authentik group to `system:masters`
|
||
> obtains `cluster-admin` on the cluster. Without `oidc-username-prefix`, an Authentik username can
|
||
> impersonate a built-in identity such as `system:kube-controller-manager`. Both prefixes must be present
|
||
> in the same change that enables OIDC. Do not ship this phase partially.
|
||
|
||
Operational notes:
|
||
|
||
- Apply is a rolling control-plane machine-config change: one node at a time, wait for `Ready` and for
|
||
etcd quorum before proceeding to the next.
|
||
- The Talos admin certificate authenticates via client cert, not OIDC, and is unaffected. It is the
|
||
recovery path if the issuer URL is wrong or Authentik is down.
|
||
- The issuer must be reachable from the control-plane nodes and its TLS chain must be trusted by them.
|
||
Confirm this before applying, because a bad issuer URL means the apiserver logs OIDC discovery failures
|
||
on every start.
|
||
|
||
**Verify** (per node, after each rolls)
|
||
```bash
|
||
kubectl auth whoami --token="$ID_TOKEN"
|
||
# expect Username: authentik:rock
|
||
# Groups: authentik:homelab-admins, authentik:grafana-admins, system:authenticated
|
||
|
||
kubectl --kubeconfig cluster-config/kubeconfig get nodes # cert path still works
|
||
```
|
||
|
||
**Rollback:** revert the template change, re-run the pipeline, roll the control plane back. Cert-based
|
||
access is never interrupted, so this rollback is safe at any point.
|
||
|
||
---
|
||
|
||
### Phase 4 — Bind Authentik groups to Kubernetes roles
|
||
|
||
New manifest `k8s/infra/iam/rbac-oidc-bindings.yaml`, referenced from
|
||
[`k8s/infra/iam/kustomization.yaml`](k8s/infra/iam/kustomization.yaml):
|
||
|
||
| Subject | Role | Kind |
|
||
|---|---|---|
|
||
| `Group:authentik:homelab-admins` | `ClusterRole/cluster-admin` | ClusterRoleBinding |
|
||
| `Group:authentik:homelab-devops` | `ClusterRole/edit` | see [Open Decisions](#open-decisions) #2 |
|
||
| `Group:authentik:homelab-viewers` | `ClusterRole/view` | ClusterRoleBinding |
|
||
|
||
`admin`, `edit` and `view` are the built-in aggregated ClusterRoles and already exist in the cluster — no
|
||
custom roles needed.
|
||
|
||
**Verify**
|
||
```bash
|
||
kubectl auth can-i '*' '*' --all-namespaces --as=probe --as-group=authentik:homelab-admins # yes
|
||
kubectl auth can-i delete pods -A --as=probe --as-group=authentik:homelab-viewers # no
|
||
kubectl auth can-i get pods -n apps --as=probe --as-group=authentik:homelab-viewers # yes
|
||
kubectl auth can-i get secrets -n iam --as=probe --as-group=authentik:homelab-devops # no
|
||
```
|
||
|
||
**Rollback:** revert the commit; Argo CD prunes the bindings.
|
||
|
||
---
|
||
|
||
### Phase 5 — Client wiring and documentation
|
||
|
||
- Ship a kubeconfig template using the `kubelogin` (`kubectl oidc-login`) exec credential plugin — no
|
||
embedded certs, no long-lived token on disk.
|
||
- Document login, token cache location, and the break-glass cert path in
|
||
[`project-usage/authentik-oidc.md`](project-usage/authentik-oidc.md).
|
||
- Same edit corrects the stale group list from §1.3.
|
||
|
||
**Verify:** on a clean machine with no certificates, `kubectl get nodes` triggers a browser login and
|
||
succeeds; `kubectl auth whoami` reports the `authentik:`-prefixed identity.
|
||
|
||
---
|
||
|
||
### Phase 6 — Converge app RBAC on the same four groups
|
||
|
||
| Target | Change |
|
||
|---|---|
|
||
| Argo CD `policy.csv` | add `g, homelab-devops, role:admin` (or a scoped custom role) and `g, homelab-viewers, role:readonly` |
|
||
| Grafana `role_attribute_path` | three-tier: `grafana-admins`/`homelab-admins` → `Admin`, `homelab-devops` → `Editor`, else `Viewer` |
|
||
| MinIO `policy` claim expression | `homelab-admins` → `consoleAdmin`, `homelab-devops` → `readwrite`, else `readonly` |
|
||
| Argo CD local accounts | retire `accounts.rock` once SSO admin is proven. **Keep `accounts.cicd`** — the pipeline needs a non-SSO apiKey. |
|
||
|
||
**Verify:** end-to-end browser login per app as a test user who is *only* in `homelab-devops`, confirming
|
||
the expected role in each of Argo CD, Grafana and MinIO. The e2e suite under
|
||
[`tests/e2e/tests/authentik.spec.ts`](tests/e2e/tests/authentik.spec.ts) is the place to encode this.
|
||
|
||
**Rollback:** per-app revert; each app's mapping is independent.
|
||
|
||
---
|
||
|
||
## 4. Risks
|
||
|
||
| Risk | Severity | Mitigation |
|
||
|---|---|---|
|
||
| Missing `oidc-groups-prefix` → group-name privilege escalation to `cluster-admin` | Critical | Prefixes ship in the same commit as the OIDC flags; Phase 4 verification asserts the prefixed form |
|
||
| Bad issuer URL / untrusted TLS bricks apiserver OIDC | Medium | Cert-based admin access is unaffected; roll one control-plane node at a time and check apiserver logs before continuing |
|
||
| Authentik outage blocks all human cluster access | Medium | Talos admin cert is the documented break-glass path and never depends on Authentik |
|
||
| Argo CD `cicd` account removed by accident → pipeline loses cluster access | Medium | Phase 6 explicitly retires only `accounts.rock` |
|
||
| Group renamed in Authentik silently revokes cluster access | Low | Group names are asserted by the provisioner on every Argo CD sync, so drift self-heals |
|
||
|
||
---
|
||
|
||
## 5. Open Decisions
|
||
|
||
1. **Is kubectl-via-OIDC actually wanted?**
|
||
Phases 3–5 only pay for themselves if humans besides the shared admin cert need API access. If the
|
||
answer is no, the plan collapses to Phases 1 and 6: no Terraform change, no control-plane roll, app-level
|
||
SSO consolidation only.
|
||
|
||
2. **`homelab-devops` blast radius** — cluster-wide `ClusterRoleBinding` to `edit`, or namespaced
|
||
`RoleBinding`s limited to application namespaces (excluding `iam`, `kube-system`, `argocd`)?
|
||
Namespaced is the tighter default; cluster-wide is less to maintain.
|
||
|
||
3. **Username claim** — `preferred_username` (human-readable in audit logs, mutable) versus `sub`
|
||
(stable, opaque). Recommendation: `preferred_username`, given a single-operator homelab where audit
|
||
readability beats rename-safety.
|
||
|
||
---
|
||
|
||
## 6. Reference
|
||
|
||
- Kubernetes OIDC authentication: https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens
|
||
- Kubernetes RBAC: https://kubernetes.io/docs/reference/access-authn-authz/rbac/
|
||
- Talos apiserver `extraArgs`: https://www.talos.dev/latest/reference/configuration/v1alpha1/config/#Config.cluster.apiServer
|
||
- Authentik OAuth2 provider: https://docs.goauthentik.io/docs/add-secure-apps/providers/oauth2/
|
||
- Authentik property mappings (scope/claims): https://docs.goauthentik.io/docs/add-secure-apps/providers/property-mappings/
|
||
- kubelogin (`kubectl oidc-login`): https://github.com/int128/kubelogin
|
||
- Argo CD RBAC: https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/
|
||
- Grafana generic OAuth: https://grafana.com/docs/grafana/latest/setup-grafana/configure-security/configure-authentication/generic-oauth/
|
||
- MinIO OpenID identity management: https://min.io/docs/minio/linux/administration/identity-access-management/oidc-access-management.html
|