85 lines
5.1 KiB
Markdown
85 lines
5.1 KiB
Markdown
|
# SSO Fix — What I'm Doing & Current State
|
||
|
|
|
||
|
|
## Goal
|
||
|
|
Every app's "Sign in with Authentik" was broken. Fix the root cause, make the provisioning
|
||
|
|
idempotent/re-runnable, and move the inline python out of YAML into real files.
|
||
|
|
|
||
|
|
## Root cause (found by replaying the OAuth2 flow, not just checking objects exist)
|
||
|
|
Authentik 2026.5.5 added a required **`grant_types`** field on OAuth2 providers. Our provision
|
||
|
|
script never set it → every provider had `grant_types = []` → `/authorize` returns
|
||
|
|
**`invalid_request` "Invalid grant_type for provider"** → **all** apps (grafana/minio/forgejo/argocd)
|
||
|
|
fail login identically. Objects (providers, apps, secrets, flows, redirect_uris) all existed and
|
||
|
|
looked correct, which is why earlier "does it exist" checks passed while SSO was 100% dead.
|
||
|
|
|
||
|
|
## Fixes made (committed + pushed to main)
|
||
|
|
1. **`grant_types: ["authorization_code", "refresh_token"]`** added to provider create + patch.
|
||
|
|
(commit `2461964`) — this is THE fix.
|
||
|
|
2. **Deprecated `ak_groups` → `groups`** in the custom groups-claim mapping. (commit `2461964`)
|
||
|
|
3. **Extract python** from the ConfigMap into `k8s/security/iam/scripts/authentik-provision.py`,
|
||
|
|
generated back via kustomize `configMapGenerator` (stable name, `disableNameSuffixHash: true`).
|
||
|
|
(commit `3d8a965`)
|
||
|
|
4. **App-list idempotency**: `get_or_create` on applications was POSTing (→ 400 "already exists")
|
||
|
|
because the applications LIST applies access-policy filtering — `count` was non-zero but the
|
||
|
|
`results` array was empty for the bootstrap user `akadmin` (not in `homelab-admins`). Added
|
||
|
|
`superuser_full_list=true` to the LIST query. (commit `3d8a965`)
|
||
|
|
5. **Don't PATCH existing applications**: the applications DETAIL endpoint (`PATCH /applications/{pk}/`)
|
||
|
|
also enforces the access policy and does **not** honor `superuser_full_list`, so it 404s for
|
||
|
|
`akadmin` once the `homelab-admins` binding exists. That 404 aborted the loop before all
|
||
|
|
providers got `grant_types`. Now it's find-or-create only (provider/launch_url are stable).
|
||
|
|
(commit `be2a56c`)
|
||
|
|
|
||
|
|
Net effect once it runs: the loop completes and patches `grant_types` onto **all four** providers.
|
||
|
|
So far only `grafana`'s provider got patched before each abort — `argocd/forgejo/minio` still `[]`.
|
||
|
|
|
||
|
|
## CURRENT BLOCKER (why it hasn't taken effect yet)
|
||
|
|
ArgoCD `iam-jobs` app is **stuck in a sync operation** (started 15:02:15) that is
|
||
|
|
`waiting for completion of hook batch/Job/authentik-provision`. That stale operation targets an
|
||
|
|
older revision and never completes, so:
|
||
|
|
- the updated **ConfigMap is still `OutOfSync`** (live cluster still runs the OLD script), and
|
||
|
|
- new commits (`be2a56c`) can't sync until the stuck op is cleared.
|
||
|
|
|
||
|
|
My `--subresource status` terminate + job delete didn't fully clear it (no `argocd` CLI available in
|
||
|
|
this shell to run `argocd app terminate-op`).
|
||
|
|
|
||
|
|
## To unblock (next action)
|
||
|
|
Clear the stuck operation, then sync to HEAD so the new ConfigMap + fixed hook run:
|
||
|
|
```bash
|
||
|
|
export KUBECONFIG=~/workplace/homelab/cluster-config/kubeconfig
|
||
|
|
# 1. cancel the stuck operation
|
||
|
|
kubectl -n argocd patch application iam-jobs --type merge --subresource status \
|
||
|
|
-p '{"status":{"operationState":{"phase":"Terminating"}}}'
|
||
|
|
# 2. delete any lingering hook job
|
||
|
|
kubectl -n iam delete job authentik-provision --ignore-not-found
|
||
|
|
# 3. hard refresh + full sync to HEAD (be2a56c)
|
||
|
|
kubectl -n argocd annotate application iam-jobs argocd.argoproj.io/refresh=hard --overwrite
|
||
|
|
kubectl -n argocd patch application iam-jobs --type merge \
|
||
|
|
-p '{"operation":{"initiatedBy":{"username":"manual"},"sync":{}}}'
|
||
|
|
```
|
||
|
|
If it stays stuck, use the ArgoCD UI (argocd.riotpiao.com) → iam-jobs → **Terminate** the running
|
||
|
|
sync, then **Sync**. (UI login itself needs the SSO fix — use local admin / `argocd` CLI if needed.)
|
||
|
|
|
||
|
|
## Verify the fix worked
|
||
|
|
```bash
|
||
|
|
SPOD=$(kubectl -n iam get pods --no-headers | grep authentik-server | grep Running | awk '{print $1}' | head -1)
|
||
|
|
TOKEN=$(kubectl -n iam get secret authentik-secrets -o jsonpath='{.data.AUTHENTIK_BOOTSTRAP_TOKEN}' | base64 -d)
|
||
|
|
# all four providers must show ['authorization_code','refresh_token']:
|
||
|
|
kubectl -n iam exec $SPOD -c server -- python3 -c "
|
||
|
|
import urllib.request,json
|
||
|
|
r=urllib.request.Request('http://localhost:9000/api/v3/providers/oauth2/?page_size=100',headers={'Authorization':'Bearer $TOKEN'})
|
||
|
|
[print(p['name'],p.get('grant_types')) for p in json.load(urllib.request.urlopen(r))['results']]"
|
||
|
|
```
|
||
|
|
Then in a browser: log into Authentik as `rock`, click each app tile → should land **logged-in**
|
||
|
|
(not an OAuth error page).
|
||
|
|
|
||
|
|
## Still TODO after SSO is green (from the approved plan in `homearr.md`)
|
||
|
|
- **B**: MinIO app-side OIDC env in `minio-tenant.yaml` (deployed tenant only sets `_SCOPES`).
|
||
|
|
- **C1**: Homarr landing page (official chart, SSO, Longhorn PVC).
|
||
|
|
- **C2**: Portainer OAuth via Portainer API job.
|
||
|
|
- **D**: `sso-verify` Job that replays the OAuth2 flow per app (would have caught this `grant_types`
|
||
|
|
bug that object-existence checks missed).
|
||
|
|
|
||
|
|
## Files changed so far
|
||
|
|
- new `k8s/security/iam/scripts/authentik-provision.py` (the real script)
|
||
|
|
- `k8s/security/iam/authentik-provision-job.yaml` (ConfigMap removed; SA/RBAC/Job kept)
|
||
|
|
- `k8s/security/iam/kustomization.yaml` (`configMapGenerator` + `disableNameSuffixHash`)
|