# SSO Fix — What I'm Doing & Current State

## Goal
Every app's "Sign in with Authentik" was broken. Fix the root cause, make the provisioning
idempotent/re-runnable, and move the inline python out of YAML into real files.

## Root cause (found by replaying the OAuth2 flow, not just checking objects exist)
Authentik 2026.5.5 added a required **`grant_types`** field on OAuth2 providers. Our provision
script never set it → every provider had `grant_types = []` → `/authorize` returns
**`invalid_request` "Invalid grant_type for provider"** → **all** apps (grafana/minio/forgejo/argocd)
fail login identically. Objects (providers, apps, secrets, flows, redirect_uris) all existed and
looked correct, which is why earlier "does it exist" checks passed while SSO was 100% dead.

## Fixes made (committed + pushed to main)
1. **`grant_types: ["authorization_code", "refresh_token"]`** added to provider create + patch.
   (commit `2461964`) — this is THE fix.
2. **Deprecated `ak_groups` → `groups`** in the custom groups-claim mapping. (commit `2461964`)
3. **Extract python** from the ConfigMap into `k8s/security/iam/scripts/authentik-provision.py`,
   generated back via kustomize `configMapGenerator` (stable name, `disableNameSuffixHash: true`).
   (commit `3d8a965`)
4. **App-list idempotency**: `get_or_create` on applications was POSTing (→ 400 "already exists")
   because the applications LIST applies access-policy filtering — `count` was non-zero but the
   `results` array was empty for the bootstrap user `akadmin` (not in `homelab-admins`). Added
   `superuser_full_list=true` to the LIST query. (commit `3d8a965`)
5. **Don't PATCH existing applications**: the applications DETAIL endpoint (`PATCH /applications/{pk}/`)
   also enforces the access policy and does **not** honor `superuser_full_list`, so it 404s for
   `akadmin` once the `homelab-admins` binding exists. That 404 aborted the loop before all
   providers got `grant_types`. Now it's find-or-create only (provider/launch_url are stable).
   (commit `be2a56c`)

Net effect once it runs: the loop completes and patches `grant_types` onto **all four** providers.
So far only `grafana`'s provider got patched before each abort — `argocd/forgejo/minio` still `[]`.

## CURRENT BLOCKER (why it hasn't taken effect yet)
ArgoCD `iam-jobs` app is **stuck in a sync operation** (started 15:02:15) that is
`waiting for completion of hook batch/Job/authentik-provision`. That stale operation targets an
older revision and never completes, so:
- the updated **ConfigMap is still `OutOfSync`** (live cluster still runs the OLD script), and
- new commits (`be2a56c`) can't sync until the stuck op is cleared.

My `--subresource status` terminate + job delete didn't fully clear it (no `argocd` CLI available in
this shell to run `argocd app terminate-op`).

## To unblock (next action)
Clear the stuck operation, then sync to HEAD so the new ConfigMap + fixed hook run:
```bash
export KUBECONFIG=~/workplace/homelab/cluster-config/kubeconfig
# 1. cancel the stuck operation
kubectl -n argocd patch application iam-jobs --type merge --subresource status \
  -p '{"status":{"operationState":{"phase":"Terminating"}}}'
# 2. delete any lingering hook job
kubectl -n iam delete job authentik-provision --ignore-not-found
# 3. hard refresh + full sync to HEAD (be2a56c)
kubectl -n argocd annotate application iam-jobs argocd.argoproj.io/refresh=hard --overwrite
kubectl -n argocd patch application iam-jobs --type merge \
  -p '{"operation":{"initiatedBy":{"username":"manual"},"sync":{}}}'
```
If it stays stuck, use the ArgoCD UI (argocd.riotpiao.com) → iam-jobs → **Terminate** the running
sync, then **Sync**. (UI login itself needs the SSO fix — use local admin / `argocd` CLI if needed.)

## Verify the fix worked
```bash
SPOD=$(kubectl -n iam get pods --no-headers | grep authentik-server | grep Running | awk '{print $1}' | head -1)
TOKEN=$(kubectl -n iam get secret authentik-secrets -o jsonpath='{.data.AUTHENTIK_BOOTSTRAP_TOKEN}' | base64 -d)
# all four providers must show ['authorization_code','refresh_token']:
kubectl -n iam exec $SPOD -c server -- python3 -c "
import urllib.request,json
r=urllib.request.Request('http://localhost:9000/api/v3/providers/oauth2/?page_size=100',headers={'Authorization':'Bearer $TOKEN'})
[print(p['name'],p.get('grant_types')) for p in json.load(urllib.request.urlopen(r))['results']]"
```
Then in a browser: log into Authentik as `rock`, click each app tile → should land **logged-in**
(not an OAuth error page).

## Still TODO after SSO is green (from the approved plan in `homearr.md`)
- **B**: MinIO app-side OIDC env in `minio-tenant.yaml` (deployed tenant only sets `_SCOPES`).
- **C1**: Homarr landing page (official chart, SSO, Longhorn PVC).
- **C2**: Portainer OAuth via Portainer API job.
- **D**: `sso-verify` Job that replays the OAuth2 flow per app (would have caught this `grant_types`
  bug that object-existence checks missed).

## Files changed so far
- new `k8s/security/iam/scripts/authentik-provision.py` (the real script)
- `k8s/security/iam/authentik-provision-job.yaml` (ConfigMap removed; SA/RBAC/Job kept)
- `k8s/security/iam/kustomization.yaml` (`configMapGenerator` + `disableNameSuffixHash`)