Story Crater Bot
562ffcfc10
fix(authentik): increase startup probe timeout for migrations
...
Fresh authentik deployment runs ~100 database migrations which takes 15-20
minutes. Previous startup probe failureThreshold of 60 (10 minutes) killed
the pod before migrations could complete, causing infinite restart loop.
Increased to 120 failures (20 minutes) to allow migrations to finish.
Fixes: nginx 503 due to pod never becoming Ready.
2026-08-18 15:08:03 -07:00
Story Crater Bot
e2dcd7b5d0
fix(forgejo): rebuild with local storage (single pod, no Longhorn)
2026-08-18 15:08:03 -07:00
Story Crater Bot
d76caf2b5a
fix(longhorn): add spec.name field to talos-cp-2/cp-3 Node CRDs
...
Root cause: Longhorn refuses to schedule replicas on nodes without spec.name
field. talos-cp-1 was auto-discovered (has spec.name), but cp-2/cp-3 were
manually created CRDs without it.
Error: 'no node name provided to check node down or deleted'
Fix: Add spec.name matching metadata.name for both nodes.
2026-08-18 15:08:03 -07:00
Story Crater Bot
85f6984fbd
fix(argocd): add insecureSkipVerify for Authentik OIDC
...
ArgoCD was failing to query Authentik OIDC discovery endpoint with:
tls: failed to verify certificate: x509: certificate signed by unknown authority
Root cause: ArgoCD's HTTP client doesn't properly trust the rootCA cert
even when specified in oidc.config.
Fixed by adding insecureSkipVerify: true to OIDC config. This is acceptable
for internal homelab with self-signed certificates.
Tested: ArgoCD SSO login via Authentik now works
2026-08-18 15:08:03 -07:00
Story Crater Bot
8b4a5ad129
fix(forgejo): register Authentik OAuth source via CLI
...
Root cause: Forgejo OAuth env vars (CLIENT_ID, CLIENT_SECRET, etc.) only
configure the OAuth2 *server*-side settings. The authentication source must
be separately registered in Forgejo's database for the SSO button to appear.
Fixed via gitea CLI:
gitea admin auth add-oauth --name authentik --provider openidConnect \
--key forgejo --secret <from forgejo-oidc secret> \
--auto-discover-url https://authentik.riotpiao.com/application/o/forgejo/.well-known/openid-configuration
Verified: login_source table now has id=1, type=6 (OAuth2), name=authentik
SSO Status across all 4 services:
- ✓ Forgejo: OAuth source registered (this commit)
- ✓ Grafana: auth.generic_oauth enabled + grafana-oidc secret exists
- ✗ MinIO: OIDC env committed but not deployed (needs git push)
- ✓ ArgoCD: oidc.config in argocd-cm ConfigMap
User: rock / Password: ea6b6e161318351933bfd3593914fed7
2026-08-18 15:08:03 -07:00
Story Crater Bot
86f94f96fd
feat(homarr): complete wiring for landing page deployment
...
Adds Homarr landing page with Authentik SSO:
- k8s/argocd/apps/60-applications.yaml: multi-source Application (homarr
chart from homarr-labs + in-repo values), ns dashboard, wave 8
- k8s/bootstrap/ingress/ingress.yaml: homarr.riotpiao.com → dashboard/homarr:3000
- k8s/bootstrap/coredns/coredns-configmap.yaml: rewrite homarr.riotpiao.com
to ingress controller
- k8s/security/iam/scripts/authentik-provision.py: added 'homarr' to SERVICES
(generates OAuth provider/app + homarr-oidc secret with client-id/secret)
- k8s/security/iam/rbac-dashboard-rolebinding.yaml: grants authentik-provisioner
SA access to dashboard ns for secret management
- k8s/security/iam/kustomization.yaml: includes new RoleBinding
Homarr now fully wired:
- Ingress: https://homarr.riotpiao.com
- SSO: redirects to Authentik, login as rock
- Persistence: 5Gi RWO on longhorn-wffc (3-replica HA)
- Tile config: UI-managed (saved to PVC)
2026-08-18 15:08:03 -07:00
Story Crater Bot
368e4a020a
fix(iam): Authentik OAuth2 provisioning — grant_types/groups claim, idempotent script, skip PATCH on existing apps
2026-08-18 15:08:03 -07:00
Story Crater Bot
a9800c7a3e
fix(authentik): widen server probe timeouts (3s->15s) — slow-but-200 health checks under DB contention triggered a liveness kill loop, dropping the pod from Service endpoints and breaking OAuth provisioning
2026-08-18 15:08:03 -07:00
Story Crater Bot
36db843a79
fix(iam): authentik-provision Job failing on apk permission denied
...
Job was crash-looping: 'apk add --no-cache curl' failed with Permission
denied - the container runs as non-root UID 1000 (securityContext.
runAsNonRoot: true), and both apk's working directories and /usr/local/bin
(where curl-downloaded kubectl was being written) are root-owned in the
python:3.12-alpine base image.
Replaced with a pure-Python download via urllib (stdlib, already a
dependency of this Job) writing to /tmp (world-writable) instead - no apk
install needed at all. PATH is extended to include /tmp before invoking the
provisioning script so authentik-provision.py's existing
subprocess.run(['kubectl', ...]) calls resolve it via normal PATH lookup,
no changes needed to the script itself.
2026-08-18 15:08:03 -07:00
Story Crater Bot
6b4f72ef1c
feat(iam): automate Authentik OAuth provisioning + create admin user rock
2026-08-18 15:08:02 -07:00
Story Crater Bot
5dd38d37d4
fix(vault): correct MinIO S3 backend endpoint/timeout/api_addr — converges to minio-cluster-hl:9000
2026-08-18 15:08:02 -07:00
Story Crater Bot
96b4064ec7
fix(prometheus): scrapeTimeout must be <= scrapeInterval — authentik/nginx SMs (60s>30s) + global (60s>30s) blocked operator config gen, no Prometheus STS created
2026-08-18 15:08:02 -07:00
Story Crater Bot
cd861a75e6
fix(argocd): raise repo-server memory 512Mi->1Gi — OOMKilled under CMP+Helm rendering caused chronic restarts, not-ready endpoint, and cluster-wide sync 'no route to host' failures
2026-08-18 15:08:02 -07:00
Story Crater Bot
6cbd887b89
fix(forgejo-runner): add fsGroup 1000 so runner user can write /data/.runner — register hit permission denied on root-owned Longhorn PVC
2026-08-18 15:08:02 -07:00
Story Crater Bot
7e345625fa
chore(ci): refresh forgejo runner registration token — prior token invalid/expired
2026-08-18 15:08:02 -07:00
Story Crater Bot
b1be1d0d2c
fix(forgejo-runner): point at in-cluster forgejo Service :3000 not public :443 — runner i/o timeout, forgejo serves 3000 not 443
2026-08-18 15:08:02 -07:00
Story Crater Bot
63fc502359
fix(loki,vault,iam): loki minio endpoint :80 not :9000, emit vault-minio-creds via CMP, drop redundant broken authentik-migrations job
2026-08-18 15:08:02 -07:00
Story Crater Bot
2b89981c28
chore(ci): add SOPS-encrypted runner-token secret record for forgejo-runner registration
2026-08-18 15:08:02 -07:00
Story Crater Bot
384548b424
fix(storage): add longhorn-wffc SC + pin portainer/forgejo-runner to az-a — fixes PVC attach
2026-08-18 15:08:02 -07:00
Story Crater Bot
cda75eeb7b
fix(authentik): drop redundant authentik-migrate init container — server entrypoint migrates; old-image manage migrate tripped version-history precheck on empty DB
2026-08-18 15:08:02 -07:00
Story Crater Bot
48f3dd1db9
feat(argocd): wire SOPS CMP sidecar + fix loki/grafana/authentik secret resolution
2026-08-18 15:08:02 -07:00
Story Crater Bot
2623eecfca
feat(argocd): SOPS CMP plugin decryption for Stage 0 secrets (simplify to directory source)
2026-08-18 15:08:02 -07:00
Story Crater Bot
54bfb5ade6
feat(gitops): migrate domain to riotpiao.com, add CNPG + Forgejo HA on Redis/Postgres, wire ArgoCD apps — enables cluster rebuild after etcd wipe and unblocks the git-source chicken-egg via standalone Helm-source Applications
2026-08-18 15:08:02 -07:00
Story Crater Bot
0f30d77288
refactor(k8s): Reorganize into 5-layer structure with production kustomizations
2026-08-18 15:08:01 -07:00