Commit Graph
147 Commits
Author SHA1 Message Date
Story Crater Bot f101b3381e fix(minio): add vault bucket to tenant spec 2026-07-21 14:58:48 -07:00
Story Crater Bot ef348d23f4 fix(vault): use minio service on port 80 (maps to 9000) 2026-07-21 14:52:24 -07:00
Story Crater Bot 04ec157c19 fix(vault): correct MinIO endpoint to minio-cluster-hl service 2026-07-21 14:46:54 -07:00
Story Crater Bot ded98329e5 Revert "fix(temporal): disable cassandra sub-chart and schema jobs, server uses PostgreSQL only"
This reverts commit d51056c684.
2026-07-21 14:04:57 -07:00
Story Crater Bot d51056c684 fix(temporal): disable cassandra sub-chart and schema jobs, server uses PostgreSQL only 2026-07-21 13:58:45 -07:00
Story Crater Bot c661d7eb77 fix(temporal): enable cassandra sub-chart with storage disabled, server uses PostgreSQL 2026-07-21 13:53:08 -07:00
Story Crater Bot edc12c388f fix(temporal): add minimal cassandra config stub to satisfy chart template 2026-07-21 13:47:40 -07:00
Story Crater Bot f0178b3bc5 fix(temporal): set cassandra.port even when disabled (chart requirement) 2026-07-21 13:44:25 -07:00
Story Crater Bot e82c4b36a4 fix(temporal): switch to PostgreSQL (CNPG ddb-cluster) instead of broken Cassandra/ES setup 2026-07-21 13:41:12 -07:00
Story Crater Bot 4ea25620dd fix(temporal): cassandra hosts as list (array) not string 2026-07-21 13:32:44 -07:00
Story Crater Bot 0588cb91b4 fix(temporal): scale elasticsearch to 1 replica (cluster constraint on single schedulable node) 2026-07-21 13:22:59 -07:00
Story Crater Bot 4bb99ef24f fix(minio): disable standalone console (use tenant built-in console instead) 2026-07-21 13:15:00 -07:00
Story Crater Bot fd07b3cff2 fix(sqs): add RBAC for temporalworkers resource 2026-07-21 13:07:42 -07:00
Story Crater Bot dc0bb63a01 fix(sqs): grant queue-operator deployments RBAC, install TemporalWorker CRD 2026-07-21 13:06:28 -07:00
Story Crater Bot b2191509fb fix(temporal): correct elasticsearch hostname to elasticsearch-master-headless 2026-07-21 12:54:14 -07:00
Story Crater Bot 5635482e0d fix(temporal): pin chart to v0.74.0 (keep original cassandra/ES config) 2026-07-21 12:43:52 -07:00
Story Crater Bot 328a713f4f Revert "fix(temporal): deploy Cassandra + Elasticsearch, pin chart to v0.74.0 (older version with sub-chart support)"
This reverts commit cc5325d905.
2026-07-21 12:42:16 -07:00
Story Crater Bot cc5325d905 fix(temporal): deploy Cassandra + Elasticsearch, pin chart to v0.74.0 (older version with sub-chart support) 2026-07-21 12:18:53 -07:00
Story Crater Bot 26f7da3610 fix(prometheus): drop ServerSideApply — conflicts with managedNamespaceMetadata forced ns apply, blocked all syncs; CRDs installed out-of-band 2026-07-21 11:26:54 -07:00
Story Crater Bot f2f4a2580f fix(prometheus): pin to az-a + longhorn-wffc SC — RWO PVC failed to attach on cp-2 (sole Longhorn node is cp-1) 2026-07-21 11:14:11 -07:00
Story Crater Bot 21e3987b11 fix(ingress): switch riotpiao-com-tls to letsencrypt-prod issuer
Wildcard cert was left on letsencrypt-staging; staging root is not
browser-trusted so HTTPS to *.riotpiao.com fails cert validation.
Switch issuerRef to letsencrypt-prod to issue a trusted wildcard.
2026-07-21 11:10:22 -07:00
Story Crater Bot 3e7238f71c fix(prometheus): scrapeTimeout must be <= scrapeInterval — authentik/nginx SMs (60s>30s) + global (60s>30s) blocked operator config gen, no Prometheus STS created 2026-07-21 11:08:23 -07:00
Story Crater Bot 88f8a764de fix(prometheus): set monitoring ns privileged via managedNamespaceMetadata — node-exporter hostNetwork/hostPID/hostPath blocked by baseline PSS 2026-07-21 11:05:04 -07:00
Story Crater Bot 34e996475f fix(promtail): set logging ns privileged via managedNamespaceMetadata — promtail hostPath/privileged/DAC_READ_SEARCH blocked by baseline PSS, DaemonSet created 0 pods 2026-07-21 11:03:58 -07:00
Story Crater Bot 3f4653ac56 fix(argocd): raise repo-server memory 512Mi->1Gi — OOMKilled under CMP+Helm rendering caused chronic restarts, not-ready endpoint, and cluster-wide sync 'no route to host' failures 2026-07-21 10:01:11 -07:00
Story Crater Bot 1dc6a2025f fix(kmsvc-redis): use bitnamilegacy/redis mirror + allowInsecureImages — docker.io/bitnami pulled version-pinned tags, ImagePullBackOff blocked redis + queue-operator 2026-07-21 09:47:19 -07:00
Story Crater Bot da925f3101 fix(forgejo-runner): add fsGroup 1000 so runner user can write /data/.runner — register hit permission denied on root-owned Longhorn PVC 2026-07-21 09:40:59 -07:00
Story Crater Bot 2443708abb chore(ci): refresh forgejo runner registration token — prior token invalid/expired 2026-07-21 09:38:15 -07:00
Story Crater Bot 9a34c12068 fix(forgejo-runner): point at in-cluster forgejo Service :3000 not public :443 — runner i/o timeout, forgejo serves 3000 not 443 2026-07-21 09:35:32 -07:00
Story Crater Bot f646bb06fd fix(minio,loki): declare loki-chunks/ruler/admin buckets in minio Tenant — loki failed with NoSuchBucket 2026-07-21 09:31:52 -07:00
Story Crater Bot 4363739d59 fix(loki,vault,iam): loki minio endpoint :80 not :9000, emit vault-minio-creds via CMP, drop redundant broken authentik-migrations job 2026-07-21 09:24:52 -07:00
Story Crater Bot 2bf543bba1 fix(ingress): add homelab-ingress ArgoCD app to apply orphaned ingress.yaml — services had no Ingress object, unreachable via LAN ingress .160 2026-07-21 09:15:58 -07:00
Story Crater Bot 6a2aacc4e6 feat(terraform): add per-node Cloudflare Tunnel cert SANs to controlplane certSANs — remote talosctl/kubectl over tunnel pass TLS verification
Adds optional cloudflare_talos_sans (machine.certSANs, talos API :50000) and
cloudflare_apiserver_sans (cluster.apiServer.certSANs, kube-apiserver :6443) per
control-plane node. cp-1 gets cp1.homelab + cp1-talos.homelab; cp-2/cp-3 get
their cpN-talos.homelab. Values set in gitignored tfvars.
2026-07-21 08:02:24 -07:00
Story Crater Bot 0471177250 chore(ci): add SOPS-encrypted runner-token secret record for forgejo-runner registration 2026-07-21 07:55:28 -07:00
Story Crater Bot 7fb73d6a4c fix(scheduling): pin portainer+forgejo-runner to az-a, add nodeSelector to runner chart template — WFFC alone insufficient with single Longhorn node (cp-1 only) 2026-07-20 23:51:46 -07:00
Story Crater Bot e0b24c83d0 fix(storage): add longhorn-wffc WaitForFirstConsumer default SC, repoint portainer/forgejo-runner — Immediate binding placed PVCs on non-storage nodes (cp-2/cp-3), attach failed 2026-07-20 23:49:01 -07:00
Story Crater Bot 9117fd777a fix(authentik): drop redundant authentik-migrate init container — server entrypoint migrates; old-image manage migrate tripped version-history precheck on empty DB 2026-07-20 23:40:08 -07:00
Story Crater Bot 5ce0b92186 feat(data): add CNPG managed roles + Database CRs for authentik/temporal — replaces missing helmfile post-sync user creation
authentik/temporal DB users+databases were never provisioned (old helmfile hook
gone; db-init-job only made schemas in shared app DB). Adds managed.roles
(authentik/temporal login roles, passwords from basic-auth secrets) + Database CRs
(dedicated DBs owned by each role). Role secrets applied out-of-band (SOPS), not in
kustomize resources so data-schemas app doesn't choke on ciphertext.
2026-07-20 23:36:21 -07:00
Story Crater Bot 89fa87f7c1 fix(sops-cmp): grafana-admin secret needs admin-user key too — chart existingSecret requires both user and password 2026-07-20 23:07:51 -07:00
Story Crater Bot 42cd0204fa fix(logging): pin grafana + loki to az-a (talos-cp-1) — sole Longhorn node, PVC fails to attach on cp-2/cp-3 2026-07-20 23:04:19 -07:00
Story Crater Bot 3a95f57b8f fix(loki): wire S3 creds from loki-s3-creds Secret via expand-env + extraEnvFrom — replaces empty helmfile-injected access keys 2026-07-20 23:00:51 -07:00
Story Crater Bot 2ec6eba9d2 fix(sops-cmp): correct loki s3 path (.loki.storage.s3), emit authentik-secrets separately, drop broken discover — merge via server/worker/migrate envFrom
Loki keys are under .loki.storage.s3 not .loki.s3 (returned null). Emit a separate
authentik-secrets Secret (not 'authentik', which the Helm chart owns) and merge it
via envFrom on server/worker/migrate. Remove discover fileName (caused MatchRepository
timeouts; app names the plugin explicitly).
2026-07-20 22:55:23 -07:00
Story Crater Bot d6f5b9ed69 feat(argocd): wire SOPS ConfigManagementPlugin properly — initContainer installs sops/yq, sidecar decrypts *.enc.yaml into app Secrets
Correct CMP setup (prior attempt used unsupported config): repoServer.initContainers
fetches sops v3.9.0 + yq v4.44.3 into a shared volume; repoServer.extraContainers
runs argocd-cmp-server with plugin.yaml from the sops-cmp-plugin ConfigMap, age key
from sops-age Secret. Plugin emits authentik/loki-s3-creds/grafana-admin/grafana-oidc
Secrets from decrypted enc files. sops-secrets Application (wave 0) uses the plugin at
repo root. Unblocks authentik/loki/grafana which were Degraded on missing secrets.
2026-07-20 13:13:47 -07:00
Story Crater Bot ce1fc4e296 fix(ingress-nginx): set privileged PodSecurity via managedNamespaceMetadata — hostPort 80/443 blocked by default baseline enforce, makes label permanent in IaC 2026-07-20 12:56:05 -07:00
Story Crater Bot d841bbdb95 feat(substrate): deploy cert-manager, ingress-nginx, reloader + LE staging issuers via app-of-apps — restores substrate ownership after Terraform removal
Substrate had no owner since Terraform was deleted (Pure GitOps). Adds 5 wave-0/1
Applications: cert-manager v1.21.0 (installCRDs, CP tolerations), ingress-nginx
4.15.1 (LB 192.168.1.160), reloader 2.2.14 at wave 0; LE ClusterIssuers +
*.riotpiao.com wildcard cert at wave 1 (DNS-01 via Cloudflare). Adds 3 chart
repos to AppProject sourceRepos and SOPS-encrypted cloudflare-api-token secret.
Cert starts on letsencrypt-staging; flip to prod after clean issue.
2026-07-20 12:50:02 -07:00
Story Crater Bot d9d2e34558 fix(minio): use configuration secret (config.env) for root creds, valid image tag — tenant now boots and authenticates
Switch Tenant from credsSecret to configuration field (v5 pods read config.env
shell exports); pin image to RELEASE.2025-07-23 (the old 2024-06 tag was pulled
from Docker Hub, ErrImagePull); drop prometheusOperator:true (made operator fail
reconcile hunting Prometheus in ns default). MinIO now serves S3, 4/4 drives OK,
root auth works. Operator's cosmetic 'empty tenant credentials' health-log is
harmless (documented inline).
2026-07-20 12:42:15 -07:00
Story Crater Bot 4cfac71a73 fix(minio): rewrite Tenant to operator-v5 schema, single-node pool, declarative buckets/users — removes invalid Bucket/Policy/User CRs and dead multi-site replication
Old Tenant used unknown v2 fields (pools[].size/storageClass, spec.console/metrics/ingress)
and referenced nonexistent minio.min.io/v1alpha1 Bucket/Policy/User kinds, so the app
never synced. Rewrites to valid v2: single erasure-coded pool (4 vols) pinned to
talos-cp-1/az-a (only schedulable+Longhorn node per 3-CP topology), spec.buckets +
spec.users declarative provisioning, prometheusOperator ServiceMonitor, features.domains.
Drops hand-rolled minio-service (operator owns it), dead multi-site replication job,
and legacy alias. Adds mc-based PostSync job for the ollama scoped policy, and
SOPS-encrypted minio-creds/oidc/user secrets for IaC record.
2026-07-20 12:27:44 -07:00
Story Crater Bot 7728f20d2b fix(argocd): allow operator.min.io in homelab AppProject sourceRepos — minio operator chart repo was blocked by allowlist 2026-07-20 12:18:45 -07:00
Story Crater Bot cea1a78a37 fix(minio): correct operator chart repoURL and pin version — charts.min.io lacks operator chart, use [email protected]
The operator chart moved to https://operator.min.io/; https://charts.min.io/ only
ships the standalone minio chart, causing 'chart operator not found in index'.
Pin to 5.0.18 (v5.x schema matches minio-operator-values.yaml operator.image.tag v5.0.0);
targetRevision '*' was fragile. Unblocks minio-tenant (needs operator CRDs).
2026-07-20 12:10:38 -07:00
Story Crater Bot abaea8823b refactor(argocd): simplify secrets approach — use directory source, manual Secrets for Stage 0
Reverts complex CMP plugin setup (helm chart doesn't support repoServer.extraContainers).
Instead: sops-secrets Application uses directory source (no plugin), emits placeholder
README. Manually-created Secrets (grafana-admin) live in target namespaces.

Full CMP plugin work deferred to future stage. Grafana values still wired to
admin.existingSecret (no-op until Secret exists, which it now does).

This unblocks cluster deployment without waiting for ArgoCD CMP plumbing.
2026-07-20 11:49:23 -07:00