Files
homelab/k8s/applications/temporal/db-secret-sync/copy-job.yaml
T
Story Crater Bot 0a323fc039 fix(temporal): db-secret-sync image bitnami/kubectl:1.30 doesn't exist
Bitnami stopped publishing versioned image tags in 2025 - only 'latest' and
sha256-pinned digests remain for their free-tier images. Confirmed via
Docker Hub API before writing this fix: no '1.30' tag exists for
bitnami/kubectl, which caused an indefinite ImagePullBackOff (job stuck
'Running' with 0 pods able to start).

Switched to python:3.12-alpine + a stdlib urllib kubectl download, matching
the exact pattern already proven working in
k8s/security/iam/authentik-provision-job.yaml (which hit its own apk
permission problem on this same base image, now fixed the same way in
both places) - avoids depending on any third party's tagging policy.
2026-07-21 17:18:32 -07:00

135 lines
5.4 KiB
YAML

# Copies the CNPG-generated temporal-db-role Secret from the ddb namespace
# into the temporal namespace, as a plain (non-SOPS) k8s Secret with the same
# keys. Kubernetes Secrets are strictly namespace-scoped - a Deployment in
# `temporal` cannot reference a Secret living in `ddb` via secretKeyRef, and
# temporal-values.yaml's server.config.persistence.*.sql.existingSecret:
# temporal-db-role expects to find it in ITS OWN namespace (temporal).
#
# Deliberately a standalone directory (no kustomization.yaml) applied as its
# own small Application - avoids the k8s/applications/temporal/kustomization.yaml
# `namespace: temporal` transformer, which would silently force-rewrite this
# Job's ddb-scoped RoleBinding back to temporal (same class of bug fixed
# earlier in k8s/security/iam/kustomization.yaml - see that file's comments).
#
# PostSync (not PreSync!) + BeforeHookCreation: reruns on every ArgoCD sync
# of this app, re-copying the password if CNPG ever rotates it.
#
# IMPORTANT: this MUST be PostSync, not PreSync. The ServiceAccount/
# ClusterRole/RoleBindings below are plain (non-hook) resources - ArgoCD
# creates those during the normal "Sync" phase, which happens AFTER PreSync
# hooks run. A PreSync-hooked Job here would try to start before its own
# ServiceAccount exists (chicken-and-egg deadlock: confirmed live - the Job
# sat 'Running' for 14 minutes, unable to create any pod at all, event log
# showed "serviceaccount temporal/temporal-db-secret-sync not found" on
# every attempt). PostSync runs after this app's own normal resources are
# already applied, and this whole app (sync-wave 7) still fully completes
# before the `temporal` Application (sync-wave 8) begins, so the ordering
# guarantee we actually need (secret exists before Temporal's pods start)
# is preserved regardless of PreSync vs PostSync here.
apiVersion: v1
kind: ServiceAccount
metadata:
name: temporal-db-secret-sync
namespace: temporal
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: temporal-db-secret-sync
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list", "create", "update", "patch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: temporal-db-secret-sync
namespace: ddb
subjects:
- kind: ServiceAccount
name: temporal-db-secret-sync
namespace: temporal
roleRef:
kind: ClusterRole
name: temporal-db-secret-sync
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: temporal-db-secret-sync
namespace: temporal
subjects:
- kind: ServiceAccount
name: temporal-db-secret-sync
namespace: temporal
roleRef:
kind: ClusterRole
name: temporal-db-secret-sync
apiGroup: rbac.authorization.k8s.io
---
apiVersion: batch/v1
kind: Job
metadata:
name: temporal-db-secret-sync
namespace: temporal
annotations:
argocd.argoproj.io/hook: PostSync
argocd.argoproj.io/hook-delete-policy: BeforeHookCreation
spec:
ttlSecondsAfterFinished: 600
backoffLimit: 5
template:
spec:
serviceAccountName: temporal-db-secret-sync
restartPolicy: Never
securityContext:
runAsNonRoot: true
runAsUser: 1000
seccompProfile:
type: RuntimeDefault
containers:
- name: copy
# bitnami/kubectl:1.30 does NOT exist - Bitnami stopped publishing
# versioned tags in 2025 (only `latest` + sha256-pinned digests
# remain), confirmed live via Docker Hub API before this fix - the
# original tag caused an indefinite ImagePullBackOff. Using
# python:3.12-alpine + a stdlib urllib kubectl download instead,
# same pattern already proven working in
# k8s/security/iam/authentik-provision-job.yaml - avoids depending
# on any third party's tagging policy at all.
image: python:3.12-alpine
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
command:
- /bin/sh
- -c
- |
set -e
echo "installing kubectl (pure python urllib, no apk - see"
echo "authentik-provision-job.yaml for why apk fails as non-root)..."
python3 -c "
import urllib.request, os, stat
kver = urllib.request.urlopen('https://dl.k8s.io/release/stable.txt').read().decode().strip()
url = f'https://dl.k8s.io/release/{kver}/bin/linux/amd64/kubectl'
urllib.request.urlretrieve(url, '/tmp/kubectl')
st = os.stat('/tmp/kubectl')
os.chmod('/tmp/kubectl', st.st_mode | stat.S_IEXEC)
"
export PATH="/tmp:$PATH"
echo "waiting for ddb/temporal-db-role..."
until kubectl -n ddb get secret temporal-db-role >/dev/null 2>&1; do
echo " not ready yet, retrying..."
sleep 5
done
USERNAME=$(kubectl -n ddb get secret temporal-db-role -o jsonpath='{.data.username}' | base64 -d)
PASSWORD=$(kubectl -n ddb get secret temporal-db-role -o jsonpath='{.data.password}' | base64 -d)
kubectl -n temporal create secret generic temporal-db-role \
--from-literal=username="$USERNAME" \
--from-literal=password="$PASSWORD" \
--dry-run=client -o yaml | kubectl apply -f -
echo "synced temporal-db-role -> temporal namespace"