ROOT CAUSE: Docker socket (/var/run/docker.sock) only existed inside the
dind container — the runner container couldn't see it. The runner connected
to dind via TCP (tcp://localhost:2376) with TLS. But workflow containers
created by the runner had NO way to access the docker daemon:
- unix socket not mounted (runner can't see it)
- DOCKER_HOST env var not passed (runner.envs not configured)
FIX: Share /var/run between dind and runner via emptyDir volume.
When dind starts, it creates /var/run/docker.sock in the shared volume.
Runner can now see the socket. docker_host: automount in config tells
the runner to mount the socket into job containers automatically.
Architecture after fix:
dind container → creates /var/run/docker.sock → shared emptyDir
runner container → sees /var/run/docker.sock → uses automount
workflow container → gets /var/run/docker.sock mounted by runner
Also removed runner.envs (TCP+TLS approach) — unix socket is simpler
and works with automount.
ROOT CAUSE: Workflow containers created by Forgejo runner don't inherit
the DOCKER_HOST/TLS env vars from the runner pod. Docker CLI defaults to
unix:///var/run/docker.sock which doesn't exist inside workflow containers.
The dind sidecar listens on tcp://localhost:2376 with TLS. With
network: host (already set), localhost inside the workflow container
reaches the dind daemon. But docker CLI needs DOCKER_HOST set explicitly.
FIX: Use runner.envs in config.yaml to pass these env vars to every
workflow container:
DOCKER_HOST=tcp://localhost:2376
DOCKER_TLS_VERIFY=1
DOCKER_CERT_PATH=/docker-certs/client
The valid_volumes already allows /docker-certs/client (TLS certs).
- Fix template to use .Values.gc.schedule instead of hardcoded cron
- Change schedule from daily 03:00 UTC to every 30 minutes
- Prevents DinD PVC fill-up (was at 93% before manual prune)
- Add gc-cronjob.yaml: daily prune of DinD Docker images/volumes/build-cache
and actcache across all forgejo-runner pods. Keeps :latest tagged images,
deletes non-latest older than 72h.
- Expand runner reg PVCs from 1Gi to 20Gi (all three runners) to prevent
action tool cache from filling disk.
- Rust runner gets explicit 20Gi persistence override.
- GC only renders from golang (default) values to avoid duplicate resources.
- Runner egress: allow 192.168.1.160/32:443. forgejo.riotpiao.com resolves to
the ingress LB, inside the 192.168.1.0/24 block the NetworkPolicy denies, so
docker push hung until timeout.
- dind CA: also mount homelab-ca at /etc/docker/certs.d/forgejo.riotpiao.com/,
the path dockerd actually reads for per-registry trust.
- Pull secret: dockerconfigjson for the api namespace; /v2/ answers 401.
- AppProject: allow the Forgejo repo as a source for api-gw.
rock
Story Crater Bot