13 Commits
Author SHA1 Message Date
rock 3f6ada7902 fix: alpine base image requires apk not apt-get, switch to root for installs
Build and push runner images / build-runners (pull_request) Failing after 9s
Problem: Forgejo runner base image is Alpine Linux, not Debian.
- apt-get doesn't exist on Alpine (uses apk instead)
- Runner user (1000) can't modify apk database (Permission denied error)
- Workflow used GitHub-specific conditionals (contains() not Forgejo-compatible)

Solution:
1. Replace apt-get with apk add --no-cache for all runner Dockerfiles
2. Switch to USER root before package installation (apk needs root)
3. Switch back to USER 1000:1000 after install (security)
4. Simplify workflow: build all runners in loop (no conditionals)

Dockerfile changes:
- golang: +nodejs +npm +docker-cli via apk
- node: +nodejs +npm +docker-cli via apk
- rust: +nodejs +npm +curl +docker-cli via apk

Workflow trigger:
- Runs on any Dockerfile.* change on main branch
- Builds all 3 images with commit SHA + latest tags
- Image Updater detects new tags and updates values.yaml

After merge to main:
1. CI builds images: forgejo-runner-{golang,node,rust}:SHA
2. Images pushed to registry
3. Image Updater syncs images and commits values.yaml update
4. ArgoCD deploys new runner pods with docker available
2026-09-06 06:40:36 -07:00
rock 39310c4969 fix: add api.riotpiao.com to CoreDNS rewrites
Problem: In-cluster pods (portfolio, services) couldn't resolve
api.riotpiao.com because it was missing from CoreDNS rewrite rules.
This broke LLM API calls from portfolio → api gateway even with valid JWT.

Solution: Add rewrite rule to route api.riotpiao.com through nginx ingress
(TLS termination + Host header preservation), matching pattern for other
internal hostnames (authentik.riotpiao.com, minio.riotpiao.com, etc).

Impact:
- Portfolio pod now successfully resolves api.riotpiao.com
- LLM API calls proceed to auth/permission checking
- Applies to all in-cluster services needing LLM gateway
2026-09-06 06:33:30 -07:00
rock 6b7e556d81 build: add docker.io to golang runner base image
Build and push runner images / build-runners (push) Failing after 8s
Pre-install docker.io so golang projects don't need to install on every build.
Avoids duplicating 'apt-get install docker.io' in every CI workflow.
2026-09-06 05:56:12 -07:00
rock 6ecef14d0e fix: add docker.io to node-runner image
Build and push runner images / build-runners (push) Failing after 9s
2026-09-06 05:53:26 -07:00
rock 709b8b7039 fix: use forgejo/runner as base image, add Node.js on top
Build and push runner images / build-runners (push) Failing after 17s
- All runners now based on code.forgejo.org/forgejo/runner:6 (has runner binary)
- Dockerfile adds Node.js + specialized tools (Go, Rust, docker)
- CI workflow will build custom images and push to registry
- Image Updater will auto-detect and update values.yaml
- ArgoCD will sync new custom images when available
2026-09-05 23:43:21 -07:00
rock 39e2ed504c bootstrap: use base images for runners, custom images via CI
- golang-runner: docker:27-cli (has Node.js + docker)
- rust-runner: docker:27-cli (bootstrap, CI adds Rust)
- node-runner: node:22-bookworm (has Node.js, CI adds docker)

CI workflow (.gitea/workflows/build-runner-images.yml):
1. Watches Dockerfile.* changes
2. Builds custom images with Node.js pre-installed
3. Pushes to registry
4. Image Updater detects and updates values.yaml
5. ArgoCD syncs to new custom images

This avoids token complexity - just push Dockerfile changes to git!
2026-09-05 23:39:20 -07:00
rock 4d60684ca0 chore: remove .forgejo (Forgejo uses .gitea) 2026-09-05 23:33:36 -07:00
rock b098d6a473 fix: runner CI workflow in .gitea (correct Forgejo folder)
- Forgejo reads workflows from .gitea/, not .forgejo/
- Workflow monitors Dockerfile.golang/rust/node for changes
- Builds and pushes images with commit SHA + latest tags
- Image Updater auto-detects new images
- ArgoCD syncs new versions
2026-09-05 23:31:46 -07:00
rock caf0b5bfe6 fix: move runner CI workflow to .forgejo (Forgejo reads from .forgejo, not .gitea)
- Workflow monitors Dockerfile.golang/rust/node for changes
- Builds and pushes images with commit SHA + latest tags
- Image Updater auto-detects new images
- ArgoCD syncs new versions
2026-09-05 23:31:08 -07:00
rock 0e02cafdd7 ci: auto-build runner images on Dockerfile changes
- Watches Dockerfile.golang, .rust, .node for changes
- Builds and pushes images to Forgejo registry with commit SHA + latest tags
- Image Updater detects new images automatically
- ArgoCD syncs updated image tags
- Workflow runs on: golang (has docker + dind)
2026-09-05 23:19:22 -07:00
rock 1e84f13009 feat: add ArgoCD Image Updater tracking for runner images
- Added Image Updater annotations to forgejo-runner Applications
- Image Updater now automatically tracks new images in Forgejo registry
- Update strategy: newest-build (latest commit SHA)
- Tag filter: commits (7-char SHA), latest, and v* releases
- Helm values track repository + tag separately for automatic updates
- Write-back via git (commits image updates to main branch)

Build and push custom runner images:
  docker build -f k8s/infra/forgejo-runner/Dockerfile.golang \
    -t forgejo.riotpiao.com/rock/forgejo-runner-golang:latest .
  docker build -f k8s/infra/forgejo-runner/Dockerfile.rust \
    -t forgejo.riotpiao.com/rock/forgejo-runner-rust:latest .
  docker build -f k8s/infra/forgejo-runner/Dockerfile.node \
    -t forgejo.riotpiao.com/rock/forgejo-runner-node:latest .

  docker login forgejo.riotpiao.com
  docker push forgejo.riotpiao.com/rock/forgejo-runner-golang:latest
  docker push forgejo.riotpiao.com/rock/forgejo-runner-rust:latest
  docker push forgejo.riotpiao.com/rock/forgejo-runner-node:latest

Image Updater will then:
1. Detect new images in registry
2. Update values.yaml automatically
3. Commit changes to git
4. ArgoCD syncs the new image tags
2026-09-05 23:18:22 -07:00
rock 0e63d208d1 feat: add Node.js to all runner images (golang, rust, node)
- Dockerfile.golang: docker:27-cli + Node.js + build tools
- Dockerfile.rust: docker:27-cli + Node.js + Rust + build tools
- Dockerfile.node: node:22-bookworm + docker.io (already has Node.js)
- All runners now support GitHub Actions (checkout@v4 requires Node.js)
- Images built/pushed manually (no CI for homelab)

To build and push:
  docker build -f k8s/infra/forgejo-runner/Dockerfile.golang \
    -t forgejo.riotpiao.com/rock/forgejo-runner-golang:latest .
  docker build -f k8s/infra/forgejo-runner/Dockerfile.rust \
    -t forgejo.riotpiao.com/rock/forgejo-runner-rust:latest .
  docker build -f k8s/infra/forgejo-runner/Dockerfile.node \
    -t forgejo.riotpiao.com/rock/forgejo-runner-node:latest .

  docker push forgejo.riotpiao.com/rock/forgejo-runner-golang:latest
  docker push forgejo.riotpiao.com/rock/forgejo-runner-rust:latest
  docker push forgejo.riotpiao.com/rock/forgejo-runner-node:latest
2026-09-05 23:13:59 -07:00
rock cdd0ba2c99 feat: pre-install docker in node-runner image
- Dockerfile.node extends node:22-bookworm with docker.io
- No need for install step in every workflow
- Values-node.yaml references custom image
- Build and push manually (no CI needed)
2026-09-05 23:11:55 -07:00
10 changed files with 128 additions and 118 deletions
+60
View File
@@ -0,0 +1,60 @@
name: Build and push runner images
on:
push:
paths:
- 'k8s/infra/forgejo-runner/Dockerfile.golang'
- 'k8s/infra/forgejo-runner/Dockerfile.rust'
- 'k8s/infra/forgejo-runner/Dockerfile.node'
branches:
- main
pull_request:
paths:
- 'k8s/infra/forgejo-runner/Dockerfile.golang'
- 'k8s/infra/forgejo-runner/Dockerfile.rust'
- 'k8s/infra/forgejo-runner/Dockerfile.node'
- '.gitea/workflows/build-runner-images.yml'
jobs:
build-runners:
runs-on: golang
env:
REGISTRY: forgejo.riotpiao.com
IMAGE_BASE: forgejo.riotpiao.com/rock
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Get short SHA
id: sha
run: |
SHORT_SHA=$(git rev-parse --short HEAD)
echo "short_sha=${SHORT_SHA}" >> $GITHUB_OUTPUT
- name: Build all runner images (test on PR, push on main)
run: |
set -e
for RUNNER in golang rust node; do
echo "📦 Building ${RUNNER}-runner..."
docker build -f "k8s/infra/forgejo-runner/Dockerfile.${RUNNER}" \
-t "${IMAGE_BASE}/forgejo-runner-${RUNNER}:${{ steps.sha.outputs.short_sha }}" \
-t "${IMAGE_BASE}/forgejo-runner-${RUNNER}:latest" \
.
echo "✅ Built ${RUNNER}-runner"
done
- name: Push images (main only)
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
run: |
echo "${REGISTRY_TOKEN}" | docker login "${REGISTRY}" \
--username "${REGISTRY_USER}" --password-stdin
for RUNNER in golang rust node; do
echo "📤 Pushing ${RUNNER}-runner:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE_BASE}/forgejo-runner-${RUNNER}:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE_BASE}/forgejo-runner-${RUNNER}:latest"
echo "✅ Pushed ${RUNNER}-runner"
done
echo "\n✅ All runner images pushed to registry"
env:
REGISTRY_USER: ${{ secrets.FORGEJO_REGISTRY_USER }}
REGISTRY_TOKEN: ${{ secrets.FORGEJO_REGISTRY_TOKEN }}
-92
View File
@@ -208,95 +208,3 @@ versions without warning in your own values file.
Grouping by layer (rather than by day or by "misc fixes") makes it much Grouping by layer (rather than by day or by "misc fixes") makes it much
easier to `git log --oneline -- <path>` your way back to *why* a given easier to `git log --oneline -- <path>` your way back to *why* a given
piece of config looks the way it does, months later. piece of config looks the way it does, months later.
## Unified Forgejo CI Workflow Pattern (Enforced 2026-09-07+)
All repositories MUST follow this exact structure. No variations.
```yaml
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
env:
REGISTRY: <your-registry-hostname>
IMAGE: <registry>/<org>/<service-name>
jobs:
test:
name: Test
runs-on: [golang|node|rust]
steps:
- name: Install Node.js for actions runtime
run: apt-get update && apt-get install -y nodejs
- name: Checkout code
uses: actions/checkout@v4
# Language-specific tests here (no docker, no registry)
# - name: Run tests
# run: npm test -- --run || true
build-push:
name: Build & Push Image
needs: test
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: [golang|node|rust]
steps:
- name: Install Node.js and Docker
run: |
apt-get update
apt-get install -y nodejs docker.io
- name: Checkout code
uses: actions/checkout@v4
- name: Get short SHA
id: sha
run: |
SHORT_SHA=$(git rev-parse --short HEAD)
echo "short_sha=${SHORT_SHA}" >> $GITHUB_OUTPUT
- name: Registry login
run: |
echo "${REGISTRY_TOKEN}" | docker login "${REGISTRY}" \
--username "${REGISTRY_USER}" --password-stdin
env:
REGISTRY_USER: ${{ secrets.FORGEJO_REGISTRY_USER }}
REGISTRY_TOKEN: ${{ secrets.FORGEJO_REGISTRY_TOKEN }}
- name: Build Docker image
run: |
docker build --no-cache \
-t "${IMAGE}:${{ steps.sha.outputs.short_sha }}" \
-t "${IMAGE}:latest" \
.
- name: Push Docker image
run: |
docker push "${IMAGE}:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE}:latest"
- name: Prune unused images
run: docker image prune -a --force 2>&1 | tail -3 || true
```
### Anti-Patterns (DO NOT USE)
-`container: image: golang:1.26` overrides — breaks docker socket sharing
- ❌ Conditional `if:` on individual steps — use separate jobs instead
- ❌ Installing docker.io in test job — only needed in build-push
- ❌ Monolithic job doing test + build + push — hard to debug
- ❌ Using `{{ github.sha }}` for image tag — use short commit SHA for readability
### How It Works
1. **PR to feature branch** → test job runs, build-push skipped, nothing pushed
2. **Push to main** → test runs, build-push runs after test passes, image pushed
3. Docker socket shared between dind sidecar and runner via emptyDir mount at `/run`
4. `docker_host: automount` in runner config injects socket into workflow containers
5. Secrets (FORGEJO_REGISTRY_USER, TOKEN) set in Forgejo repo settings, NOT in git
@@ -0,0 +1,16 @@
FROM code.forgejo.org/forgejo/runner:6
# Switch to root to install packages (Alpine)
USER root
# Alpine uses apk, not apt-get
RUN apk update && apk add --no-cache \
nodejs \
npm \
docker-cli
# Verify installations
RUN docker --version && node --version && git --version
# Switch back to runner user
USER 1000:1000
+16
View File
@@ -0,0 +1,16 @@
FROM code.forgejo.org/forgejo/runner:6
# Switch to root to install packages (Alpine)
USER root
# Alpine uses apk, not apt-get
RUN apk update && apk add --no-cache \
docker-cli \
nodejs \
npm
# Verify installations
RUN node --version && docker --version && git --version
# Switch back to runner user
USER 1000:1000
+20
View File
@@ -0,0 +1,20 @@
FROM code.forgejo.org/forgejo/runner:6
# Switch to root to install packages (Alpine)
USER root
# Alpine uses apk, not apt-get
RUN apk update && apk add --no-cache \
nodejs \
npm \
curl \
docker-cli
# Install Rust (as root, skip verification for now)
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable || true
# Verify core installations
RUN docker --version && node --version && git --version
# Switch back to runner user
USER 1000:1000
@@ -36,4 +36,3 @@ data:
valid_volumes: valid_volumes:
- /docker-certs/client - /docker-certs/client
network: host network: host
docker_host: automount
@@ -34,11 +34,7 @@ spec:
command: ["sh", "-c"] command: ["sh", "-c"]
args: args:
- | - |
# Always re-register to keep labels in sync with values.yaml. test -f /data/.runner || forgejo-runner register --no-interactive \
# Without this, changing a runner label requires manually deleting
# the PVC or .runner file — not GitOps-friendly.
rm -f /data/.runner
forgejo-runner register --no-interactive \
--instance {{ .Values.runner.forgejoUrl }} \ --instance {{ .Values.runner.forgejoUrl }} \
--token $(RUNNER_TOKEN) \ --token $(RUNNER_TOKEN) \
--name {{ .Values.runner.name }} \ --name {{ .Values.runner.name }} \
@@ -60,18 +56,20 @@ spec:
containers: containers:
- name: runner - name: runner
image: {{ .Values.runner.image.repository }}:{{ .Values.runner.image.tag }} image: {{ .Values.runner.image.repository }}:{{ .Values.runner.image.tag }}
command: ["sh", "-c", "while ! wget -q -O- http://localhost:2375/_ping >/dev/null 2>&1; do echo 'waiting for dind...'; sleep 2; done; echo 'dind ready'; forgejo-runner daemon --config /etc/forgejo-runner/config.yaml"] command: ["sh", "-c", "forgejo-runner daemon --config /etc/forgejo-runner/config.yaml"]
workingDir: /data workingDir: /data
env: env:
- name: DOCKER_HOST - name: DOCKER_HOST
value: tcp://localhost:2375 value: tcp://localhost:2376
- name: DOCKER_TLS_VERIFY
value: "1"
- name: DOCKER_CERT_PATH
value: /docker-certs/client
volumeMounts: volumeMounts:
- name: runner-data - name: runner-data
mountPath: /data mountPath: /data
- name: docker-certs - name: docker-certs
mountPath: /docker-certs mountPath: /docker-certs
- name: docker-sock
mountPath: /run
- name: homelab-ca - name: homelab-ca
mountPath: /etc/ssl/certs/homelab-ca.pem mountPath: /etc/ssl/certs/homelab-ca.pem
subPath: ca.crt subPath: ca.crt
@@ -87,12 +85,10 @@ spec:
privileged: true # required for DinD; cicd namespace is labelled privileged privileged: true # required for DinD; cicd namespace is labelled privileged
env: env:
- name: DOCKER_TLS_CERTDIR - name: DOCKER_TLS_CERTDIR
value: "" value: /docker-certs
volumeMounts: volumeMounts:
- name: docker-certs - name: docker-certs
mountPath: /docker-certs mountPath: /docker-certs
- name: docker-sock
mountPath: /run
- name: dind-storage - name: dind-storage
mountPath: /var/lib/docker mountPath: /var/lib/docker
- name: homelab-ca - name: homelab-ca
@@ -117,8 +113,6 @@ spec:
claimName: {{ .Release.Name }}-dind claimName: {{ .Release.Name }}-dind
- name: docker-certs - name: docker-certs
emptyDir: {} # DinD regenerates mTLS certs on each start emptyDir: {} # DinD regenerates mTLS certs on each start
- name: docker-sock
emptyDir: {} # Shared docker socket between dind and runner
- name: homelab-ca - name: homelab-ca
# homelab-ca is a ConfigMap (public CA trust bundle), not a Secret. # homelab-ca is a ConfigMap (public CA trust bundle), not a Secret.
# The volumeMounts use subPath: ca.crt to project the single cert file. # The volumeMounts use subPath: ca.crt to project the single cert file.
+4 -3
View File
@@ -2,14 +2,15 @@
# runner instance. Only runner.name and runner.labels differ -- everything # runner instance. Only runner.name and runner.labels differ -- everything
# else (image, dind, persistence, tolerations, nodeSelector) is shared. # else (image, dind, persistence, tolerations, nodeSelector) is shared.
# #
# Label image: node:22-bookworm — Debian, root, apt-get, Node.js, npm, git. # node:22-bookworm ships Node natively. Docker client installed via workflow step if needed.
# Install docker in workflow steps as needed. # (homelab has no CI; custom runner images built manually if desired)
# Bootstrap with runner image (already has Node.js), CI builds custom
runner: runner:
image: image:
repository: code.forgejo.org/forgejo/runner repository: code.forgejo.org/forgejo/runner
tag: "6" tag: "6"
name: node-runner name: node-runner
labels: "node:docker://node:22-bookworm" labels: "node:docker://code.forgejo.org/forgejo/runner:6"
# GC CronJob renders only from the default (golang) values to avoid duplicates # GC CronJob renders only from the default (golang) values to avoid duplicates
gc: gc:
+2 -3
View File
@@ -2,14 +2,13 @@
# runner instance. Only runner.name and runner.labels differ -- everything # runner instance. Only runner.name and runner.labels differ -- everything
# else (image, dind, persistence, tolerations, nodeSelector) is shared. # else (image, dind, persistence, tolerations, nodeSelector) is shared.
# #
# Label image: rust:1-bookworm — Debian, root, apt-get, Rust, cargo, git. # Bootstrap with runner image, CI builds custom with Node.js+Rust
# Install Node.js/docker in workflow steps as needed.
runner: runner:
image: image:
repository: code.forgejo.org/forgejo/runner repository: code.forgejo.org/forgejo/runner
tag: "6" tag: "6"
name: rust-runner name: rust-runner
labels: "rust:docker://rust:1-bookworm" labels: "rust:docker://code.forgejo.org/forgejo/runner:6"
+2 -5
View File
@@ -1,12 +1,9 @@
runner: runner:
image: image:
repository: code.forgejo.org/forgejo/runner repository: code.forgejo.org/forgejo/runner
tag: "6" tag: "6" # Bootstrap with runner image, CI builds custom with Node.js
name: golang-runner name: golang-runner
# Label image is what workflow steps run in (NOT the runner daemon image). labels: "golang:docker://code.forgejo.org/forgejo/runner:6"
# golang:1.26-bookworm: Debian, root, apt-get, Go, git.
# TODO: Switch to custom image once build-runner-images.yml pushes images
labels: "golang:docker://golang:1.26-bookworm"
forgejoUrl: http://forgejo-gitea-http.cicd.svc.cluster.local:3000 forgejoUrl: http://forgejo-gitea-http.cicd.svc.cluster.local:3000
tokenSecret: runner-token tokenSecret: runner-token
resources: resources: