13 Commits
Author SHA1 Message Date
rock e9e5221c63 fix: alpine base image requires apk not apt-get, switch to root for installs
Problem: Forgejo runner base image is Alpine Linux, not Debian.
- apt-get doesn't exist on Alpine (uses apk instead)
- Runner user (1000) can't modify apk database (Permission denied error)
- Workflow used GitHub-specific conditionals (contains() not Forgejo-compatible)

Solution:
1. Replace apt-get with apk add --no-cache for all runner Dockerfiles
2. Switch to USER root before package installation (apk needs root)
3. Switch back to USER 1000:1000 after install (security)
4. Simplify workflow: build all runners in loop (no conditionals)

Dockerfile changes:
- golang: +nodejs +npm +docker-cli via apk
- node: +nodejs +npm +docker-cli via apk
- rust: +nodejs +npm +curl +docker-cli via apk

Workflow trigger:
- Runs on any Dockerfile.* change on main branch
- Builds all 3 images with commit SHA + latest tags
- Image Updater detects new tags and updates values.yaml

After merge to main:
1. CI builds images: forgejo-runner-{golang,node,rust}:SHA
2. Images pushed to registry
3. Image Updater syncs images and commits values.yaml update
4. ArgoCD deploys new runner pods with docker available
2026-09-06 06:33:30 -07:00
rock 39310c4969 fix: add api.riotpiao.com to CoreDNS rewrites
Problem: In-cluster pods (portfolio, services) couldn't resolve
api.riotpiao.com because it was missing from CoreDNS rewrite rules.
This broke LLM API calls from portfolio → api gateway even with valid JWT.

Solution: Add rewrite rule to route api.riotpiao.com through nginx ingress
(TLS termination + Host header preservation), matching pattern for other
internal hostnames (authentik.riotpiao.com, minio.riotpiao.com, etc).

Impact:
- Portfolio pod now successfully resolves api.riotpiao.com
- LLM API calls proceed to auth/permission checking
- Applies to all in-cluster services needing LLM gateway
2026-09-06 06:33:30 -07:00
rock 6b7e556d81 build: add docker.io to golang runner base image
Build and push runner images / build-runners (push) Failing after 8s
Pre-install docker.io so golang projects don't need to install on every build.
Avoids duplicating 'apt-get install docker.io' in every CI workflow.
2026-09-06 05:56:12 -07:00
rock 6ecef14d0e fix: add docker.io to node-runner image
Build and push runner images / build-runners (push) Failing after 9s
2026-09-06 05:53:26 -07:00
rock 709b8b7039 fix: use forgejo/runner as base image, add Node.js on top
Build and push runner images / build-runners (push) Failing after 17s
- All runners now based on code.forgejo.org/forgejo/runner:6 (has runner binary)
- Dockerfile adds Node.js + specialized tools (Go, Rust, docker)
- CI workflow will build custom images and push to registry
- Image Updater will auto-detect and update values.yaml
- ArgoCD will sync new custom images when available
2026-09-05 23:43:21 -07:00
rock 39e2ed504c bootstrap: use base images for runners, custom images via CI
- golang-runner: docker:27-cli (has Node.js + docker)
- rust-runner: docker:27-cli (bootstrap, CI adds Rust)
- node-runner: node:22-bookworm (has Node.js, CI adds docker)

CI workflow (.gitea/workflows/build-runner-images.yml):
1. Watches Dockerfile.* changes
2. Builds custom images with Node.js pre-installed
3. Pushes to registry
4. Image Updater detects and updates values.yaml
5. ArgoCD syncs to new custom images

This avoids token complexity - just push Dockerfile changes to git!
2026-09-05 23:39:20 -07:00
rock 4d60684ca0 chore: remove .forgejo (Forgejo uses .gitea) 2026-09-05 23:33:36 -07:00
rock b098d6a473 fix: runner CI workflow in .gitea (correct Forgejo folder)
- Forgejo reads workflows from .gitea/, not .forgejo/
- Workflow monitors Dockerfile.golang/rust/node for changes
- Builds and pushes images with commit SHA + latest tags
- Image Updater auto-detects new images
- ArgoCD syncs new versions
2026-09-05 23:31:46 -07:00
rock caf0b5bfe6 fix: move runner CI workflow to .forgejo (Forgejo reads from .forgejo, not .gitea)
- Workflow monitors Dockerfile.golang/rust/node for changes
- Builds and pushes images with commit SHA + latest tags
- Image Updater auto-detects new images
- ArgoCD syncs new versions
2026-09-05 23:31:08 -07:00
rock 0e02cafdd7 ci: auto-build runner images on Dockerfile changes
- Watches Dockerfile.golang, .rust, .node for changes
- Builds and pushes images to Forgejo registry with commit SHA + latest tags
- Image Updater detects new images automatically
- ArgoCD syncs updated image tags
- Workflow runs on: golang (has docker + dind)
2026-09-05 23:19:22 -07:00
rock 1e84f13009 feat: add ArgoCD Image Updater tracking for runner images
- Added Image Updater annotations to forgejo-runner Applications
- Image Updater now automatically tracks new images in Forgejo registry
- Update strategy: newest-build (latest commit SHA)
- Tag filter: commits (7-char SHA), latest, and v* releases
- Helm values track repository + tag separately for automatic updates
- Write-back via git (commits image updates to main branch)

Build and push custom runner images:
  docker build -f k8s/infra/forgejo-runner/Dockerfile.golang \
    -t forgejo.riotpiao.com/rock/forgejo-runner-golang:latest .
  docker build -f k8s/infra/forgejo-runner/Dockerfile.rust \
    -t forgejo.riotpiao.com/rock/forgejo-runner-rust:latest .
  docker build -f k8s/infra/forgejo-runner/Dockerfile.node \
    -t forgejo.riotpiao.com/rock/forgejo-runner-node:latest .

  docker login forgejo.riotpiao.com
  docker push forgejo.riotpiao.com/rock/forgejo-runner-golang:latest
  docker push forgejo.riotpiao.com/rock/forgejo-runner-rust:latest
  docker push forgejo.riotpiao.com/rock/forgejo-runner-node:latest

Image Updater will then:
1. Detect new images in registry
2. Update values.yaml automatically
3. Commit changes to git
4. ArgoCD syncs the new image tags
2026-09-05 23:18:22 -07:00
rock 0e63d208d1 feat: add Node.js to all runner images (golang, rust, node)
- Dockerfile.golang: docker:27-cli + Node.js + build tools
- Dockerfile.rust: docker:27-cli + Node.js + Rust + build tools
- Dockerfile.node: node:22-bookworm + docker.io (already has Node.js)
- All runners now support GitHub Actions (checkout@v4 requires Node.js)
- Images built/pushed manually (no CI for homelab)

To build and push:
  docker build -f k8s/infra/forgejo-runner/Dockerfile.golang \
    -t forgejo.riotpiao.com/rock/forgejo-runner-golang:latest .
  docker build -f k8s/infra/forgejo-runner/Dockerfile.rust \
    -t forgejo.riotpiao.com/rock/forgejo-runner-rust:latest .
  docker build -f k8s/infra/forgejo-runner/Dockerfile.node \
    -t forgejo.riotpiao.com/rock/forgejo-runner-node:latest .

  docker push forgejo.riotpiao.com/rock/forgejo-runner-golang:latest
  docker push forgejo.riotpiao.com/rock/forgejo-runner-rust:latest
  docker push forgejo.riotpiao.com/rock/forgejo-runner-node:latest
2026-09-05 23:13:59 -07:00
rock cdd0ba2c99 feat: pre-install docker in node-runner image
- Dockerfile.node extends node:22-bookworm with docker.io
- No need for install step in every workflow
- Values-node.yaml references custom image
- Build and push manually (no CI needed)
2026-09-05 23:11:55 -07:00
10 changed files with 117 additions and 118 deletions
+49
View File
@@ -0,0 +1,49 @@
name: Build and push runner images
on:
push:
paths:
- 'k8s/infra/forgejo-runner/Dockerfile.golang'
- 'k8s/infra/forgejo-runner/Dockerfile.rust'
- 'k8s/infra/forgejo-runner/Dockerfile.node'
branches:
- main
jobs:
build-runners:
runs-on: golang
env:
REGISTRY: forgejo.riotpiao.com
IMAGE_BASE: forgejo.riotpiao.com/rock
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Get short SHA
id: sha
run: |
SHORT_SHA=$(git rev-parse --short HEAD)
echo "short_sha=${SHORT_SHA}" >> $GITHUB_OUTPUT
- name: Registry login
run: |
echo "${REGISTRY_TOKEN}" | docker login "${REGISTRY}" \
--username "${REGISTRY_USER}" --password-stdin
env:
REGISTRY_USER: ${{ secrets.FORGEJO_REGISTRY_USER }}
REGISTRY_TOKEN: ${{ secrets.FORGEJO_REGISTRY_TOKEN }}
- name: Build and push all runner images
run: |
set -e
for RUNNER in golang rust node; do
echo "📦 Building ${RUNNER}-runner..."
docker build -f "k8s/infra/forgejo-runner/Dockerfile.${RUNNER}" \
-t "${IMAGE_BASE}/forgejo-runner-${RUNNER}:${{ steps.sha.outputs.short_sha }}" \
-t "${IMAGE_BASE}/forgejo-runner-${RUNNER}:latest" \
.
docker push "${IMAGE_BASE}/forgejo-runner-${RUNNER}:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE_BASE}/forgejo-runner-${RUNNER}:latest"
echo "✅ Pushed ${RUNNER}-runner"
done
echo "\n✅ All runner images built and pushed"
-92
View File
@@ -208,95 +208,3 @@ versions without warning in your own values file.
Grouping by layer (rather than by day or by "misc fixes") makes it much
easier to `git log --oneline -- <path>` your way back to *why* a given
piece of config looks the way it does, months later.
## Unified Forgejo CI Workflow Pattern (Enforced 2026-09-07+)
All repositories MUST follow this exact structure. No variations.
```yaml
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
env:
REGISTRY: <your-registry-hostname>
IMAGE: <registry>/<org>/<service-name>
jobs:
test:
name: Test
runs-on: [golang|node|rust]
steps:
- name: Install Node.js for actions runtime
run: apt-get update && apt-get install -y nodejs
- name: Checkout code
uses: actions/checkout@v4
# Language-specific tests here (no docker, no registry)
# - name: Run tests
# run: npm test -- --run || true
build-push:
name: Build & Push Image
needs: test
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: [golang|node|rust]
steps:
- name: Install Node.js and Docker
run: |
apt-get update
apt-get install -y nodejs docker.io
- name: Checkout code
uses: actions/checkout@v4
- name: Get short SHA
id: sha
run: |
SHORT_SHA=$(git rev-parse --short HEAD)
echo "short_sha=${SHORT_SHA}" >> $GITHUB_OUTPUT
- name: Registry login
run: |
echo "${REGISTRY_TOKEN}" | docker login "${REGISTRY}" \
--username "${REGISTRY_USER}" --password-stdin
env:
REGISTRY_USER: ${{ secrets.FORGEJO_REGISTRY_USER }}
REGISTRY_TOKEN: ${{ secrets.FORGEJO_REGISTRY_TOKEN }}
- name: Build Docker image
run: |
docker build --no-cache \
-t "${IMAGE}:${{ steps.sha.outputs.short_sha }}" \
-t "${IMAGE}:latest" \
.
- name: Push Docker image
run: |
docker push "${IMAGE}:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE}:latest"
- name: Prune unused images
run: docker image prune -a --force 2>&1 | tail -3 || true
```
### Anti-Patterns (DO NOT USE)
-`container: image: golang:1.26` overrides — breaks docker socket sharing
- ❌ Conditional `if:` on individual steps — use separate jobs instead
- ❌ Installing docker.io in test job — only needed in build-push
- ❌ Monolithic job doing test + build + push — hard to debug
- ❌ Using `{{ github.sha }}` for image tag — use short commit SHA for readability
### How It Works
1. **PR to feature branch** → test job runs, build-push skipped, nothing pushed
2. **Push to main** → test runs, build-push runs after test passes, image pushed
3. Docker socket shared between dind sidecar and runner via emptyDir mount at `/run`
4. `docker_host: automount` in runner config injects socket into workflow containers
5. Secrets (FORGEJO_REGISTRY_USER, TOKEN) set in Forgejo repo settings, NOT in git
@@ -0,0 +1,16 @@
FROM code.forgejo.org/forgejo/runner:6
# Switch to root to install packages (Alpine)
USER root
# Alpine uses apk, not apt-get
RUN apk update && apk add --no-cache \
nodejs \
npm \
docker-cli
# Verify installations
RUN docker --version && node --version && git --version
# Switch back to runner user
USER 1000:1000
+16
View File
@@ -0,0 +1,16 @@
FROM code.forgejo.org/forgejo/runner:6
# Switch to root to install packages (Alpine)
USER root
# Alpine uses apk, not apt-get
RUN apk update && apk add --no-cache \
docker-cli \
nodejs \
npm
# Verify installations
RUN node --version && docker --version && git --version
# Switch back to runner user
USER 1000:1000
+20
View File
@@ -0,0 +1,20 @@
FROM code.forgejo.org/forgejo/runner:6
# Switch to root to install packages (Alpine)
USER root
# Alpine uses apk, not apt-get
RUN apk update && apk add --no-cache \
nodejs \
npm \
curl \
docker-cli
# Install Rust (as root, skip verification for now)
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable || true
# Verify core installations
RUN docker --version && node --version && git --version
# Switch back to runner user
USER 1000:1000
@@ -36,4 +36,3 @@ data:
valid_volumes:
- /docker-certs/client
network: host
docker_host: automount
@@ -34,11 +34,7 @@ spec:
command: ["sh", "-c"]
args:
- |
# Always re-register to keep labels in sync with values.yaml.
# Without this, changing a runner label requires manually deleting
# the PVC or .runner file — not GitOps-friendly.
rm -f /data/.runner
forgejo-runner register --no-interactive \
test -f /data/.runner || forgejo-runner register --no-interactive \
--instance {{ .Values.runner.forgejoUrl }} \
--token $(RUNNER_TOKEN) \
--name {{ .Values.runner.name }} \
@@ -60,18 +56,20 @@ spec:
containers:
- name: runner
image: {{ .Values.runner.image.repository }}:{{ .Values.runner.image.tag }}
command: ["sh", "-c", "while ! wget -q -O- http://localhost:2375/_ping >/dev/null 2>&1; do echo 'waiting for dind...'; sleep 2; done; echo 'dind ready'; forgejo-runner daemon --config /etc/forgejo-runner/config.yaml"]
command: ["sh", "-c", "forgejo-runner daemon --config /etc/forgejo-runner/config.yaml"]
workingDir: /data
env:
- name: DOCKER_HOST
value: tcp://localhost:2375
value: tcp://localhost:2376
- name: DOCKER_TLS_VERIFY
value: "1"
- name: DOCKER_CERT_PATH
value: /docker-certs/client
volumeMounts:
- name: runner-data
mountPath: /data
- name: docker-certs
mountPath: /docker-certs
- name: docker-sock
mountPath: /run
- name: homelab-ca
mountPath: /etc/ssl/certs/homelab-ca.pem
subPath: ca.crt
@@ -87,12 +85,10 @@ spec:
privileged: true # required for DinD; cicd namespace is labelled privileged
env:
- name: DOCKER_TLS_CERTDIR
value: ""
value: /docker-certs
volumeMounts:
- name: docker-certs
mountPath: /docker-certs
- name: docker-sock
mountPath: /run
- name: dind-storage
mountPath: /var/lib/docker
- name: homelab-ca
@@ -117,8 +113,6 @@ spec:
claimName: {{ .Release.Name }}-dind
- name: docker-certs
emptyDir: {} # DinD regenerates mTLS certs on each start
- name: docker-sock
emptyDir: {} # Shared docker socket between dind and runner
- name: homelab-ca
# homelab-ca is a ConfigMap (public CA trust bundle), not a Secret.
# The volumeMounts use subPath: ca.crt to project the single cert file.
+4 -3
View File
@@ -2,14 +2,15 @@
# runner instance. Only runner.name and runner.labels differ -- everything
# else (image, dind, persistence, tolerations, nodeSelector) is shared.
#
# Label image: node:22-bookworm — Debian, root, apt-get, Node.js, npm, git.
# Install docker in workflow steps as needed.
# node:22-bookworm ships Node natively. Docker client installed via workflow step if needed.
# (homelab has no CI; custom runner images built manually if desired)
# Bootstrap with runner image (already has Node.js), CI builds custom
runner:
image:
repository: code.forgejo.org/forgejo/runner
tag: "6"
name: node-runner
labels: "node:docker://node:22-bookworm"
labels: "node:docker://code.forgejo.org/forgejo/runner:6"
# GC CronJob renders only from the default (golang) values to avoid duplicates
gc:
+2 -3
View File
@@ -2,14 +2,13 @@
# runner instance. Only runner.name and runner.labels differ -- everything
# else (image, dind, persistence, tolerations, nodeSelector) is shared.
#
# Label image: rust:1-bookworm — Debian, root, apt-get, Rust, cargo, git.
# Install Node.js/docker in workflow steps as needed.
# Bootstrap with runner image, CI builds custom with Node.js+Rust
runner:
image:
repository: code.forgejo.org/forgejo/runner
tag: "6"
name: rust-runner
labels: "rust:docker://rust:1-bookworm"
labels: "rust:docker://code.forgejo.org/forgejo/runner:6"
+2 -5
View File
@@ -1,12 +1,9 @@
runner:
image:
repository: code.forgejo.org/forgejo/runner
tag: "6"
tag: "6" # Bootstrap with runner image, CI builds custom with Node.js
name: golang-runner
# Label image is what workflow steps run in (NOT the runner daemon image).
# golang:1.26-bookworm: Debian, root, apt-get, Go, git.
# TODO: Switch to custom image once build-runner-images.yml pushes images
labels: "golang:docker://golang:1.26-bookworm"
labels: "golang:docker://code.forgejo.org/forgejo/runner:6"
forgejoUrl: http://forgejo-gitea-http.cicd.svc.cluster.local:3000
tokenSecret: runner-token
resources: