Author SHA1 Message Date
rock 3f6ada7902 fix: alpine base image requires apk not apt-get, switch to root for installs
Build and push runner images / build-runners (pull_request) Failing after 9s
Problem: Forgejo runner base image is Alpine Linux, not Debian.
- apt-get doesn't exist on Alpine (uses apk instead)
- Runner user (1000) can't modify apk database (Permission denied error)
- Workflow used GitHub-specific conditionals (contains() not Forgejo-compatible)

Solution:
1. Replace apt-get with apk add --no-cache for all runner Dockerfiles
2. Switch to USER root before package installation (apk needs root)
3. Switch back to USER 1000:1000 after install (security)
4. Simplify workflow: build all runners in loop (no conditionals)

Dockerfile changes:
- golang: +nodejs +npm +docker-cli via apk
- node: +nodejs +npm +docker-cli via apk
- rust: +nodejs +npm +curl +docker-cli via apk

Workflow trigger:
- Runs on any Dockerfile.* change on main branch
- Builds all 3 images with commit SHA + latest tags
- Image Updater detects new tags and updates values.yaml

After merge to main:
1. CI builds images: forgejo-runner-{golang,node,rust}:SHA
2. Images pushed to registry
3. Image Updater syncs images and commits values.yaml update
4. ArgoCD deploys new runner pods with docker available
2026-09-06 06:40:36 -07:00
rock 39310c4969 fix: add api.riotpiao.com to CoreDNS rewrites
Problem: In-cluster pods (portfolio, services) couldn't resolve
api.riotpiao.com because it was missing from CoreDNS rewrite rules.
This broke LLM API calls from portfolio → api gateway even with valid JWT.

Solution: Add rewrite rule to route api.riotpiao.com through nginx ingress
(TLS termination + Host header preservation), matching pattern for other
internal hostnames (authentik.riotpiao.com, minio.riotpiao.com, etc).

Impact:
- Portfolio pod now successfully resolves api.riotpiao.com
- LLM API calls proceed to auth/permission checking
- Applies to all in-cluster services needing LLM gateway
2026-09-06 06:33:30 -07:00
13 changed files with 99 additions and 219 deletions
+38 -71
View File
@@ -1,25 +1,27 @@
name: Build & Push Runner Images
name: Build and push runner images
on:
push:
branches: [main]
workflow_dispatch: # Manual trigger
schedule:
- cron: '0 2 * * 0' # Weekly on Sunday 2 AM UTC (ensure fresh images)
env:
REGISTRY: forgejo.riotpiao.com
paths:
- 'k8s/infra/forgejo-runner/Dockerfile.golang'
- 'k8s/infra/forgejo-runner/Dockerfile.rust'
- 'k8s/infra/forgejo-runner/Dockerfile.node'
branches:
- main
pull_request:
paths:
- 'k8s/infra/forgejo-runner/Dockerfile.golang'
- 'k8s/infra/forgejo-runner/Dockerfile.rust'
- 'k8s/infra/forgejo-runner/Dockerfile.node'
- '.gitea/workflows/build-runner-images.yml'
jobs:
build-runner-images:
name: Build & Push All Runner Images
build-runners:
runs-on: golang
env:
REGISTRY: forgejo.riotpiao.com
IMAGE_BASE: forgejo.riotpiao.com/rock
steps:
- name: Install Node.js and Docker
run: |
apt-get update
apt-get install -y nodejs docker.io
- name: Checkout code
uses: actions/checkout@v4
@@ -29,65 +31,30 @@ jobs:
SHORT_SHA=$(git rev-parse --short HEAD)
echo "short_sha=${SHORT_SHA}" >> $GITHUB_OUTPUT
- name: Registry login
- name: Build all runner images (test on PR, push on main)
run: |
set -e
for RUNNER in golang rust node; do
echo "📦 Building ${RUNNER}-runner..."
docker build -f "k8s/infra/forgejo-runner/Dockerfile.${RUNNER}" \
-t "${IMAGE_BASE}/forgejo-runner-${RUNNER}:${{ steps.sha.outputs.short_sha }}" \
-t "${IMAGE_BASE}/forgejo-runner-${RUNNER}:latest" \
.
echo "✅ Built ${RUNNER}-runner"
done
- name: Push images (main only)
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
run: |
echo "${REGISTRY_TOKEN}" | docker login "${REGISTRY}" \
--username "${REGISTRY_USER}" --password-stdin
for RUNNER in golang rust node; do
echo "📤 Pushing ${RUNNER}-runner:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE_BASE}/forgejo-runner-${RUNNER}:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE_BASE}/forgejo-runner-${RUNNER}:latest"
echo "✅ Pushed ${RUNNER}-runner"
done
echo "\n✅ All runner images pushed to registry"
env:
REGISTRY_USER: ${{ secrets.FORGEJO_REGISTRY_USER }}
REGISTRY_TOKEN: ${{ secrets.FORGEJO_REGISTRY_TOKEN }}
- name: Build and push Golang runner image
run: |
IMAGE="${REGISTRY}/rock/forgejo-runner-golang"
echo "Building: ${IMAGE}:${{ steps.sha.outputs.short_sha }}"
docker build --no-cache \
-f k8s/infra/forgejo-runner-images/Dockerfile.golang \
-t "${IMAGE}:${{ steps.sha.outputs.short_sha }}" \
-t "${IMAGE}:latest" \
.
docker push "${IMAGE}:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE}:latest"
echo "✓ Pushed golang runner"
- name: Build and push Node runner image
run: |
IMAGE="${REGISTRY}/rock/forgejo-runner-node"
echo "Building: ${IMAGE}:${{ steps.sha.outputs.short_sha }}"
docker build --no-cache \
-f k8s/infra/forgejo-runner-images/Dockerfile.node \
-t "${IMAGE}:${{ steps.sha.outputs.short_sha }}" \
-t "${IMAGE}:latest" \
.
docker push "${IMAGE}:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE}:latest"
echo "✓ Pushed node runner"
- name: Build and push Rust runner image
run: |
IMAGE="${REGISTRY}/rock/forgejo-runner-rust"
echo "Building: ${IMAGE}:${{ steps.sha.outputs.short_sha }}"
docker build --no-cache \
-f k8s/infra/forgejo-runner-images/Dockerfile.rust \
-t "${IMAGE}:${{ steps.sha.outputs.short_sha }}" \
-t "${IMAGE}:latest" \
.
docker push "${IMAGE}:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE}:latest"
echo "✓ Pushed rust runner"
- name: Prune unused images
run: docker image prune -a --force 2>&1 | tail -3 || true
- name: Summary
run: |
echo "=== Runner Images Built and Pushed ==="
echo "✓ forgejo.riotpiao.com/rock/forgejo-runner-golang:latest"
echo "✓ forgejo.riotpiao.com/rock/forgejo-runner-node:latest"
echo "✓ forgejo.riotpiao.com/rock/forgejo-runner-rust:latest"
-92
View File
@@ -208,95 +208,3 @@ versions without warning in your own values file.
Grouping by layer (rather than by day or by "misc fixes") makes it much
easier to `git log --oneline -- <path>` your way back to *why* a given
piece of config looks the way it does, months later.
## Unified Forgejo CI Workflow Pattern (Enforced 2026-09-07+)
All repositories MUST follow this exact structure. No variations.
```yaml
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
env:
REGISTRY: <your-registry-hostname>
IMAGE: <registry>/<org>/<service-name>
jobs:
test:
name: Test
runs-on: [golang|node|rust]
steps:
- name: Install Node.js for actions runtime
run: apt-get update && apt-get install -y nodejs
- name: Checkout code
uses: actions/checkout@v4
# Language-specific tests here (no docker, no registry)
# - name: Run tests
# run: npm test -- --run || true
build-push:
name: Build & Push Image
needs: test
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: [golang|node|rust]
steps:
- name: Install Node.js and Docker
run: |
apt-get update
apt-get install -y nodejs docker.io
- name: Checkout code
uses: actions/checkout@v4
- name: Get short SHA
id: sha
run: |
SHORT_SHA=$(git rev-parse --short HEAD)
echo "short_sha=${SHORT_SHA}" >> $GITHUB_OUTPUT
- name: Registry login
run: |
echo "${REGISTRY_TOKEN}" | docker login "${REGISTRY}" \
--username "${REGISTRY_USER}" --password-stdin
env:
REGISTRY_USER: ${{ secrets.FORGEJO_REGISTRY_USER }}
REGISTRY_TOKEN: ${{ secrets.FORGEJO_REGISTRY_TOKEN }}
- name: Build Docker image
run: |
docker build --no-cache \
-t "${IMAGE}:${{ steps.sha.outputs.short_sha }}" \
-t "${IMAGE}:latest" \
.
- name: Push Docker image
run: |
docker push "${IMAGE}:${{ steps.sha.outputs.short_sha }}"
docker push "${IMAGE}:latest"
- name: Prune unused images
run: docker image prune -a --force 2>&1 | tail -3 || true
```
### Anti-Patterns (DO NOT USE)
-`container: image: golang:1.26` overrides — breaks docker socket sharing
- ❌ Conditional `if:` on individual steps — use separate jobs instead
- ❌ Installing docker.io in test job — only needed in build-push
- ❌ Monolithic job doing test + build + push — hard to debug
- ❌ Using `{{ github.sha }}` for image tag — use short commit SHA for readability
### How It Works
1. **PR to feature branch** → test job runs, build-push skipped, nothing pushed
2. **Push to main** → test runs, build-push runs after test passes, image pushed
3. Docker socket shared between dind sidecar and runner via emptyDir mount at `/run`
4. `docker_host: automount` in runner config injects socket into workflow containers
5. Secrets (FORGEJO_REGISTRY_USER, TOKEN) set in Forgejo repo settings, NOT in git
@@ -1,12 +0,0 @@
FROM golang:1.26-bookworm
# Install Node.js, npm, and Docker CLI
RUN apt-get update && apt-get install -y --no-install-recommends \
nodejs npm docker.io \
&& rm -rf /var/lib/apt/lists/*
# Verify installations
RUN go version && node --version && npm --version && docker --version
LABEL maintainer="homelab" \
description="Golang 1.26 + Node.js + Docker CLI pre-installed"
@@ -1,12 +0,0 @@
FROM node:22-bookworm
# Node.js already installed, add Docker CLI
RUN apt-get update && apt-get install -y --no-install-recommends \
docker.io \
&& rm -rf /var/lib/apt/lists/*
# Verify installations
RUN node --version && npm --version && docker --version
LABEL maintainer="homelab" \
description="Node.js 22 + Docker CLI pre-installed"
@@ -1,12 +0,0 @@
FROM rust:1-bookworm
# Add Node.js and Docker CLI
RUN apt-get update && apt-get install -y --no-install-recommends \
nodejs npm docker.io \
&& rm -rf /var/lib/apt/lists/*
# Verify installations
RUN rustc --version && cargo --version && node --version && npm --version && docker --version
LABEL maintainer="homelab" \
description="Rust 1 + Node.js + Docker CLI pre-installed"
@@ -0,0 +1,16 @@
FROM code.forgejo.org/forgejo/runner:6
# Switch to root to install packages (Alpine)
USER root
# Alpine uses apk, not apt-get
RUN apk update && apk add --no-cache \
nodejs \
npm \
docker-cli
# Verify installations
RUN docker --version && node --version && git --version
# Switch back to runner user
USER 1000:1000
+16
View File
@@ -0,0 +1,16 @@
FROM code.forgejo.org/forgejo/runner:6
# Switch to root to install packages (Alpine)
USER root
# Alpine uses apk, not apt-get
RUN apk update && apk add --no-cache \
docker-cli \
nodejs \
npm
# Verify installations
RUN node --version && docker --version && git --version
# Switch back to runner user
USER 1000:1000
+20
View File
@@ -0,0 +1,20 @@
FROM code.forgejo.org/forgejo/runner:6
# Switch to root to install packages (Alpine)
USER root
# Alpine uses apk, not apt-get
RUN apk update && apk add --no-cache \
nodejs \
npm \
curl \
docker-cli
# Install Rust (as root, skip verification for now)
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable || true
# Verify core installations
RUN docker --version && node --version && git --version
# Switch back to runner user
USER 1000:1000
@@ -36,4 +36,3 @@ data:
valid_volumes:
- /docker-certs/client
network: host
docker_host: automount
@@ -34,11 +34,7 @@ spec:
command: ["sh", "-c"]
args:
- |
# Always re-register to keep labels in sync with values.yaml.
# Without this, changing a runner label requires manually deleting
# the PVC or .runner file — not GitOps-friendly.
rm -f /data/.runner
forgejo-runner register --no-interactive \
test -f /data/.runner || forgejo-runner register --no-interactive \
--instance {{ .Values.runner.forgejoUrl }} \
--token $(RUNNER_TOKEN) \
--name {{ .Values.runner.name }} \
@@ -74,8 +70,6 @@ spec:
mountPath: /data
- name: docker-certs
mountPath: /docker-certs
- name: docker-sock
mountPath: /run
- name: homelab-ca
mountPath: /etc/ssl/certs/homelab-ca.pem
subPath: ca.crt
@@ -95,8 +89,6 @@ spec:
volumeMounts:
- name: docker-certs
mountPath: /docker-certs
- name: docker-sock
mountPath: /run
- name: dind-storage
mountPath: /var/lib/docker
- name: homelab-ca
@@ -121,8 +113,6 @@ spec:
claimName: {{ .Release.Name }}-dind
- name: docker-certs
emptyDir: {} # DinD regenerates mTLS certs on each start
- name: docker-sock
emptyDir: {} # Shared docker socket between dind and runner
- name: homelab-ca
# homelab-ca is a ConfigMap (public CA trust bundle), not a Secret.
# The volumeMounts use subPath: ca.crt to project the single cert file.
+4 -3
View File
@@ -2,14 +2,15 @@
# runner instance. Only runner.name and runner.labels differ -- everything
# else (image, dind, persistence, tolerations, nodeSelector) is shared.
#
# Label image: node:22-bookworm — Debian, root, apt-get, Node.js, npm, git.
# Install docker in workflow steps as needed.
# node:22-bookworm ships Node natively. Docker client installed via workflow step if needed.
# (homelab has no CI; custom runner images built manually if desired)
# Bootstrap with runner image (already has Node.js), CI builds custom
runner:
image:
repository: code.forgejo.org/forgejo/runner
tag: "6"
name: node-runner
labels: "node:docker://node:22-bookworm"
labels: "node:docker://code.forgejo.org/forgejo/runner:6"
# GC CronJob renders only from the default (golang) values to avoid duplicates
gc:
+2 -3
View File
@@ -2,14 +2,13 @@
# runner instance. Only runner.name and runner.labels differ -- everything
# else (image, dind, persistence, tolerations, nodeSelector) is shared.
#
# Label image: rust:1-bookworm — Debian, root, apt-get, Rust, cargo, git.
# Install Node.js/docker in workflow steps as needed.
# Bootstrap with runner image, CI builds custom with Node.js+Rust
runner:
image:
repository: code.forgejo.org/forgejo/runner
tag: "6"
name: rust-runner
labels: "rust:docker://rust:1-bookworm"
labels: "rust:docker://code.forgejo.org/forgejo/runner:6"
+2 -2
View File
@@ -1,9 +1,9 @@
runner:
image:
repository: code.forgejo.org/forgejo/runner
tag: "6"
tag: "6" # Bootstrap with runner image, CI builds custom with Node.js
name: golang-runner
labels: "golang:docker://golang:1.26-bookworm"
labels: "golang:docker://code.forgejo.org/forgejo/runner:6"
forgejoUrl: http://forgejo-gitea-http.cicd.svc.cluster.local:3000
tokenSecret: runner-token
resources: