rock/homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat:Fix the bootstrap to be deploy key application

Browse Source
This commit is contained in:
Story Crater Bot
2026-08-18 15:08:03 -07:00
parent eac3a2a227
commit 54fa540b33
37 changed files with 1216 additions and 1202 deletions
Download Patch File Download Diff File Expand all files Collapse all files
k8s/bootstrap/README.md
+128
View File
@@ -0,0 +1,128 @@
# Homelab Bootstrap — Single-Cluster, GitOps-Ready
**Run once manually, GitOps forever after.**
This bootstrap breaks the ArgoCD ↔ Forgejo circular dependency by:
1. Installing infrastructure in correct dependency order
2. Pointing ArgoCD at a GitHub mirror initially
3. Cutting over to Forgejo once healthy
4. Using Helm for reproducible installs
5. Ensuring ArgoCD adopts (not duplicates) bootstrap resources
## Prerequisites
- Talos cluster running (terraform applied)
- kubectl configured (`KUBECONFIG` points at cluster)
- Helm 3 installed
- SOPS age key at `~/.sops/homelab-age.key`
- GitHub mirror of this repo (for initial ArgoCD source)
## Directory Structure
```
bootstrap/
├── phase1-storage/ # Longhorn via Helm
├── phase2-cnpg/ # CNPG operator via Helm
├── phase3-forgejo/ # Forgejo DB + Forgejo via Helm
├── phase4-argocd/ # ArgoCD via Helm → GitHub initially
└── phase5-cutover/ # Switch ArgoCD source to Forgejo
```
## Usage
```bash
# From repo root:
./bootstrap.sh
# Or step-by-step:
./bootstrap.sh phase1 # Storage
./bootstrap.sh phase2 # CNPG
./bootstrap.sh phase3 # Forgejo
./bootstrap.sh phase4 # ArgoCD (GitHub mirror)
./bootstrap.sh phase5 # Cut over to Forgejo
```
## Design Principles
1. **DRY**: Helm values used by both bootstrap and ArgoCD
2. **Single Source of Truth**: Manifests match what ArgoCD will manage
3. **Idempotent**: Can re-run phases safely
4. **Adoption Ready**: Resources have `argocd.argoproj.io/sync-options: Prune=false`
5. **Dependency Ordered**: Each phase waits for previous to be Ready
## Phase Details
### Phase 1: Storage (Longhorn)
Installs Longhorn with:
- 3-node HA configuration
- Unified `longhorn` StorageClass (default)
- Special `longhorn-cnpg` StorageClass with postgres UID/GID mount options
- CSI plugin tolerations for control-plane nodes
**Source of Truth**: `phase1-storage/longhorn-values.yaml`
### Phase 2: CNPG Operator
Installs CloudNativePG operator with:
- CRD registration (blocks until CRD available)
- Webhook configuration
- Monitoring enabled
**Source of Truth**: `phase2-cnpg/cnpg-values.yaml`
### Phase 3: Forgejo Database + Forgejo
1. Creates `forgejo-db` CNPG Cluster
2. Waits for cluster Ready (PostgreSQL accepting connections)
3. Installs Forgejo via Helm pointing at `forgejo-db-rw` service
4. Waits for Forgejo healthy
**Source of Truth**:
- `phase3-forgejo/forgejo-db.yaml` (CNPG Cluster CR)
- `phase3-forgejo/forgejo-values.yaml` (Helm values)
### Phase 4: ArgoCD (GitHub Mirror)
Installs ArgoCD via Helm, then applies root app-of-apps pointing at **GitHub mirror**.
This is the circle-breaker: ArgoCD syncs from GitHub (not Forgejo) initially.
**Source of Truth**:
- `phase4-argocd/argocd-values.yaml`
- `phase4-argocd/root-app-github.yaml` (repoURL = GitHub)
ArgoCD **adopts** Phases 1-3 resources (no duplication) because manifests match.
### Phase 5: Cut Over to Forgejo
1. Push repo to Forgejo
2. Update root app `repoURL` from GitHub → Forgejo
3. ArgoCD re-syncs from Forgejo
**The circle is broken. GitHub mirror is now disaster recovery only.**
## Post-Bootstrap
All changes via Git:
```bash
git commit -m "feat(app): add new service"
git push forgejo main
# ArgoCD auto-syncs
```
## Troubleshooting
- **Phase stuck?** Check `kubectl get events -n <namespace> --sort-by='.lastTimestamp'`
- **ArgoCD duplicating?** Verify manifests match exactly (Helm values ↔ ArgoCD Application)
- **Forgejo won't start?** Check CNPG cluster Ready: `kubectl get cluster forgejo-db -n forgejo`
- **Can't push to Forgejo?** Verify ingress-nginx healthy, DNS resolves `forgejo.riotpiao.com`
## Migration from Old Bootstrap
If you have existing `k8s/bootstrap-local/`:
1. **DO NOT delete** existing resources (Longhorn data!)
2. Run refined bootstrap in "adoption mode" (no delete, just apply)
3. Verify ArgoCD shows "Synced" for all apps
4. Archive old bootstrap: `git mv k8s/bootstrap-local k8s/archive/bootstrap-local-v1`
k8s/bootstrap/phase1-storage/longhorn-values.yaml
+56
View File
@@ -0,0 +1,56 @@
# Longhorn Helm Values — Single Source of Truth
# Used by both bootstrap.sh (Helm install) and ArgoCD (adoption)
# Chart: https://github.com/longhorn/charts
defaultSettings:
# 3-node HA configuration
replicaReplenishmentWaitInterval: 600 # 10min before auto-repair
replicaSoftAntiAffinity: false # REQUIRED for true HA
replicaAutoBalance: best-effort
storageMinimalAvailablePercentage: 10
# Performance tuning
defaultDataPath: /var/lib/longhorn
defaultDataLocality: best-effort
backupTarget: "" # TODO: Add MinIO backup target later
# Monitoring
guaranteedEngineManagerCPU: 12 # mCPU
guaranteedReplicaManagerCPU: 12
persistence:
defaultClass: true # Make 'longhorn' the default StorageClass
defaultClassReplicaCount: 3
defaultFsType: ext4
reclaimPolicy: Delete
# CSI plugin must tolerate control-plane taints
csi:
kubeletRootDir: /var/lib/kubelet
attacherReplicaCount: 3
provisionerReplicaCount: 3
resizerReplicaCount: 3
snapshotterReplicaCount: 3
# Longhorn manager on all nodes
longhornManager:
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
# Driver deployer tolerations
longhornDriver:
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
# UI for debugging
longhornUI:
replicas: 1
# Monitoring (Prometheus ServiceMonitor)
metrics:
serviceMonitor:
enabled: true
k8s/bootstrap/phase1-storage/storageclasses.yaml
+27
View File
@@ -0,0 +1,27 @@
# StorageClasses — Applied after Longhorn installation
# The default 'longhorn' SC is created by Helm chart
# These are additional specialized classes
---
# CNPG-specific StorageClass with postgres UID/GID mount options
# Fixes "read-only filesystem" error when PostgreSQL (UID 26) tries to write
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: longhorn-cnpg
annotations:
storageclass.kubernetes.io/is-default-class: "false"
argocd.argoproj.io/sync-options: Prune=false # Allow ArgoCD adoption
provisioner: driver.longhorn.io
allowVolumeExpansion: true
parameters:
numberOfReplicas: "3"
staleReplicaTimeout: "30"
fromBackup: ""
dataLocality: "best-effort"
fsType: "ext4"
mountOptions:
- "noatime"
- "uid=26" # postgres user
- "gid=26" # postgres group
reclaimPolicy: Delete
volumeBindingMode: Immediate
k8s/bootstrap/phase2-cnpg/cnpg-values.yaml
+55
View File
@@ -0,0 +1,55 @@
# CloudNativePG Operator Helm Values — Single Source of Truth
# Chart: https://github.com/cloudnative-pg/charts
# Basic operator configuration
replicaCount: 1
# CRDs must be installed (Helm default behavior)
crds:
create: true
# Webhook configuration
webhook:
port: 9443
mutating:
create: true
failurePolicy: Fail
validating:
create: true
failurePolicy: Fail
# Monitoring
monitoring:
podMonitorEnabled: true
grafanaDashboard:
create: false # We'll manage dashboards via ArgoCD later
# Resource limits for operator
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 512Mi
# Security context
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000
capabilities:
drop:
- ALL
# Tolerate control-plane taints
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
# Additional labels for ArgoCD adoption
commonLabels:
app.kubernetes.io/managed-by: Helm
argocd.argoproj.io/instance: cnpg-operator
k8s/bootstrap/phase3-forgejo/forgejo-db.yaml
+122
View File
@@ -0,0 +1,122 @@
# Forgejo PostgreSQL Database — CNPG Cluster CR
# This is the source of truth for Forgejo's database
# ArgoCD will adopt this (not recreate it)
---
apiVersion: v1
kind: Namespace
metadata:
name: forgejo
annotations:
argocd.argoproj.io/sync-options: Prune=false
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: forgejo-db
namespace: forgejo
annotations:
argocd.argoproj.io/sync-options: Prune=false # Let ArgoCD adopt, don't delete
labels:
app: forgejo-db
layer: data
spec:
instances: 3 # HA configuration
imageName: ghcr.io/cloudnative-pg/postgresql:16.2
bootstrap:
initdb:
database: forgejo
owner: forgejo
encoding: UTF8
localeCollate: C
localeCType: C
enableSuperuserAccess: false
# Resource limits per best practices
resources:
requests:
memory: "4Gi"
cpu: "1"
limits:
memory: "8Gi"
cpu: "2"
postgresql:
parameters:
# Tuned for 4-8GB RAM
shared_buffers: "1GB"
effective_cache_size: "3GB"
maintenance_work_mem: "512MB"
work_mem: "64MB"
max_connections: "100"
max_parallel_workers: "2"
max_parallel_workers_per_gather: "1"
# WAL
wal_buffers: "16MB"
checkpoint_completion_target: "0.9"
min_wal_size: "512MB"
max_wal_size: "2GB"
# Logging
log_destination: "csvlog"
log_directory: "/controller/log"
log_filename: "postgres"
storage:
size: 50Gi
storageClass: longhorn-cnpg # Uses postgres UID/GID mount options
monitoring:
enablePodMonitor: true
affinity:
podAntiAffinityType: required
topologyKey: kubernetes.io/hostname
---
# Forgejo Redis (cache, session, queue)
apiVersion: v1
kind: Service
metadata:
name: forgejo-redis
namespace: forgejo
annotations:
argocd.argoproj.io/sync-options: Prune=false
spec:
ports:
- port: 6379
targetPort: 6379
protocol: TCP
selector:
app: forgejo-redis
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: forgejo-redis
namespace: forgejo
annotations:
argocd.argoproj.io/sync-options: Prune=false
spec:
replicas: 1
selector:
matchLabels:
app: forgejo-redis
template:
metadata:
labels:
app: forgejo-redis
spec:
containers:
- name: redis
image: redis:7-alpine
ports:
- containerPort: 6379
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 200m
memory: 256Mi
k8s/bootstrap/phase3-forgejo/forgejo-values.yaml
+86
View File
@@ -0,0 +1,86 @@
# Forgejo Helm Values — Single Source of Truth
# Chart: https://codeberg.org/forgejo-contrib/forgejo-helm
gitea:
admin:
username: "admin"
email: "[email protected]"
# Password set via secret (not in values)
config:
server:
DOMAIN: forgejo.riotpiao.com
ROOT_URL: https://forgejo.riotpiao.com
SSH_DOMAIN: forgejo.riotpiao.com
SSH_PORT: 22
database:
DB_TYPE: postgres
HOST: forgejo-db-rw.forgejo.svc.cluster.local:5432
NAME: forgejo
# User/password from CNPG-generated secret
USER:
valueFrom:
secretKeyRef:
name: forgejo-db-app
key: username
PASSWD:
valueFrom:
secretKeyRef:
name: forgejo-db-app
key: password
cache:
ADAPTER: redis
HOST: redis://forgejo-redis.forgejo.svc.cluster.local:6379/0
session:
PROVIDER: redis
PROVIDER_CONFIG: redis://forgejo-redis.forgejo.svc.cluster.local:6379/1
queue:
TYPE: redis
CONN_STR: redis://forgejo-redis.forgejo.svc.cluster.local:6379/2
# Persistence (shared storage for repos)
persistence:
enabled: true
storageClass: longhorn
size: 20Gi
accessModes:
- ReadWriteOnce
# Ingress
ingress:
enabled: true
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
hosts:
- host: forgejo.riotpiao.com
paths:
- path: /
pathType: Prefix
tls:
- secretName: forgejo-tls
hosts:
- forgejo.riotpiao.com
# Resources
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
cpu: 1000m
memory: 2Gi
# Tolerations for control-plane
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
# ArgoCD adoption labels
labels:
argocd.argoproj.io/instance: forgejo
k8s/bootstrap/phase4-argocd/argocd-values.yaml
+124
View File
@@ -0,0 +1,124 @@
# ArgoCD Helm Values — Single Source of Truth
# Chart: https://github.com/argoproj/argo-helm
global:
domain: argocd.riotpiao.com
# Server configuration
server:
ingress:
enabled: true
ingressClassName: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
hosts:
- argocd.riotpiao.com
tls:
- secretName: argocd-server-tls
hosts:
- argocd.riotpiao.com
# Allow insecure mode (terminate TLS at ingress)
extraArgs:
- --insecure
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 1Gi
# Repo server configuration
repoServer:
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 1Gi
# SOPS plugin for encrypted secrets
volumes:
- name: sops-age
secret:
secretName: sops-age
optional: true
volumeMounts:
- name: sops-age
mountPath: /home/argocd/.config/sops/age
readOnly: true
# Environment for SOPS
env:
- name: SOPS_AGE_KEY_FILE
value: /home/argocd/.config/sops/age/keys.txt
# Controller configuration
controller:
resources:
requests:
cpu: 200m
memory: 512Mi
limits:
cpu: 1000m
memory: 2Gi
# Application controller configuration
applicationSet:
enabled: true
# Notifications (optional, for Slack/Discord alerts)
notifications:
enabled: false
# Redis for caching
redis:
enabled: true
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 200m
memory: 256Mi
# Tolerations for control-plane
server:
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
repoServer:
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
controller:
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
# ArgoCD projects
configs:
# Default project allows all repos
cm:
admin.enabled: "true"
application.instanceLabelKey: argocd.argoproj.io/instance
params:
server.insecure: true
# RBAC (allow admin full access)
configs:
rbac:
policy.default: role:readonly
policy.csv: |
g, admin, role:admin
k8s/bootstrap/phase4-argocd/root-app-github.yaml
+66
View File
@@ -0,0 +1,66 @@
# ArgoCD Root App-of-Apps — GitHub Mirror Source
# This is the initial configuration that breaks the circular dependency
# Points at GitHub mirror, not Forgejo (Forgejo isn't ready yet)
#
# After Forgejo is healthy and repo is pushed, use phase5-cutover/root-app-forgejo.yaml
---
apiVersion: v1
kind: Namespace
metadata:
name: argocd
---
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: homelab
namespace: argocd
spec:
description: Homelab infrastructure and applications
sourceRepos:
- 'https://github.com/YOUR-ORG/homelab.git' # ← REPLACE with your GitHub mirror
- 'https://forgejo.riotpiao.com/YOUR-ORG/homelab.git'
- 'https://*.github.io/*' # Helm charts from GitHub Pages
- 'https://charts.*' # Public Helm repos
destinations:
- namespace: '*'
server: 'https://kubernetes.default.svc'
clusterResourceWhitelist:
- group: '*'
kind: '*'
namespaceResourceWhitelist:
- group: '*'
kind: '*'
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: homelab-root
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: homelab
source:
repoURL: https://github.com/YOUR-ORG/homelab.git # ← REPLACE with your GitHub mirror
targetRevision: main
path: k8s/argocd/apps
destination:
server: https://kubernetes.default.svc
namespace: argocd
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
retry:
limit: 5
backoff:
duration: 5s
factor: 2
maxDuration: 3m
k8s/bootstrap/phase5-cutover/root-app-forgejo.yaml
+39
View File
@@ -0,0 +1,39 @@
# ArgoCD Root App-of-Apps — Forgejo Source (Final State)
# This replaces the GitHub mirror with Forgejo as the source of truth
# Apply this AFTER Forgejo is healthy and you've pushed the repo
#
# The circle is broken: ArgoCD → Forgejo works because both already exist
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: homelab-root
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: homelab
source:
repoURL: https://forgejo.riotpiao.com/YOUR-ORG/homelab.git # ← REPLACE with your Forgejo URL
targetRevision: main
path: k8s/argocd/apps
destination:
server: https://kubernetes.default.svc
namespace: argocd
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
retry:
limit: 5
backoff:
duration: 5s
factor: 2
maxDuration: 3m