refactor(argocd): remove orphaned infrastructure Applications - bootstrap is source of truth
REMOVED ORPHANED APPLICATIONS: - cnpg-operator (OutOfSync, conflicted with bootstrap) - forgejo (OutOfSync, conflicted with bootstrap) - ingress-nginx-bootstrap (orphaned, no ownerReferences) ARCHITECTURE NOW CLEAN: ✅ Bootstrap: 7 manifests (infrastructure base for regional deployment) - ArgoCD, CNPG operator, DDB, Forgejo, ingress-nginx, namespaces, wait-for-databases ✅ ArgoCD: 32 Applications (all services/apps) ✅ No duplicate management DEPLOYMENT FLOW: 1. kubectl apply -k k8s/bootstrap-local/ (infrastructure) 2. kubectl apply -k k8s/argocd/root/ (app-of-apps) 3. ArgoCD auto-syncs from Forgejo (applications) CLEANUP: - Archived old bootstrap configs (k8s/argocd/bootstrap.archived/) - Deleted orphaned Applications (ArgoCD tracking only, resources untouched) Bootstrap remains single source of truth for infrastructure. ArgoCD manages all applications and services.
This commit is contained in:
Story Crater Bot
@@ -1,127 +0,0 @@
|
||||
# Cluster bootstrap & CD — two scopes
|
||||
|
||||
The GitOps repo is hosted on the cluster's own Forgejo, so ArgoCD cannot pull it
|
||||
until Forgejo exists. There are two distinct workflows — pick the one that
|
||||
matches your situation.
|
||||
|
||||
```
|
||||
Scope 1 — from scratch cold cluster, nothing running yet → full bootstrap
|
||||
Scope 2 — iterating CD cluster + Forgejo + ArgoCD are up → just git push
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Scope 1 — Cluster from scratch (cold bootstrap)
|
||||
|
||||
Strict ordering, because each layer depends on the previous one existing.
|
||||
|
||||
```
|
||||
Talos + Cilium CNI Terraform (nodes) — cluster reachable
|
||||
Substrate cert-manager, ingress-nginx, reloader — Terraform helm_releases
|
||||
Longhorn, ArgoCD — imperative install
|
||||
Phase 0 (git-independent) CNPG + ddb-cluster + redis + Forgejo — manual, below
|
||||
Seed git push repo into Forgejo
|
||||
Phase 1 app-of-apps root — ArgoCD deploys the rest
|
||||
```
|
||||
|
||||
Run every command from a local checkout with `KUBECONFIG` pointed at the cluster.
|
||||
|
||||
### Phase 0 — bootstrap Forgejo and its data plane (git-independent)
|
||||
|
||||
```bash
|
||||
# 1. CNPG operator (public Helm) + its CRDs.
|
||||
kubectl apply -f k8s/argocd/bootstrap/cnpg-operator.yaml
|
||||
argocd app sync cnpg-operator
|
||||
# CNPG ships CRDs in the chart's crds/ folder, which ArgoCD's helm-template does
|
||||
# NOT install. If the ddb-cluster apply below fails with "no matches for kind
|
||||
# Cluster", install them once:
|
||||
# kubectl apply --server-side -f \
|
||||
# https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.24/releases/cnpg-1.24.1.yaml
|
||||
|
||||
# 2. Namespaces first. cicd MUST be labelled privileged (DinD/runner needs it);
|
||||
# the Forgejo app deliberately does NOT create it.
|
||||
kubectl create namespace ddb --dry-run=client -o yaml | kubectl apply -f -
|
||||
kubectl create namespace cicd --dry-run=client -o yaml | kubectl apply -f -
|
||||
kubectl label namespace cicd pod-security.kubernetes.io/enforce=privileged --overwrite
|
||||
|
||||
# 3. Postgres cluster + Forgejo database + Redis (raw manifests, local checkout).
|
||||
kubectl apply -f k8s/data/ddb-cluster.yaml
|
||||
kubectl apply -f k8s/data/forgejo-database.yaml
|
||||
kubectl apply -f k8s/security/ci-cd/forgejo-redis.yaml
|
||||
|
||||
# 4. Copy the CNPG-generated app password into cicd (Secrets are ns-scoped).
|
||||
kubectl get secret ddb-cluster-app -n ddb -o yaml \
|
||||
| sed 's/namespace: ddb/namespace: cicd/' \
|
||||
| kubectl apply -n cicd -f -
|
||||
|
||||
# 5. Forgejo (public Helm, values inlined — git-independent).
|
||||
kubectl apply -f k8s/argocd/bootstrap/forgejo.yaml
|
||||
argocd app sync forgejo
|
||||
```
|
||||
|
||||
### Seed + Phase 1
|
||||
|
||||
```bash
|
||||
# 6. Push this repo to the freshly-created Forgejo so Phase 1 has a source.
|
||||
git remote add forgejo https://forgejo.riotpiao.com/riotpiao.com/homelab.git
|
||||
git push forgejo main
|
||||
|
||||
# 7. (Optional) Cloudflared tunnel secret — if the SOPS CMP plugin is NOT yet
|
||||
# wired up: create the cloudflared-token Secret manually before wave 8 syncs.
|
||||
# Skip this step if the CMP plugin is already live and handling SOPS decryption.
|
||||
kubectl create namespace cloudflared --dry-run=client -o yaml | kubectl apply -f -
|
||||
sops -d k8s/applications/cloudflared/cloudflared-secrets.enc.yaml \
|
||||
| yq '.cloudflared.tunnelToken' \
|
||||
| kubectl create secret generic cloudflared-token -n cloudflared \
|
||||
--from-literal=token=- --dry-run=client -o yaml | kubectl apply -f -
|
||||
# Once the SOPS CMP plugin is deployed (Phase 1 wave 0), migrate this Secret
|
||||
# into the plugin's script output and remove this manual step.
|
||||
|
||||
# 8. Deploy everything else from git — one sync brings up the whole cluster.
|
||||
kubectl apply -f k8s/argocd/projects/homelab-project.yaml # AppProject
|
||||
kubectl apply -k k8s/argocd/root # app-of-apps root
|
||||
argocd app sync homelab-root
|
||||
```
|
||||
|
||||
`homelab-root` renders every Application under `k8s/argocd/apps/` and syncs them
|
||||
in sync-wave order (networking → storage/observability → logging → security →
|
||||
data → messaging → applications).
|
||||
|
||||
---
|
||||
|
||||
## Scope 2 — Iterating CD (day-2, cluster already up)
|
||||
|
||||
The app-of-apps and its children already exist and reconcile from Forgejo. To
|
||||
change anything, you do NOT re-run the bootstrap — you just push:
|
||||
|
||||
```bash
|
||||
# edit manifests under k8s/**
|
||||
git add -A && git commit -m "..."
|
||||
git push # to Forgejo main (origin)
|
||||
# ArgoCD auto-syncs (child apps have syncPolicy.automated); to force it:
|
||||
argocd app sync homelab-root
|
||||
```
|
||||
|
||||
- Adding a new service = add an Application manifest under `k8s/argocd/apps/`
|
||||
and its manifests under the path it references, then push. The app-of-apps
|
||||
picks it up on next sync.
|
||||
- Changing a Helm value = edit the values file the Application points at
|
||||
(`$values/...`), push.
|
||||
- Forgejo itself is Phase-0 / bootstrap (manual sync only) — a bad CI commit
|
||||
must not be able to break the system CI depends on.
|
||||
|
||||
---
|
||||
|
||||
## Notes / prerequisites
|
||||
|
||||
- **Secrets:** several values files (loki S3, grafana admin, authentik, vault)
|
||||
expect secrets that helmfile used to inject via `--set`. Under ArgoCD these
|
||||
come from the `*.enc.yaml` SOPS files via the ArgoCD SOPS plugin — confirm the
|
||||
plugin is configured before syncing waves 2–3, or those charts render with
|
||||
empty secrets.
|
||||
- **Substrate** (cert-manager, ingress-nginx, reloader) is installed by Terraform
|
||||
as bootstrap Helm releases, not ArgoCD. Cilium CNI, Longhorn, and ArgoCD
|
||||
itself are cluster-bootstrap installs.
|
||||
- **Single storage node:** only `talos-cp-1` runs workloads/storage, so stateful
|
||||
services are single-instance (`ddb-cluster` = 1). See the repo `USAGE.md` and
|
||||
root `CLAUDE.md` topology section.
|
||||
@@ -1,22 +0,0 @@
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: cnpg-operator
|
||||
namespace: argocd
|
||||
annotations:
|
||||
argocd.argoproj.io/sync-wave: "0"
|
||||
spec:
|
||||
project: homelab
|
||||
source:
|
||||
repoURL: https://cloudnative-pg.github.io/charts
|
||||
chart: cloudnative-pg
|
||||
targetRevision: "~0.20"
|
||||
destination:
|
||||
server: https://kubernetes.default.svc
|
||||
namespace: ddb
|
||||
syncPolicy:
|
||||
automated:
|
||||
prune: true
|
||||
selfHeal: true
|
||||
syncOptions:
|
||||
- CreateNamespace=true
|
||||
@@ -1,155 +0,0 @@
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: forgejo
|
||||
namespace: argocd
|
||||
annotations:
|
||||
argocd.argoproj.io/sync-wave: "7"
|
||||
# Must sync after cnpg-operator (wave 0) and the ddb-cluster/forgejo-database
|
||||
# (wave 6, k8s/data) — Forgejo's DB connection depends on both existing first.
|
||||
spec:
|
||||
project: homelab
|
||||
source:
|
||||
repoURL: https://dl.gitea.com/charts/
|
||||
chart: gitea
|
||||
targetRevision: "~10"
|
||||
helm:
|
||||
# Mirrors k8s/security/ci-cd/forgejo-values.yaml inline — avoids the
|
||||
# git-repo chicken-egg (Forgejo can't source its own values from a repo
|
||||
# it hosts). Keep both files in sync when editing either one.
|
||||
valuesObject:
|
||||
image:
|
||||
repository: codeberg.org/forgejo/forgejo
|
||||
tag: "13"
|
||||
pullPolicy: IfNotPresent
|
||||
gitea:
|
||||
admin:
|
||||
username: rock
|
||||
email: [email protected]
|
||||
config:
|
||||
server:
|
||||
PROTOCOL: http
|
||||
DOMAIN: forgejo.riotpiao.com
|
||||
ROOT_URL: https://forgejo.riotpiao.com/
|
||||
HTTP_PORT: 3000
|
||||
START_SSH_SERVER: true
|
||||
SSH_DOMAIN: forgejo.riotpiao.com
|
||||
SSH_PORT: 2222
|
||||
SSH_LISTEN_PORT: 2222
|
||||
database:
|
||||
DB_TYPE: postgres
|
||||
HOST: ddb-cluster-rw.ddb.svc:5432
|
||||
NAME: forgejo
|
||||
USER: app
|
||||
repository:
|
||||
ROOT: /data/git
|
||||
actions:
|
||||
ENABLED: true
|
||||
packages:
|
||||
ENABLED: true
|
||||
metrics:
|
||||
ENABLED: true
|
||||
service:
|
||||
DISABLE_REGISTRATION: true
|
||||
oauth2:
|
||||
ENABLED: true
|
||||
PROVIDER: openidconnect
|
||||
OPENID_CONNECT_DISCOVERY_URL: https://authentik.riotpiao.com/application/o/forgejo/.well-known/openid-configuration
|
||||
CLIENT_ID: forgejo
|
||||
AUTO_DISCOVER_URL: https://authentik.riotpiao.com/application/o/forgejo/.well-known/openid-configuration
|
||||
cache:
|
||||
ADAPTER: redis
|
||||
HOST: "redis://forgejo-redis.cicd.svc:6379/0"
|
||||
session:
|
||||
PROVIDER: redis
|
||||
PROVIDER_CONFIG: "redis://forgejo-redis.cicd.svc:6379/1"
|
||||
queue:
|
||||
TYPE: redis
|
||||
CONN_STR: "redis://forgejo-redis.cicd.svc:6379/2"
|
||||
metrics:
|
||||
enabled: true
|
||||
serviceMonitor:
|
||||
enabled: false
|
||||
persistence:
|
||||
enabled: true
|
||||
storageClass: longhorn
|
||||
size: 20Gi
|
||||
accessModes:
|
||||
- ReadWriteMany
|
||||
replicaCount: 2
|
||||
deployment:
|
||||
strategy:
|
||||
type: RollingUpdate
|
||||
env:
|
||||
- name: SSL_CERT_DIR
|
||||
value: /homelab-ca
|
||||
- name: GITEA__database__PASSWD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: ddb-cluster-app
|
||||
key: password
|
||||
- name: GITEA__oauth2__CLIENT_SECRET
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: forgejo-oidc
|
||||
key: CLIENT_SECRET
|
||||
podAnnotations:
|
||||
configmap.reloader.stakater.com/reload: "homelab-ca"
|
||||
service:
|
||||
http:
|
||||
type: LoadBalancer
|
||||
port: 3000
|
||||
targetPort: 3000
|
||||
annotations:
|
||||
io.cilium/lb-ipam-ips: "192.168.1.165"
|
||||
io.cilium/lb-ipam-sharing-key: "forgejo"
|
||||
ssh:
|
||||
type: LoadBalancer
|
||||
port: 2222
|
||||
targetPort: 2222
|
||||
annotations:
|
||||
io.cilium/lb-ipam-ips: "192.168.1.165"
|
||||
io.cilium/lb-ipam-sharing-key: "forgejo"
|
||||
resources:
|
||||
requests:
|
||||
cpu: 250m
|
||||
memory: 512Mi
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: 1Gi
|
||||
tolerations:
|
||||
- key: node-role.kubernetes.io/control-plane
|
||||
operator: Exists
|
||||
effect: NoSchedule
|
||||
extraVolumes:
|
||||
- name: homelab-ca
|
||||
configMap:
|
||||
name: homelab-ca
|
||||
extraVolumeMounts:
|
||||
- name: homelab-ca
|
||||
mountPath: /homelab-ca
|
||||
readOnly: true
|
||||
ingress:
|
||||
enabled: false
|
||||
postgresql:
|
||||
enabled: false
|
||||
postgresql-ha:
|
||||
enabled: false
|
||||
mysql:
|
||||
enabled: false
|
||||
redis-cluster:
|
||||
enabled: false
|
||||
act_runner:
|
||||
enabled: false
|
||||
destination:
|
||||
server: https://kubernetes.default.svc
|
||||
namespace: cicd
|
||||
syncPolicy:
|
||||
# NO CreateNamespace — cicd must be pre-created and labelled
|
||||
# pod-security.kubernetes.io/enforce=privileged (DinD/runner needs it);
|
||||
# ArgoCD's CreateNamespace would make it with the cluster-default baseline.
|
||||
# The Phase 0 runbook creates+labels cicd before this app is applied.
|
||||
#
|
||||
# NO automated sync — Forgejo is what CI uses to push commits; auto-sync
|
||||
# would let a bad CI commit break the system CI depends on. Manual only.
|
||||
syncOptions: []
|
||||
@@ -1,32 +0,0 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
export SOPS_AGE_KEY_FILE=/sops-age/keys.txt
|
||||
|
||||
# Decrypt grafana secrets once
|
||||
GRAFANA_SECRETS=$(sops -d k8s/platform/logging/grafana-secrets.enc.yaml)
|
||||
|
||||
# Stage 0: grafana
|
||||
OIDC_SECRET=$(echo "$GRAFANA_SECRETS" | yq -r '.env.GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET')
|
||||
ADMIN_PASSWORD=$(echo "$GRAFANA_SECRETS" | yq -r '.adminPassword')
|
||||
|
||||
cat <<EOF
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: grafana-oidc
|
||||
namespace: logging
|
||||
type: Opaque
|
||||
data:
|
||||
GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: $(echo -n "$OIDC_SECRET" | base64 -w0)
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: grafana-admin
|
||||
namespace: logging
|
||||
type: Opaque
|
||||
data:
|
||||
admin-password: $(echo -n "$ADMIN_PASSWORD" | base64 -w0)
|
||||
EOF
|
||||
Reference in New Issue
Block a user