refactor(argocd): remove orphaned infrastructure Applications - bootstrap is source of truth
REMOVED ORPHANED APPLICATIONS: - cnpg-operator (OutOfSync, conflicted with bootstrap) - forgejo (OutOfSync, conflicted with bootstrap) - ingress-nginx-bootstrap (orphaned, no ownerReferences) ARCHITECTURE NOW CLEAN: ✅ Bootstrap: 7 manifests (infrastructure base for regional deployment) - ArgoCD, CNPG operator, DDB, Forgejo, ingress-nginx, namespaces, wait-for-databases ✅ ArgoCD: 32 Applications (all services/apps) ✅ No duplicate management DEPLOYMENT FLOW: 1. kubectl apply -k k8s/bootstrap-local/ (infrastructure) 2. kubectl apply -k k8s/argocd/root/ (app-of-apps) 3. ArgoCD auto-syncs from Forgejo (applications) CLEANUP: - Archived old bootstrap configs (k8s/argocd/bootstrap.archived/) - Deleted orphaned Applications (ArgoCD tracking only, resources untouched) Bootstrap remains single source of truth for infrastructure. ArgoCD manages all applications and services.
This commit is contained in:
@@ -1,127 +0,0 @@
|
|||||||
# Cluster bootstrap & CD — two scopes
|
|
||||||
|
|
||||||
The GitOps repo is hosted on the cluster's own Forgejo, so ArgoCD cannot pull it
|
|
||||||
until Forgejo exists. There are two distinct workflows — pick the one that
|
|
||||||
matches your situation.
|
|
||||||
|
|
||||||
```
|
|
||||||
Scope 1 — from scratch cold cluster, nothing running yet → full bootstrap
|
|
||||||
Scope 2 — iterating CD cluster + Forgejo + ArgoCD are up → just git push
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Scope 1 — Cluster from scratch (cold bootstrap)
|
|
||||||
|
|
||||||
Strict ordering, because each layer depends on the previous one existing.
|
|
||||||
|
|
||||||
```
|
|
||||||
Talos + Cilium CNI Terraform (nodes) — cluster reachable
|
|
||||||
Substrate cert-manager, ingress-nginx, reloader — Terraform helm_releases
|
|
||||||
Longhorn, ArgoCD — imperative install
|
|
||||||
Phase 0 (git-independent) CNPG + ddb-cluster + redis + Forgejo — manual, below
|
|
||||||
Seed git push repo into Forgejo
|
|
||||||
Phase 1 app-of-apps root — ArgoCD deploys the rest
|
|
||||||
```
|
|
||||||
|
|
||||||
Run every command from a local checkout with `KUBECONFIG` pointed at the cluster.
|
|
||||||
|
|
||||||
### Phase 0 — bootstrap Forgejo and its data plane (git-independent)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. CNPG operator (public Helm) + its CRDs.
|
|
||||||
kubectl apply -f k8s/argocd/bootstrap/cnpg-operator.yaml
|
|
||||||
argocd app sync cnpg-operator
|
|
||||||
# CNPG ships CRDs in the chart's crds/ folder, which ArgoCD's helm-template does
|
|
||||||
# NOT install. If the ddb-cluster apply below fails with "no matches for kind
|
|
||||||
# Cluster", install them once:
|
|
||||||
# kubectl apply --server-side -f \
|
|
||||||
# https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.24/releases/cnpg-1.24.1.yaml
|
|
||||||
|
|
||||||
# 2. Namespaces first. cicd MUST be labelled privileged (DinD/runner needs it);
|
|
||||||
# the Forgejo app deliberately does NOT create it.
|
|
||||||
kubectl create namespace ddb --dry-run=client -o yaml | kubectl apply -f -
|
|
||||||
kubectl create namespace cicd --dry-run=client -o yaml | kubectl apply -f -
|
|
||||||
kubectl label namespace cicd pod-security.kubernetes.io/enforce=privileged --overwrite
|
|
||||||
|
|
||||||
# 3. Postgres cluster + Forgejo database + Redis (raw manifests, local checkout).
|
|
||||||
kubectl apply -f k8s/data/ddb-cluster.yaml
|
|
||||||
kubectl apply -f k8s/data/forgejo-database.yaml
|
|
||||||
kubectl apply -f k8s/security/ci-cd/forgejo-redis.yaml
|
|
||||||
|
|
||||||
# 4. Copy the CNPG-generated app password into cicd (Secrets are ns-scoped).
|
|
||||||
kubectl get secret ddb-cluster-app -n ddb -o yaml \
|
|
||||||
| sed 's/namespace: ddb/namespace: cicd/' \
|
|
||||||
| kubectl apply -n cicd -f -
|
|
||||||
|
|
||||||
# 5. Forgejo (public Helm, values inlined — git-independent).
|
|
||||||
kubectl apply -f k8s/argocd/bootstrap/forgejo.yaml
|
|
||||||
argocd app sync forgejo
|
|
||||||
```
|
|
||||||
|
|
||||||
### Seed + Phase 1
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 6. Push this repo to the freshly-created Forgejo so Phase 1 has a source.
|
|
||||||
git remote add forgejo https://forgejo.riotpiao.com/riotpiao.com/homelab.git
|
|
||||||
git push forgejo main
|
|
||||||
|
|
||||||
# 7. (Optional) Cloudflared tunnel secret — if the SOPS CMP plugin is NOT yet
|
|
||||||
# wired up: create the cloudflared-token Secret manually before wave 8 syncs.
|
|
||||||
# Skip this step if the CMP plugin is already live and handling SOPS decryption.
|
|
||||||
kubectl create namespace cloudflared --dry-run=client -o yaml | kubectl apply -f -
|
|
||||||
sops -d k8s/applications/cloudflared/cloudflared-secrets.enc.yaml \
|
|
||||||
| yq '.cloudflared.tunnelToken' \
|
|
||||||
| kubectl create secret generic cloudflared-token -n cloudflared \
|
|
||||||
--from-literal=token=- --dry-run=client -o yaml | kubectl apply -f -
|
|
||||||
# Once the SOPS CMP plugin is deployed (Phase 1 wave 0), migrate this Secret
|
|
||||||
# into the plugin's script output and remove this manual step.
|
|
||||||
|
|
||||||
# 8. Deploy everything else from git — one sync brings up the whole cluster.
|
|
||||||
kubectl apply -f k8s/argocd/projects/homelab-project.yaml # AppProject
|
|
||||||
kubectl apply -k k8s/argocd/root # app-of-apps root
|
|
||||||
argocd app sync homelab-root
|
|
||||||
```
|
|
||||||
|
|
||||||
`homelab-root` renders every Application under `k8s/argocd/apps/` and syncs them
|
|
||||||
in sync-wave order (networking → storage/observability → logging → security →
|
|
||||||
data → messaging → applications).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Scope 2 — Iterating CD (day-2, cluster already up)
|
|
||||||
|
|
||||||
The app-of-apps and its children already exist and reconcile from Forgejo. To
|
|
||||||
change anything, you do NOT re-run the bootstrap — you just push:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# edit manifests under k8s/**
|
|
||||||
git add -A && git commit -m "..."
|
|
||||||
git push # to Forgejo main (origin)
|
|
||||||
# ArgoCD auto-syncs (child apps have syncPolicy.automated); to force it:
|
|
||||||
argocd app sync homelab-root
|
|
||||||
```
|
|
||||||
|
|
||||||
- Adding a new service = add an Application manifest under `k8s/argocd/apps/`
|
|
||||||
and its manifests under the path it references, then push. The app-of-apps
|
|
||||||
picks it up on next sync.
|
|
||||||
- Changing a Helm value = edit the values file the Application points at
|
|
||||||
(`$values/...`), push.
|
|
||||||
- Forgejo itself is Phase-0 / bootstrap (manual sync only) — a bad CI commit
|
|
||||||
must not be able to break the system CI depends on.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Notes / prerequisites
|
|
||||||
|
|
||||||
- **Secrets:** several values files (loki S3, grafana admin, authentik, vault)
|
|
||||||
expect secrets that helmfile used to inject via `--set`. Under ArgoCD these
|
|
||||||
come from the `*.enc.yaml` SOPS files via the ArgoCD SOPS plugin — confirm the
|
|
||||||
plugin is configured before syncing waves 2–3, or those charts render with
|
|
||||||
empty secrets.
|
|
||||||
- **Substrate** (cert-manager, ingress-nginx, reloader) is installed by Terraform
|
|
||||||
as bootstrap Helm releases, not ArgoCD. Cilium CNI, Longhorn, and ArgoCD
|
|
||||||
itself are cluster-bootstrap installs.
|
|
||||||
- **Single storage node:** only `talos-cp-1` runs workloads/storage, so stateful
|
|
||||||
services are single-instance (`ddb-cluster` = 1). See the repo `USAGE.md` and
|
|
||||||
root `CLAUDE.md` topology section.
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: cnpg-operator
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "0"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://cloudnative-pg.github.io/charts
|
|
||||||
chart: cloudnative-pg
|
|
||||||
targetRevision: "~0.20"
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: ddb
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
@@ -1,155 +0,0 @@
|
|||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: forgejo
|
|
||||||
namespace: argocd
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "7"
|
|
||||||
# Must sync after cnpg-operator (wave 0) and the ddb-cluster/forgejo-database
|
|
||||||
# (wave 6, k8s/data) — Forgejo's DB connection depends on both existing first.
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://dl.gitea.com/charts/
|
|
||||||
chart: gitea
|
|
||||||
targetRevision: "~10"
|
|
||||||
helm:
|
|
||||||
# Mirrors k8s/security/ci-cd/forgejo-values.yaml inline — avoids the
|
|
||||||
# git-repo chicken-egg (Forgejo can't source its own values from a repo
|
|
||||||
# it hosts). Keep both files in sync when editing either one.
|
|
||||||
valuesObject:
|
|
||||||
image:
|
|
||||||
repository: codeberg.org/forgejo/forgejo
|
|
||||||
tag: "13"
|
|
||||||
pullPolicy: IfNotPresent
|
|
||||||
gitea:
|
|
||||||
admin:
|
|
||||||
username: rock
|
|
||||||
email: [email protected]
|
|
||||||
config:
|
|
||||||
server:
|
|
||||||
PROTOCOL: http
|
|
||||||
DOMAIN: forgejo.riotpiao.com
|
|
||||||
ROOT_URL: https://forgejo.riotpiao.com/
|
|
||||||
HTTP_PORT: 3000
|
|
||||||
START_SSH_SERVER: true
|
|
||||||
SSH_DOMAIN: forgejo.riotpiao.com
|
|
||||||
SSH_PORT: 2222
|
|
||||||
SSH_LISTEN_PORT: 2222
|
|
||||||
database:
|
|
||||||
DB_TYPE: postgres
|
|
||||||
HOST: ddb-cluster-rw.ddb.svc:5432
|
|
||||||
NAME: forgejo
|
|
||||||
USER: app
|
|
||||||
repository:
|
|
||||||
ROOT: /data/git
|
|
||||||
actions:
|
|
||||||
ENABLED: true
|
|
||||||
packages:
|
|
||||||
ENABLED: true
|
|
||||||
metrics:
|
|
||||||
ENABLED: true
|
|
||||||
service:
|
|
||||||
DISABLE_REGISTRATION: true
|
|
||||||
oauth2:
|
|
||||||
ENABLED: true
|
|
||||||
PROVIDER: openidconnect
|
|
||||||
OPENID_CONNECT_DISCOVERY_URL: https://authentik.riotpiao.com/application/o/forgejo/.well-known/openid-configuration
|
|
||||||
CLIENT_ID: forgejo
|
|
||||||
AUTO_DISCOVER_URL: https://authentik.riotpiao.com/application/o/forgejo/.well-known/openid-configuration
|
|
||||||
cache:
|
|
||||||
ADAPTER: redis
|
|
||||||
HOST: "redis://forgejo-redis.cicd.svc:6379/0"
|
|
||||||
session:
|
|
||||||
PROVIDER: redis
|
|
||||||
PROVIDER_CONFIG: "redis://forgejo-redis.cicd.svc:6379/1"
|
|
||||||
queue:
|
|
||||||
TYPE: redis
|
|
||||||
CONN_STR: "redis://forgejo-redis.cicd.svc:6379/2"
|
|
||||||
metrics:
|
|
||||||
enabled: true
|
|
||||||
serviceMonitor:
|
|
||||||
enabled: false
|
|
||||||
persistence:
|
|
||||||
enabled: true
|
|
||||||
storageClass: longhorn
|
|
||||||
size: 20Gi
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteMany
|
|
||||||
replicaCount: 2
|
|
||||||
deployment:
|
|
||||||
strategy:
|
|
||||||
type: RollingUpdate
|
|
||||||
env:
|
|
||||||
- name: SSL_CERT_DIR
|
|
||||||
value: /homelab-ca
|
|
||||||
- name: GITEA__database__PASSWD
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: ddb-cluster-app
|
|
||||||
key: password
|
|
||||||
- name: GITEA__oauth2__CLIENT_SECRET
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: forgejo-oidc
|
|
||||||
key: CLIENT_SECRET
|
|
||||||
podAnnotations:
|
|
||||||
configmap.reloader.stakater.com/reload: "homelab-ca"
|
|
||||||
service:
|
|
||||||
http:
|
|
||||||
type: LoadBalancer
|
|
||||||
port: 3000
|
|
||||||
targetPort: 3000
|
|
||||||
annotations:
|
|
||||||
io.cilium/lb-ipam-ips: "192.168.1.165"
|
|
||||||
io.cilium/lb-ipam-sharing-key: "forgejo"
|
|
||||||
ssh:
|
|
||||||
type: LoadBalancer
|
|
||||||
port: 2222
|
|
||||||
targetPort: 2222
|
|
||||||
annotations:
|
|
||||||
io.cilium/lb-ipam-ips: "192.168.1.165"
|
|
||||||
io.cilium/lb-ipam-sharing-key: "forgejo"
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
cpu: 250m
|
|
||||||
memory: 512Mi
|
|
||||||
limits:
|
|
||||||
cpu: "1"
|
|
||||||
memory: 1Gi
|
|
||||||
tolerations:
|
|
||||||
- key: node-role.kubernetes.io/control-plane
|
|
||||||
operator: Exists
|
|
||||||
effect: NoSchedule
|
|
||||||
extraVolumes:
|
|
||||||
- name: homelab-ca
|
|
||||||
configMap:
|
|
||||||
name: homelab-ca
|
|
||||||
extraVolumeMounts:
|
|
||||||
- name: homelab-ca
|
|
||||||
mountPath: /homelab-ca
|
|
||||||
readOnly: true
|
|
||||||
ingress:
|
|
||||||
enabled: false
|
|
||||||
postgresql:
|
|
||||||
enabled: false
|
|
||||||
postgresql-ha:
|
|
||||||
enabled: false
|
|
||||||
mysql:
|
|
||||||
enabled: false
|
|
||||||
redis-cluster:
|
|
||||||
enabled: false
|
|
||||||
act_runner:
|
|
||||||
enabled: false
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: cicd
|
|
||||||
syncPolicy:
|
|
||||||
# NO CreateNamespace — cicd must be pre-created and labelled
|
|
||||||
# pod-security.kubernetes.io/enforce=privileged (DinD/runner needs it);
|
|
||||||
# ArgoCD's CreateNamespace would make it with the cluster-default baseline.
|
|
||||||
# The Phase 0 runbook creates+labels cicd before this app is applied.
|
|
||||||
#
|
|
||||||
# NO automated sync — Forgejo is what CI uses to push commits; auto-sync
|
|
||||||
# would let a bad CI commit break the system CI depends on. Manual only.
|
|
||||||
syncOptions: []
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
#!/bin/bash
|
|
||||||
set -e
|
|
||||||
|
|
||||||
export SOPS_AGE_KEY_FILE=/sops-age/keys.txt
|
|
||||||
|
|
||||||
# Decrypt grafana secrets once
|
|
||||||
GRAFANA_SECRETS=$(sops -d k8s/platform/logging/grafana-secrets.enc.yaml)
|
|
||||||
|
|
||||||
# Stage 0: grafana
|
|
||||||
OIDC_SECRET=$(echo "$GRAFANA_SECRETS" | yq -r '.env.GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET')
|
|
||||||
ADMIN_PASSWORD=$(echo "$GRAFANA_SECRETS" | yq -r '.adminPassword')
|
|
||||||
|
|
||||||
cat <<EOF
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: grafana-oidc
|
|
||||||
namespace: logging
|
|
||||||
type: Opaque
|
|
||||||
data:
|
|
||||||
GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: $(echo -n "$OIDC_SECRET" | base64 -w0)
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: grafana-admin
|
|
||||||
namespace: logging
|
|
||||||
type: Opaque
|
|
||||||
data:
|
|
||||||
admin-password: $(echo -n "$ADMIN_PASSWORD" | base64 -w0)
|
|
||||||
EOF
|
|
||||||
Binary file not shown.
Reference in New Issue
Block a user