homelab/CLAUDE.example.md

CLAUDE.md — Homelab Integration Guide

Homelab: A bare-metal three-node Kubernetes cluster running Talos Linux with a full observability stack, SSO via Authentik, secret management via Vault, and CI/CD infrastructure (Forgejo + Argo CD, deployed).

Cluster Topology (3 control-plane HA, since 2026-07-20)

Node	IP	Zone	Scheduling	Storage
talos-cp-1	.213	az-a	schedulable (all workloads)	sole Longhorn node
talos-cp-2	.163	az-b	dedicated (NoSchedule)	none
talos-cp-3	.166	az-c	dedicated (NoSchedule)	none

3 voting etcd members peering on the LAN. Only talos-cp-1 runs workloads and holds storage → stateful services are single-instance (e.g. CNPG ddb-cluster = 1 instance). API endpoint is single-homed to .213 (no VIP yet).

Service Integration Routes

New service? Pick your stack below:

Need	Doc	Example
Authentication	project-usage/authentik-oidc.md	OAuth2 login, RBAC groups, JWT tokens
Async messaging	project-usage/sqs-messaging.md	Kafka topic consumers, fire-and-forget, DLQ
Object storage	project-usage/minio-s3.md	File uploads, backups, log backend
CI/CD pipeline	project-usage/cicd-workflow.md	GitHub Actions syntax, image push, Argo CD sync
Workflows	project-usage/temporal-workflows.md	Long-running jobs, retries, state machines
Database	project-usage/database-postgres.md	CloudNativePG setup, schema migrations, replicas
Monitoring	project-usage/monitoring-metrics.md	Prometheus scrape, Grafana dashboard, alerts
Secrets	project-usage/vault-secrets.md	Store credentials, rotate tokens, seal/unseal
Networking	project-usage/networking-ingress.md	Public HTTPS, hostname routing, TLS

Cluster Essentials

22 namespaces, 18 releases:

Core: cert-manager, ingress-nginx, kube-system, cilium
Storage: longhorn-system, storage (MinIO)
Data: ddb (PostgreSQL), iam (Authentik + Vault)
Observability: logging (Loki + Grafana), monitoring (Prometheus)
Apps: cicd (Forgejo + Argo CD), sqs (Kafka + kmsvc), temporal, story-crater-backend

Architecture principles:

• Immutable OS (Talos — no SSH, declarative config)
• Secrets in Vault + SOPS-encrypted (*.enc.yaml, age); never commit .env
• ArgoCD app-of-apps = CD source of truth (k8s/argocd/root → k8s/argocd/apps/*); helmfile is deprecated. Two-phase bootstrap in k8s/argocd/bootstrap/BOOTSTRAP.md
• Pull-based GitOps (Argo CD, no kubeconfig in CI); iterate = git push to Forgejo → auto-sync
• Federated OIDC (Authentik provider for all services)

Deployment Checklist

• Service has Prometheus /metrics endpoint or ServiceMonitor
• All credentials in Vault (never in pod env, ConfigMap, or code)
• Ingress rule in k8s/ingress/ with TLS cert
• Grafana dashboard in k8s/monitoring/dashboards/svc-<name>.yaml
• Alert rules in k8s/monitoring/alerts/svc-<name>-rules.yaml (if needed)
• Helm release in helmfile.yaml.gotmpl with correct needs: dependencies

Hard Rules

1. No kubeconfig in CI — Argo CD bridges gap (pull-based, never push secrets to runner)
2. Field name = variable name — In Vault: talos put cluster/KAFKA_BOOTSTRAP KAFKA_BOOTSTRAP="..."
3. Secrets via volumes — Never --env flag in pod specs (exposes in kubectl describe)
4. External services via Ingress — All public endpoints via TLS (homelab-ca)
5. Never commit .env — Only .env.example in git; real secrets in Vault
6. Never rename or wipe talos-cp-1 (.213) — sole Longhorn storage node; renaming orphans its node CR and faults every volume (permanent data loss). Rename/reprovision only the dedicated CPs.
7. Control-plane etcd advertises on the LAN — keep cluster.etcd.advertisedSubnets: ["192.168.1.0/24"], else Talos advertises on WireGuard and new members hang as etcd learners.

Git & Release

Multi-remote push:

git push origin main

Incremental commits (service-layer grouped):

• Foundation & Docs
• Helmfile & Core Infra
• Storage Layer
• Observability Stack
• IAM & Secrets
• CI/CD & GitOps
• Messaging Infrastructure
• Applications & Utilities