Bundle memory database into homelab orchestration (remove separate app)
ci / markdown (push) Waiting to run
ci / markdown (push) Waiting to run
This commit is contained in:
Story Crater Bot
+70
-216
@@ -1,246 +1,100 @@
|
||||
# Poimen Memory — ArgoCD Tracking & Deployment
|
||||
# Poimen Memory — ArgoCD Deployment (Bundled into Homelab)
|
||||
|
||||
## Status
|
||||
|
||||
✅ **Poimen Memory is tracked in homelab ArgoCD**
|
||||
✅ **All config files in place**
|
||||
✅ **Ready to deploy on next homelab sync**
|
||||
✅ **Memory database bundled into homelab wave 2 (databases)**
|
||||
✅ **No separate app needed**
|
||||
✅ **Deployed as part of existing homelab orchestration**
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Homelab ArgoCD Root
|
||||
↓ (wave 7)
|
||||
71-poimen-memory.yaml (homelab repo)
|
||||
↓ syncs k8s/argocd/ from Poimen Memory repo
|
||||
Poimen Memory k8s/argocd/
|
||||
├── kustomization.yaml
|
||||
└── memory-database-app.yaml
|
||||
↓ syncs k8s/infra/databases/
|
||||
Memory CNPG Cluster (wave 2 within Poimen Memory)
|
||||
Homelab ArgoCD Root (wave 0)
|
||||
↓
|
||||
Wave 2: Databases App (syncs k8s/infra/databases)
|
||||
├── authentik-db (ns: iam)
|
||||
├── temporal-db (ns: temporal)
|
||||
└── memory-db (ns: poimen) ← NEW
|
||||
├── Cluster: memory-db (3 instances)
|
||||
├── Extension: pgvector (768-dim embeddings)
|
||||
├── Secret: memory-db-app (auto-generated in poimen)
|
||||
└── Service: memory-db-rw (auto-generated in poimen)
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### How It Works
|
||||
|
||||
Memory database is deployed **as part of homelab wave 2** (same wave as authentik-db and temporal-db):
|
||||
|
||||
1. **Homelab repository** contains the cluster definition
|
||||
- `k8s/infra/databases/memory-db.yaml` (deployed to: poimen namespace)
|
||||
- `k8s/infra/databases/kustomization.yaml` (includes memory-db.yaml)
|
||||
|
||||
2. **Homelab "databases" app** (wave 2) syncs these manifests
|
||||
- No separate poimen-memory app needed
|
||||
- Bundles all DB clusters in one wave
|
||||
- Simple and consistent
|
||||
|
||||
3. **Poimen Memory repo** contains reference copies
|
||||
- `k8s/infra/databases/memory-db.yaml` (for CI/documentation)
|
||||
- Deployed via homelab, not its own app
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
# After homelab syncs wave 2:
|
||||
kubectl get clusters -n poimen
|
||||
# NAME PHASE INSTANCES
|
||||
# memory-db Healthy 3/3
|
||||
|
||||
kubectl get secret -n poimen memory-db-app
|
||||
# memory-db-app kubernetes.io/basic-auth 2
|
||||
|
||||
kubectl get svc -n poimen memory-db-rw
|
||||
# memory-db-rw ClusterIP 10.x.x.x 5432/TCP
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration Files
|
||||
|
||||
### 1. Homelab Repository
|
||||
**Location:** `/Users/rockliang/workplace/homelab/k8s/argocd/`
|
||||
|
||||
**File 1: `projects/homelab-project.yaml` (UPDATED)**
|
||||
```yaml
|
||||
sourceRepos:
|
||||
- https://github.com/Riotpiaole/riotpiao.homelab.com.git
|
||||
- https://github.com/Riotpiaole/Poimen-memory.git # ← ADDED
|
||||
- https://forgejo.riotpiao.com/rock/*
|
||||
# ... public Helm repos
|
||||
```
|
||||
**What it does:** Authorizes ArgoCD to sync from Poimen Memory repo
|
||||
|
||||
**File 2: `apps/71-poimen-memory.yaml` (NEW)**
|
||||
```yaml
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: poimen-memory-root
|
||||
annotations:
|
||||
argocd.argoproj.io/sync-wave: "7"
|
||||
spec:
|
||||
project: homelab
|
||||
source:
|
||||
repoURL: https://github.com/Riotpiaole/Poimen-memory.git
|
||||
path: k8s/argocd
|
||||
destination:
|
||||
server: https://kubernetes.default.svc
|
||||
```
|
||||
**What it does:** Root application that syncs the Poimen Memory app-of-apps
|
||||
|
||||
### 2. Poimen Memory Repository
|
||||
**Location:** `/Users/rockliang/workplace/Poimen/memory/k8s/argocd/`
|
||||
|
||||
**File 1: `kustomization.yaml` (NEW)**
|
||||
```yaml
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- memory-database-app.yaml
|
||||
```
|
||||
**What it does:** Declares the application manifests for Poimen Memory
|
||||
|
||||
**File 2: `memory-database-app.yaml` (NEW)**
|
||||
```yaml
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: poimen-memory-database
|
||||
annotations:
|
||||
argocd.argoproj.io/sync-wave: "2" # Wave 2 = databases
|
||||
spec:
|
||||
project: homelab
|
||||
source:
|
||||
repoURL: https://github.com/Riotpiaole/Poimen-memory.git
|
||||
path: k8s/infra/databases
|
||||
destination:
|
||||
server: https://kubernetes.default.svc
|
||||
```
|
||||
**What it does:** Syncs the CNPG Postgres cluster config from `k8s/infra/databases/`
|
||||
|
||||
---
|
||||
|
||||
## Deployment Wave Order
|
||||
## Connection String
|
||||
|
||||
```
|
||||
Wave 0: Homelab Root (homelab-root.yaml)
|
||||
↓
|
||||
Wave 2: Databases (k8s/infra/databases/)
|
||||
├── 40-data.yaml (homelab databases)
|
||||
│ ├── authentik-db
|
||||
│ ├── temporal-db
|
||||
│ └── memory-db (from k8s/infra/databases/kustomization.yaml)
|
||||
└── 71-poimen-memory.yaml syncs its own wave 2 (memory-database-app)
|
||||
↓
|
||||
Wave 7: Applications
|
||||
├── Poimen Memory Root (poimen-memory-root)
|
||||
├── Poimen Services (future)
|
||||
└── Other applications
|
||||
postgresql://app:<password>@memory-db-rw.poimen.svc.cluster.local:5432/memory?sslmode=disable
|
||||
```
|
||||
|
||||
**Credentials:** `kubectl get secret -n poimen memory-db-app`
|
||||
|
||||
---
|
||||
|
||||
## Deployment Checklist
|
||||
|
||||
### Prerequisites
|
||||
- [ ] Poimen Memory repo exists on GitHub at `Riotpiaole/Poimen-memory`
|
||||
- [ ] Main branch is up-to-date with:
|
||||
- [ ] `k8s/infra/databases/memory-db.yaml`
|
||||
- [ ] `k8s/infra/databases/kustomization.yaml`
|
||||
- [ ] `k8s/argocd/memory-database-app.yaml`
|
||||
- [ ] `k8s/argocd/kustomization.yaml`
|
||||
- [x] Memory-db manifest in homelab k8s/infra/databases/
|
||||
- [x] Namespace set to: poimen
|
||||
- [x] Bundled into wave 2 (no separate app)
|
||||
- [x] Homelab changes committed and pushed
|
||||
- [x] Poimen Memory repo has reference copy
|
||||
|
||||
### Homelab Sync
|
||||
- [ ] Push homelab changes:
|
||||
```bash
|
||||
cd ~/workplace/homelab
|
||||
git add k8s/argocd/projects/homelab-project.yaml
|
||||
git add k8s/argocd/apps/71-poimen-memory.yaml
|
||||
git commit -m "feat: add Poimen Memory to ArgoCD tracking (M2.2)"
|
||||
git push
|
||||
```
|
||||
- [ ] Trigger homelab sync (ArgoCD UI or CLI)
|
||||
```bash
|
||||
argocd app sync homelab-root
|
||||
```
|
||||
- [ ] Verify wave 7 (Poimen Memory Root app appears)
|
||||
## What Changed
|
||||
|
||||
### Deployment
|
||||
- [ ] Wait for wave 2 (databases sync)
|
||||
```bash
|
||||
kubectl get app -n argocd | grep poimen-memory
|
||||
# Should show: poimen-memory-root, poimen-memory-database
|
||||
```
|
||||
- [ ] Verify cluster health
|
||||
```bash
|
||||
kubectl get clusters -n poimen
|
||||
# NAME PHASE INSTANCES
|
||||
# memory-db Healthy 3/3
|
||||
```
|
||||
**Old approach (NOT USED):**
|
||||
- ❌ Separate `poimen-memory-root` app in homelab
|
||||
- ❌ Separate `memory-database-app.yaml`
|
||||
- ❌ Separate k8s/argocd/ in Poimen Memory
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### App not appearing in ArgoCD
|
||||
```bash
|
||||
# Check if homelab-root synced the new app
|
||||
argocd app get homelab-root
|
||||
# Look for poimen-memory-root in the "Application Details"
|
||||
|
||||
# Check AppProject allows the repo
|
||||
kubectl get appproject homelab -n argocd -o yaml | grep -A5 sourceRepos
|
||||
# Should include: https://github.com/Riotpiaole/Poimen-memory.git
|
||||
```
|
||||
|
||||
### Cluster not deploying
|
||||
```bash
|
||||
# Check app sync status
|
||||
argocd app get poimen-memory-root
|
||||
# Look for sync status and any error messages
|
||||
|
||||
# Check kustomize
|
||||
kubectl kustomize /Users/rockliang/workplace/Poimen/memory/k8s/infra/databases/
|
||||
# Should output memory-db Namespace + Cluster CR
|
||||
|
||||
# Check CNPG operator
|
||||
kubectl get crds | grep postgresql
|
||||
# Should show postgresql.cnpg.io
|
||||
```
|
||||
|
||||
### CNPG cluster pending
|
||||
```bash
|
||||
# Check cluster status
|
||||
kubectl describe cluster memory-db -n poimen
|
||||
|
||||
# Check node affinity
|
||||
kubectl get nodes -L kubernetes.io/hostname
|
||||
|
||||
# Check storage class
|
||||
kubectl get storageclass longhorn-cnpg
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Push Poimen Memory repo to GitHub** (required for ArgoCD to access it)
|
||||
```bash
|
||||
cd ~/workplace/Poimen/memory
|
||||
git remote add origin https://github.com/Riotpiaole/Poimen-memory.git
|
||||
git push -u origin main
|
||||
```
|
||||
|
||||
2. **Push homelab changes**
|
||||
```bash
|
||||
cd ~/workplace/homelab
|
||||
git add k8s/argocd/
|
||||
git commit -m "feat: add Poimen Memory to ArgoCD tracking"
|
||||
git push
|
||||
```
|
||||
|
||||
3. **Sync homelab ArgoCD**
|
||||
- ArgoCD detects changes automatically (or manual sync in UI)
|
||||
- Wave 7 app appears and syncs wave 2 (databases)
|
||||
|
||||
4. **Verify CNPG cluster** (M2.2 complete)
|
||||
```bash
|
||||
kubectl get clusters -n poimen
|
||||
```
|
||||
|
||||
5. **Create Poimen Memory app deployment** (next phase, M3)
|
||||
- Deployment manifest that reads `memory-db-app` secret
|
||||
- Helm chart or StatefulSet for HTTP API server
|
||||
- Runs PgRepo against `memory-db-rw.poimen.svc.cluster.local`
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
**Homelab repo:**
|
||||
- ✅ `k8s/argocd/projects/homelab-project.yaml` — Added Poimen Memory repo
|
||||
- ✅ `k8s/argocd/apps/71-poimen-memory.yaml` — New root app
|
||||
|
||||
**Poimen Memory repo:**
|
||||
- ✅ `k8s/argocd/kustomization.yaml` — New app-of-apps root
|
||||
- ✅ `k8s/argocd/memory-database-app.yaml` — Database application
|
||||
- ✅ `k8s/infra/databases/memory-db.yaml` — CNPG cluster manifest
|
||||
- ✅ `k8s/infra/databases/kustomization.yaml` — Kustomization
|
||||
**New approach (CURRENT):**
|
||||
- ✅ Memory-db bundled into homelab wave 2 "databases" app
|
||||
- ✅ Same pattern as authentik-db, temporal-db
|
||||
- ✅ Poimen Memory repo has manifests (not ArgoCD apps)
|
||||
- ✅ Simple, consistent, one deployment source
|
||||
|
||||
---
|
||||
|
||||
## Status
|
||||
|
||||
✅ **M2.2 CNPG Postgres** — Complete + ArgoCD tracked
|
||||
✅ **Ready to deploy** — Push repos and sync homelab
|
||||
⏳ **M3 Application** — Next: Poimen Memory app deployment
|
||||
✅ **M2.2 CNPG Postgres** — Complete (bundled into homelab)
|
||||
✅ **Deployed by homelab wave 2** — No separate orchestration
|
||||
✅ **Ready for next phase** — Poimen app deployment (M3)
|
||||
|
||||
|
||||
@@ -1,6 +0,0 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
# Poimen Memory ArgoCD applications (tracked by poimen-orchestrator)
|
||||
# Each app syncs from this repo's main branch
|
||||
resources:
|
||||
- memory-database-app.yaml
|
||||
@@ -1,34 +0,0 @@
|
||||
# Poimen Memory Database — CNPG Postgres cluster with pgvector
|
||||
# Synced by poimen-orchestrator (or standalone if deployed directly)
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: poimen-memory-database
|
||||
namespace: argocd
|
||||
labels:
|
||||
app.kubernetes.io/name: poimen-memory
|
||||
app.kubernetes.io/component: database
|
||||
annotations:
|
||||
argocd.argoproj.io/sync-wave: "2" # Wave 2: databases (after bootstrap, before apps)
|
||||
spec:
|
||||
project: homelab
|
||||
source:
|
||||
repoURL: https://github.com/Riotpiaole/Poimen-memory.git
|
||||
targetRevision: main
|
||||
path: k8s/infra/databases
|
||||
destination:
|
||||
server: https://kubernetes.default.svc
|
||||
namespace: default
|
||||
syncPolicy:
|
||||
automated:
|
||||
prune: true
|
||||
selfHeal: true
|
||||
syncOptions:
|
||||
- CreateNamespace=true
|
||||
- ServerSideApply=true
|
||||
retry:
|
||||
limit: 5
|
||||
backoff:
|
||||
duration: 5s
|
||||
factor: 2
|
||||
maxDuration: 3m
|
||||
Reference in New Issue
Block a user