Bundle memory database into homelab orchestration (remove separate app)
This commit is contained in:
Story Crater Bot
+70
-216
@@ -1,246 +1,100 @@
|
|||||||
# Poimen Memory — ArgoCD Tracking & Deployment
|
# Poimen Memory — ArgoCD Deployment (Bundled into Homelab)
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
✅ **Poimen Memory is tracked in homelab ArgoCD**
|
✅ **Memory database bundled into homelab wave 2 (databases)**
|
||||||
✅ **All config files in place**
|
✅ **No separate app needed**
|
||||||
✅ **Ready to deploy on next homelab sync**
|
✅ **Deployed as part of existing homelab orchestration**
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
```
|
```
|
||||||
Homelab ArgoCD Root
|
Homelab ArgoCD Root (wave 0)
|
||||||
↓ (wave 7)
|
↓
|
||||||
71-poimen-memory.yaml (homelab repo)
|
Wave 2: Databases App (syncs k8s/infra/databases)
|
||||||
↓ syncs k8s/argocd/ from Poimen Memory repo
|
├── authentik-db (ns: iam)
|
||||||
Poimen Memory k8s/argocd/
|
├── temporal-db (ns: temporal)
|
||||||
├── kustomization.yaml
|
└── memory-db (ns: poimen) ← NEW
|
||||||
└── memory-database-app.yaml
|
├── Cluster: memory-db (3 instances)
|
||||||
↓ syncs k8s/infra/databases/
|
├── Extension: pgvector (768-dim embeddings)
|
||||||
Memory CNPG Cluster (wave 2 within Poimen Memory)
|
├── Secret: memory-db-app (auto-generated in poimen)
|
||||||
|
└── Service: memory-db-rw (auto-generated in poimen)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
### How It Works
|
||||||
|
|
||||||
|
Memory database is deployed **as part of homelab wave 2** (same wave as authentik-db and temporal-db):
|
||||||
|
|
||||||
|
1. **Homelab repository** contains the cluster definition
|
||||||
|
- `k8s/infra/databases/memory-db.yaml` (deployed to: poimen namespace)
|
||||||
|
- `k8s/infra/databases/kustomization.yaml` (includes memory-db.yaml)
|
||||||
|
|
||||||
|
2. **Homelab "databases" app** (wave 2) syncs these manifests
|
||||||
|
- No separate poimen-memory app needed
|
||||||
|
- Bundles all DB clusters in one wave
|
||||||
|
- Simple and consistent
|
||||||
|
|
||||||
|
3. **Poimen Memory repo** contains reference copies
|
||||||
|
- `k8s/infra/databases/memory-db.yaml` (for CI/documentation)
|
||||||
|
- Deployed via homelab, not its own app
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# After homelab syncs wave 2:
|
||||||
|
kubectl get clusters -n poimen
|
||||||
|
# NAME PHASE INSTANCES
|
||||||
|
# memory-db Healthy 3/3
|
||||||
|
|
||||||
|
kubectl get secret -n poimen memory-db-app
|
||||||
|
# memory-db-app kubernetes.io/basic-auth 2
|
||||||
|
|
||||||
|
kubectl get svc -n poimen memory-db-rw
|
||||||
|
# memory-db-rw ClusterIP 10.x.x.x 5432/TCP
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Configuration Files
|
## Connection String
|
||||||
|
|
||||||
### 1. Homelab Repository
|
|
||||||
**Location:** `/Users/rockliang/workplace/homelab/k8s/argocd/`
|
|
||||||
|
|
||||||
**File 1: `projects/homelab-project.yaml` (UPDATED)**
|
|
||||||
```yaml
|
|
||||||
sourceRepos:
|
|
||||||
- https://github.com/Riotpiaole/riotpiao.homelab.com.git
|
|
||||||
- https://github.com/Riotpiaole/Poimen-memory.git # ← ADDED
|
|
||||||
- https://forgejo.riotpiao.com/rock/*
|
|
||||||
# ... public Helm repos
|
|
||||||
```
|
|
||||||
**What it does:** Authorizes ArgoCD to sync from Poimen Memory repo
|
|
||||||
|
|
||||||
**File 2: `apps/71-poimen-memory.yaml` (NEW)**
|
|
||||||
```yaml
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: poimen-memory-root
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "7"
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://github.com/Riotpiaole/Poimen-memory.git
|
|
||||||
path: k8s/argocd
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
```
|
|
||||||
**What it does:** Root application that syncs the Poimen Memory app-of-apps
|
|
||||||
|
|
||||||
### 2. Poimen Memory Repository
|
|
||||||
**Location:** `/Users/rockliang/workplace/Poimen/memory/k8s/argocd/`
|
|
||||||
|
|
||||||
**File 1: `kustomization.yaml` (NEW)**
|
|
||||||
```yaml
|
|
||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
||||||
kind: Kustomization
|
|
||||||
resources:
|
|
||||||
- memory-database-app.yaml
|
|
||||||
```
|
|
||||||
**What it does:** Declares the application manifests for Poimen Memory
|
|
||||||
|
|
||||||
**File 2: `memory-database-app.yaml` (NEW)**
|
|
||||||
```yaml
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: poimen-memory-database
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "2" # Wave 2 = databases
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://github.com/Riotpiaole/Poimen-memory.git
|
|
||||||
path: k8s/infra/databases
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
```
|
|
||||||
**What it does:** Syncs the CNPG Postgres cluster config from `k8s/infra/databases/`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Deployment Wave Order
|
|
||||||
|
|
||||||
```
|
```
|
||||||
Wave 0: Homelab Root (homelab-root.yaml)
|
postgresql://app:<password>@memory-db-rw.poimen.svc.cluster.local:5432/memory?sslmode=disable
|
||||||
↓
|
|
||||||
Wave 2: Databases (k8s/infra/databases/)
|
|
||||||
├── 40-data.yaml (homelab databases)
|
|
||||||
│ ├── authentik-db
|
|
||||||
│ ├── temporal-db
|
|
||||||
│ └── memory-db (from k8s/infra/databases/kustomization.yaml)
|
|
||||||
└── 71-poimen-memory.yaml syncs its own wave 2 (memory-database-app)
|
|
||||||
↓
|
|
||||||
Wave 7: Applications
|
|
||||||
├── Poimen Memory Root (poimen-memory-root)
|
|
||||||
├── Poimen Services (future)
|
|
||||||
└── Other applications
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Credentials:** `kubectl get secret -n poimen memory-db-app`
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Deployment Checklist
|
## Deployment Checklist
|
||||||
|
|
||||||
### Prerequisites
|
- [x] Memory-db manifest in homelab k8s/infra/databases/
|
||||||
- [ ] Poimen Memory repo exists on GitHub at `Riotpiaole/Poimen-memory`
|
- [x] Namespace set to: poimen
|
||||||
- [ ] Main branch is up-to-date with:
|
- [x] Bundled into wave 2 (no separate app)
|
||||||
- [ ] `k8s/infra/databases/memory-db.yaml`
|
- [x] Homelab changes committed and pushed
|
||||||
- [ ] `k8s/infra/databases/kustomization.yaml`
|
- [x] Poimen Memory repo has reference copy
|
||||||
- [ ] `k8s/argocd/memory-database-app.yaml`
|
|
||||||
- [ ] `k8s/argocd/kustomization.yaml`
|
|
||||||
|
|
||||||
### Homelab Sync
|
## What Changed
|
||||||
- [ ] Push homelab changes:
|
|
||||||
```bash
|
|
||||||
cd ~/workplace/homelab
|
|
||||||
git add k8s/argocd/projects/homelab-project.yaml
|
|
||||||
git add k8s/argocd/apps/71-poimen-memory.yaml
|
|
||||||
git commit -m "feat: add Poimen Memory to ArgoCD tracking (M2.2)"
|
|
||||||
git push
|
|
||||||
```
|
|
||||||
- [ ] Trigger homelab sync (ArgoCD UI or CLI)
|
|
||||||
```bash
|
|
||||||
argocd app sync homelab-root
|
|
||||||
```
|
|
||||||
- [ ] Verify wave 7 (Poimen Memory Root app appears)
|
|
||||||
|
|
||||||
### Deployment
|
**Old approach (NOT USED):**
|
||||||
- [ ] Wait for wave 2 (databases sync)
|
- ❌ Separate `poimen-memory-root` app in homelab
|
||||||
```bash
|
- ❌ Separate `memory-database-app.yaml`
|
||||||
kubectl get app -n argocd | grep poimen-memory
|
- ❌ Separate k8s/argocd/ in Poimen Memory
|
||||||
# Should show: poimen-memory-root, poimen-memory-database
|
|
||||||
```
|
|
||||||
- [ ] Verify cluster health
|
|
||||||
```bash
|
|
||||||
kubectl get clusters -n poimen
|
|
||||||
# NAME PHASE INSTANCES
|
|
||||||
# memory-db Healthy 3/3
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
**New approach (CURRENT):**
|
||||||
|
- ✅ Memory-db bundled into homelab wave 2 "databases" app
|
||||||
## Troubleshooting
|
- ✅ Same pattern as authentik-db, temporal-db
|
||||||
|
- ✅ Poimen Memory repo has manifests (not ArgoCD apps)
|
||||||
### App not appearing in ArgoCD
|
- ✅ Simple, consistent, one deployment source
|
||||||
```bash
|
|
||||||
# Check if homelab-root synced the new app
|
|
||||||
argocd app get homelab-root
|
|
||||||
# Look for poimen-memory-root in the "Application Details"
|
|
||||||
|
|
||||||
# Check AppProject allows the repo
|
|
||||||
kubectl get appproject homelab -n argocd -o yaml | grep -A5 sourceRepos
|
|
||||||
# Should include: https://github.com/Riotpiaole/Poimen-memory.git
|
|
||||||
```
|
|
||||||
|
|
||||||
### Cluster not deploying
|
|
||||||
```bash
|
|
||||||
# Check app sync status
|
|
||||||
argocd app get poimen-memory-root
|
|
||||||
# Look for sync status and any error messages
|
|
||||||
|
|
||||||
# Check kustomize
|
|
||||||
kubectl kustomize /Users/rockliang/workplace/Poimen/memory/k8s/infra/databases/
|
|
||||||
# Should output memory-db Namespace + Cluster CR
|
|
||||||
|
|
||||||
# Check CNPG operator
|
|
||||||
kubectl get crds | grep postgresql
|
|
||||||
# Should show postgresql.cnpg.io
|
|
||||||
```
|
|
||||||
|
|
||||||
### CNPG cluster pending
|
|
||||||
```bash
|
|
||||||
# Check cluster status
|
|
||||||
kubectl describe cluster memory-db -n poimen
|
|
||||||
|
|
||||||
# Check node affinity
|
|
||||||
kubectl get nodes -L kubernetes.io/hostname
|
|
||||||
|
|
||||||
# Check storage class
|
|
||||||
kubectl get storageclass longhorn-cnpg
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Next Steps
|
|
||||||
|
|
||||||
1. **Push Poimen Memory repo to GitHub** (required for ArgoCD to access it)
|
|
||||||
```bash
|
|
||||||
cd ~/workplace/Poimen/memory
|
|
||||||
git remote add origin https://github.com/Riotpiaole/Poimen-memory.git
|
|
||||||
git push -u origin main
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Push homelab changes**
|
|
||||||
```bash
|
|
||||||
cd ~/workplace/homelab
|
|
||||||
git add k8s/argocd/
|
|
||||||
git commit -m "feat: add Poimen Memory to ArgoCD tracking"
|
|
||||||
git push
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Sync homelab ArgoCD**
|
|
||||||
- ArgoCD detects changes automatically (or manual sync in UI)
|
|
||||||
- Wave 7 app appears and syncs wave 2 (databases)
|
|
||||||
|
|
||||||
4. **Verify CNPG cluster** (M2.2 complete)
|
|
||||||
```bash
|
|
||||||
kubectl get clusters -n poimen
|
|
||||||
```
|
|
||||||
|
|
||||||
5. **Create Poimen Memory app deployment** (next phase, M3)
|
|
||||||
- Deployment manifest that reads `memory-db-app` secret
|
|
||||||
- Helm chart or StatefulSet for HTTP API server
|
|
||||||
- Runs PgRepo against `memory-db-rw.poimen.svc.cluster.local`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Files Modified
|
|
||||||
|
|
||||||
**Homelab repo:**
|
|
||||||
- ✅ `k8s/argocd/projects/homelab-project.yaml` — Added Poimen Memory repo
|
|
||||||
- ✅ `k8s/argocd/apps/71-poimen-memory.yaml` — New root app
|
|
||||||
|
|
||||||
**Poimen Memory repo:**
|
|
||||||
- ✅ `k8s/argocd/kustomization.yaml` — New app-of-apps root
|
|
||||||
- ✅ `k8s/argocd/memory-database-app.yaml` — Database application
|
|
||||||
- ✅ `k8s/infra/databases/memory-db.yaml` — CNPG cluster manifest
|
|
||||||
- ✅ `k8s/infra/databases/kustomization.yaml` — Kustomization
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
✅ **M2.2 CNPG Postgres** — Complete + ArgoCD tracked
|
✅ **M2.2 CNPG Postgres** — Complete (bundled into homelab)
|
||||||
✅ **Ready to deploy** — Push repos and sync homelab
|
✅ **Deployed by homelab wave 2** — No separate orchestration
|
||||||
⏳ **M3 Application** — Next: Poimen Memory app deployment
|
✅ **Ready for next phase** — Poimen app deployment (M3)
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +0,0 @@
|
|||||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
|
||||||
kind: Kustomization
|
|
||||||
# Poimen Memory ArgoCD applications (tracked by poimen-orchestrator)
|
|
||||||
# Each app syncs from this repo's main branch
|
|
||||||
resources:
|
|
||||||
- memory-database-app.yaml
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
# Poimen Memory Database — CNPG Postgres cluster with pgvector
|
|
||||||
# Synced by poimen-orchestrator (or standalone if deployed directly)
|
|
||||||
apiVersion: argoproj.io/v1alpha1
|
|
||||||
kind: Application
|
|
||||||
metadata:
|
|
||||||
name: poimen-memory-database
|
|
||||||
namespace: argocd
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: poimen-memory
|
|
||||||
app.kubernetes.io/component: database
|
|
||||||
annotations:
|
|
||||||
argocd.argoproj.io/sync-wave: "2" # Wave 2: databases (after bootstrap, before apps)
|
|
||||||
spec:
|
|
||||||
project: homelab
|
|
||||||
source:
|
|
||||||
repoURL: https://github.com/Riotpiaole/Poimen-memory.git
|
|
||||||
targetRevision: main
|
|
||||||
path: k8s/infra/databases
|
|
||||||
destination:
|
|
||||||
server: https://kubernetes.default.svc
|
|
||||||
namespace: default
|
|
||||||
syncPolicy:
|
|
||||||
automated:
|
|
||||||
prune: true
|
|
||||||
selfHeal: true
|
|
||||||
syncOptions:
|
|
||||||
- CreateNamespace=true
|
|
||||||
- ServerSideApply=true
|
|
||||||
retry:
|
|
||||||
limit: 5
|
|
||||||
backoff:
|
|
||||||
duration: 5s
|
|
||||||
factor: 2
|
|
||||||
maxDuration: 3m
|
|
||||||
Reference in New Issue
Block a user