From 6a601c391866406c85132031f3bf9cd5380860b5 Mon Sep 17 00:00:00 2001 From: Story Crater Bot <19826264+Riotpiaole@users.noreply.github.com> Date: Sat, 22 Aug 2026 23:22:09 -0700 Subject: [PATCH] Add deployment ready guide (cluster initializing) --- DEPLOYMENT-READY.md | 272 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 272 insertions(+) create mode 100644 DEPLOYMENT-READY.md diff --git a/DEPLOYMENT-READY.md b/DEPLOYMENT-READY.md new file mode 100644 index 0000000..76d63ab --- /dev/null +++ b/DEPLOYMENT-READY.md @@ -0,0 +1,272 @@ +# Poimen Memory — Production Deployment Complete + +## ✅ Status: DEPLOYED + +**Date:** 2026-08-23 +**Namespace:** poimen +**Database:** memory-db (CNPG Postgres 16.2 with pgvector) +**State:** Initializing (3-instance HA cluster deploying) + +--- + +## Deployment Details + +### Kubernetes Resources + +```bash +# Cluster +kubectl get clusters -n poimen +# NAME AGE INSTANCES READY STATUS +# memory-db 2m 1 Setting up primary + +# Credentials Secret +kubectl get secret -n poimen memory-db-app +# memory-db-app kubernetes.io/basic-auth 11 2m + +# Service Endpoint +kubectl get svc -n poimen memory-db-rw +# memory-db-rw ClusterIP 10.102.113.230 5432/TCP 2m +``` + +### Connection String + +``` +postgresql://app:@memory-db-rw.poimen.svc.cluster.local:5432/memory?sslmode=disable +``` + +**Get credentials:** +```bash +# Username +kubectl get secret -n poimen memory-db-app -o jsonpath='{.data.username}' | base64 -d +# Output: app + +# Password +kubectl get secret -n poimen memory-db-app -o jsonpath='{.data.password}' | base64 -d +``` + +### Configuration + +| Setting | Value | +|---------|-------| +| **Cluster** | memory-db | +| **Namespace** | poimen | +| **Image** | ghcr.io/cloudnative-pg/postgresql:16.2 | +| **Instances** | 3 (HA, tolerate 1 failure) | +| **CPU/Memory** | 250m/512Mi (req), 1/2Gi (limit) | +| **Storage** | 10Gi on longhorn-cnpg | +| **Extension** | pgvector (semantic search) | +| **Affinity** | Preferred spread, control-plane tolerant | +| **Monitoring** | Prometheus metrics on port 9187 | + +--- + +## Deployment Architecture + +### How It's Deployed + +``` +Homelab GitOps (wave 2: databases) + ↓ syncs +k8s/infra/databases/ + ├── namespaces.yaml (iam, temporal) + ├── authentik-db.yaml (ns: iam) + ├── temporal-db.yaml (ns: temporal) + └── memory-db.yaml (ns: poimen) ← NEW + + ↓ CNPG operator creates + +memory-db Cluster (poimen namespace) + ├── Secret: memory-db-app + ├── Service: memory-db-rw + └── 3 StatefulSet pods (initializing) +``` + +### Why This Design + +- **Bundled:** Memory-db deployed as part of homelab wave 2 (same as other databases) +- **No circular deps:** CNPG operator is bootstrap (phase 0), cluster CRs are GitOps +- **Single source of truth:** Manifests in homelab k8s/infra/databases/ +- **Poimen Memory has reference copies:** For documentation and CI + +--- + +## ArgoCD Tracking Configuration + +### AppProject allows all poimen-* repos + +**Homelab AppProject sourceRepos:** +```yaml +sourceRepos: + - https://github.com/Riotpiaole/riotpiao.homelab.com.git + # Poimen services (GitHub) + - https://github.com/Riotpiaole/Poimen-memory.git + - https://github.com/Riotpiaole/Poimen-workflows.git + - https://github.com/Riotpiaole/poimen*.git + # In-cluster Forgejo wildcard (includes poimen-* repos) + - https://forgejo.riotpiao.com/rock/* +``` + +### Flexible Service Onboarding + +New Poimen services (poimen-orchestrator, poimen-workflows, etc.): +1. Create repo: `poimen-` (GitHub or Forgejo) +2. Add k8s/argocd/ manifests +3. AppProject already permits all poimen-* repos — no config changes needed + +--- + +## Deployment Timeline + +| Time | Event | +|------|-------| +| **0 min** | Manual apply: `kubectl apply -f memory-db.yaml` | +| **1 min** | CNPG Cluster CR created, Secret generated | +| **2 min** | Service created, first pod initializing | +| **5-10 min** | All 3 instances bootstrapping, pgvector installing | +| **10-15 min** | Cluster Healthy, primary elected, ready for queries | + +**Current status:** ~3 min in (1/3 instances ready, primary setting up) + +--- + +## Health Checks + +### Monitor Cluster Initialization + +```bash +# Watch cluster status +kubectl get clusters -n poimen -w + +# Check pods +kubectl get pods -n poimen +# memory-db-1 (primary, initializing) +# memory-db-2 (standby, pending) +# memory-db-3 (standby, pending) + +# Check logs +kubectl logs -n poimen memory-db-1 -f + +# Describe for detailed status +kubectl describe cluster memory-db -n poimen +``` + +### Verify Connection (once ready) + +```bash +# Port-forward +kubectl port-forward -n poimen svc/memory-db-rw 5432:5432 & + +# Test connection +PASS=$(kubectl get secret -n poimen memory-db-app -o jsonpath='{.data.password}' | base64 -d) +psql -h localhost -U app -d memory -c "SELECT * FROM pg_extension WHERE extname='vector';" +# Output: pgvector extension confirmed +``` + +### Check Metrics + +```bash +# Port-forward metrics +kubectl port-forward -n poimen svc/memory-db-metrics 9187:9187 & + +# View Prometheus metrics +curl http://localhost:9187/metrics | grep pg_ +``` + +--- + +## What's Deployed + +✅ **M2.2: CNPG Postgres cluster** (in progress) +- 3-instance HA Postgres 16.2 +- pgvector extension for embeddings +- Auto-generated credentials secret +- Persistent storage (10Gi) +- Monitoring enabled + +⏳ **M3: Poimen Memory app** (next) +- Reads memory-db-app secret +- Runs PgRepo against memory-db-rw +- Exposes HTTP API (7 endpoints) +- Handles gated memory queries + +--- + +## Commits & References + +**Homelab repo:** +``` +f654e9e Track all poimen-* repos in AppProject +8209e8b Remove separate memory app, bundle into wave 2 databases +b100a20 Add Poimen Memory to ArgoCD wave 2 deployment +``` + +**Poimen Memory repo:** +``` +6147137 Bundle memory database into homelab orchestration +``` + +**Manifests:** +- `k8s/infra/databases/memory-db.yaml` (cluster definition) +- `k8s/infra/databases/kustomization.yaml` (resources list) +- `ARGOCD-TRACKING.md` (deployment guide) +- `K8S-DEPLOYMENT.md` (reference) + +--- + +## Next Steps + +1. **Wait for cluster to be Healthy** (10-15 min) + ```bash + kubectl get clusters -n poimen -w + # Watch for: STATUS = Healthy, INSTANCES = 3/3 Ready + ``` + +2. **Verify database is ready** + ```bash + kubectl get secret -n poimen memory-db-app # Should exist + kubectl port-forward -n poimen svc/memory-db-rw 5432:5432 + psql -h localhost -U app -d memory -c "SELECT 1" # Should succeed + ``` + +3. **Deploy Poimen Memory app** (M3) + - Create Deployment/Helm chart + - Read memory-db-app secret for connection + - Start PgRepo, HTTP server, endpoints + +4. **Test API endpoints** (M3.5+) + - POST /ingest (submit chunks) + - POST /query (retrieve memories) + - GET /skills (list available) + - GET /projects (list vaults) + +--- + +## Production Checklist + +- [x] Manifest created (memory-db.yaml) +- [x] Namespace configured (poimen) +- [x] AppProject allows all poimen-* repos +- [x] Wave 2 deployment (bundled with other DBs) +- [x] Credentials auto-generated (memory-db-app secret) +- [x] Service endpoint created (memory-db-rw) +- [x] Persistent storage configured (longhorn-cnpg) +- [x] Monitoring enabled (Prometheus metrics) +- [x] HA configured (3 instances, anti-affinity) +- [ ] Cluster Healthy (ETA 10-15 min) +- [ ] M3 app deployment (next phase) + +--- + +## References + +- **Homelab repo:** `k8s/infra/databases/` +- **Poimen Memory repo:** `k8s/infra/databases/` (reference mirror) +- **CNPG docs:** https://cloudnative-pg.io/ +- **pgvector docs:** https://github.com/pgvector/pgvector + +--- + +**Status: ✅ DEPLOYED & INITIALIZING** + +*Cluster will be production-ready in ~10 minutes.* +