diff --git a/CI-FOR-ALL-POIMEN-REPOS.md b/CI-FOR-ALL-POIMEN-REPOS.md deleted file mode 100644 index 88336d7..0000000 --- a/CI-FOR-ALL-POIMEN-REPOS.md +++ /dev/null @@ -1,224 +0,0 @@ -# CI/CD for All Poimen Repos — Standardized Pattern - -## Overview - -All Poimen repos should follow the same CI/CD pattern for consistency and maintainability. - -**Pattern**: Test locally → Build image → Push to registry → ArgoCD deploys - -**Based on**: homelab-frontend (proven production pattern) - ---- - -## Repos & Status - -### Repos That Need Docker Deployment - -| Repo | Status | Dockerfile | Notes | -|------|--------|-----------|-------| -| **poimen-memory** | ✅ Ready | Yes | This repo - see `.forgejo/workflows/build.yaml` | -| **poimen** | ⏳ TBD | Yes (assumed) | Orchestrator - needs deployment | -| **poimen-workflows** | ⏳ TBD | Maybe | Check if containerized | - -### Repos That Don't Need Docker - -| Repo | Status | Type | Notes | -|------|--------|------|-------| -| **homelab** | ✅ Done | K8s manifests | Validates with yamllint + kubeval | - ---- - -## Implementation Checklist for Each Repo - -### Step 0: Prerequisites -- [ ] Repo has a `Dockerfile` -- [ ] Repo has a `.forgejo/` or `.gitea/` directory -- [ ] Docker builds successfully: `docker build -t test:latest .` -- [ ] Tests pass: `cargo test` / `npm test` / etc - -### Step 1: Create Workflow File -```bash -# Copy from poimen-memory: -cp ~/workplace/Poimen/memory/.forgejo/workflows/build.yaml \ - ~/workplace/Poimen//.forgejo/workflows/build.yaml - -# Edit if needed: -# - Change IMAGE_NAME from "rock/poimen-memory" to "rock/" -# - Adjust test command if not Rust (cargo test) -``` - -### Step 2: Set Repository Secret -``` -https://git.riotpiao.com/rock//settings/secrets - -Add: -- Name: REGISTRY_PAT -- Value: -``` - -### Step 3: Commit & Push -```bash -git add .forgejo/workflows/build.yaml -git commit -m "Add CI/CD: auto-build and push to registry" -git push origin main -``` - -### Step 4: Create ArgoCD Application -```bash -# Create k8s/argocd/-app.yaml - -apiVersion: argoproj.io/v1alpha1 -kind: Application -metadata: - name: poimen--app - namespace: argocd -spec: - project: homelab - source: - repoURL: https://forgejo.riotpiao.com/rock/poimen-.git - targetRevision: main - path: k8s/app # adjust if different - destination: - server: https://kubernetes.default.svc - namespace: poimen - syncPolicy: - automated: - prune: true - selfHeal: true -``` - -### Step 5: Apply Application -```bash -kubectl apply -f k8s/argocd/-app.yaml -``` - -### Done! ✅ -- Every push to main triggers: - 1. Test suite - 2. Docker build - 3. Push to `forgejo.riotpiao.com/rock/:latest` - 4. ArgoCD auto-deploys - ---- - -## File Reference - -### Workflow Comparison - -**poimen-memory** (current): -```yaml -runs-on: golang -container: - image: docker:27-cli - volumes: - - /docker-certs/client:/docker-certs/client:ro -env: - DOCKER_HOST: tcp://localhost:2376 - DOCKER_TLS_VERIFY: "1" - DOCKER_CERT_PATH: /docker-certs/client -``` - -**Why this setup:** -- Runs on `golang` runner (has Docker daemon) -- Uses Docker CLI in container with DinD (Docker-in-Docker) -- TLS certs mounted for secure daemon access -- Allows building AND pushing in same job - -### Test Job - -Adjust for your language: - -**Rust** (poimen-memory): -```yaml -runs-on: rust -steps: - - uses: actions/checkout@v4 - - run: cargo test --all -``` - -**Go**: -```yaml -runs-on: golang -steps: - - uses: actions/checkout@v4 - - run: go test ./... -``` - -**Node.js**: -```yaml -runs-on: docker -steps: - - uses: actions/checkout@v4 - - run: npm install && npm test -``` - ---- - -## Organization-Wide Setup - -### One-Time: Set Organization Secret -Instead of per-repo secrets, Forgejo supports organization secrets. - -**If available**, set `REGISTRY_PAT` at org level: -``` -https://git.riotpiao.com/rock/settings/secrets -``` - -Then all repos automatically inherit it (no per-repo setup needed). - -**Check**: Try accessing org secrets settings -- If available: set once, use everywhere -- If not: set per-repo (5 minutes per repo) - ---- - -## Monitoring & Troubleshooting - -### Build Failures - -**Check logs:** -``` -https://git.riotpiao.com/rock//actions -``` - -**Common issues:** -- Test failures → Fix tests locally -- Docker build error → Check Dockerfile syntax -- Push fails → Verify REGISTRY_PAT token - -### Deployment Issues - -**Watch ArgoCD:** -```bash -kubectl get application -n argocd poimen--app -w -kubectl logs -n argocd argocd-application-controller | grep poimen -``` - -**Check pods:** -```bash -kubectl get pods -n poimen -l app.kubernetes.io/name=poimen- -w -kubectl describe pod -n poimen -``` - ---- - -## Summary - -**Effort**: ~10 minutes per repo (once) - -**Benefit**: -- Zero-touch deployments -- Every commit automatically tested & deployed -- Consistent across organization -- No manual image pushes ever - -**Best practice**: Use org-level secret if available (1 setup, unlimited repos) - ---- - -## Next Steps - -1. **poimen-memory**: ✅ Done (this repo) -2. **poimen**: Set up workflow + secret -3. **poimen-workflows**: Set up workflow + secret -4. **Document in**: homelab-poimen-standard.md (org wiki) diff --git a/CI-SETUP.md b/CI-SETUP.md deleted file mode 100644 index e8fff1a..0000000 --- a/CI-SETUP.md +++ /dev/null @@ -1,147 +0,0 @@ -# CI/CD Setup — Registry Push Configuration - -## One-Time Setup - -The CI/CD pipeline automatically builds and pushes Docker images when you push to `main`. - -### 1. Create or Get Registry Token - -**Option A: Use Organization Token** (Recommended) -```bash -# Ask Rock for the existing 'rock' organization PAT -# It should already have write:package permissions -``` - -**Option B: Create Personal Token** -```bash -# In browser: https://git.riotpiao.com/user/settings/tokens -# 1. Click "Generate New Token" -# 2. Name: "Docker Registry" -# 3. Scope: Check `write:package` -# 4. Generate and copy the token -``` - -### 2. Add Repository Secret - -Go to: **https://git.riotpiao.com/rock/poimen-memory/settings/secrets** - -Add secret: -- **Name**: `REGISTRY_PAT` -- **Value**: `` -- **Save** - -### 3. Verify Setup - -```bash -# Push a commit (any change will do) -cd ~/workplace/Poimen/memory -git commit --allow-empty -m "Trigger CI build" -git push origin main - -# Check Actions tab -# https://git.riotpiao.com/rock/poimen-memory/actions -``` - ---- - -## How It Works - -``` -Push to main - ↓ -Forgejo Actions triggered - ↓ -Test: cargo test --all - ↓ (only if tests pass) -Build: docker build -t forgejo.riotpiao.com/rock/poimen-memory:latest . - ↓ -Push: docker push (using REGISTRY_PAT secret) - ↓ -ArgoCD detects new image - ↓ -Auto-deploy to poimen namespace -``` - ---- - -## Check Status - -**Web UI** — See build progress: -``` -https://git.riotpiao.com/rock/poimen-memory/actions -``` - -**CLI** — Watch deployment: -```bash -kubectl get application -n argocd poimen-memory-app -w -kubectl get pods -n poimen -l app.kubernetes.io/name=poimen-memory -w -``` - -**Verify Image** — Check registry: -```bash -docker pull forgejo.riotpiao.com/rock/poimen-memory:latest -``` - ---- - -## Once Image is Ready - -```bash -# Port forward to local -kubectl port-forward -n poimen svc/poimen-memory 8080:80 & - -# Test -curl http://localhost:8080/health -``` - ---- - -## Troubleshooting - -### Secret Not Found Error -- Go to: https://git.riotpiao.com/rock/poimen-memory/settings/secrets -- Verify `REGISTRY_PAT` is set - -### Login Failed -- Token might be expired or revoked -- Create a new token and update the secret - -### Build Failed -- Check Actions logs for the error -- Usually: tests failed -- Fix locally: `cargo test --all` - -### Image Exists But Pods Not Running -- Check pod events: `kubectl describe pod -n poimen ` -- Usually: image pull policy issue or pod crashed -- Check logs: `kubectl logs -n poimen deployment/poimen-memory` - ---- - -## Apply to Other Repos - -The same setup works for all Poimen repos: - -```bash -# For poimen, poimen-workflows, etc: -# 1. Create .forgejo/workflows/build.yaml (copy from template below) -# 2. Add REGISTRY_PAT secret -# 3. Push and watch it deploy -``` - -**Template**: See `.forgejo/workflows/TEMPLATE.md` in this repo - ---- - -## Pattern Overview - -**Based on**: homelab-frontend (proven production pattern) -- Uses `REGISTRY_PAT` secret ✓ -- Docker login + push ✓ -- Tags: commit SHA + latest ✓ -- ArgoCD watches tags ✓ - -**Consistency**: All Poimen repos use same pattern -- Same secret name: `REGISTRY_PAT` -- Same workflow structure -- Same deployment process diff --git a/DESIGN_SUMMARY.md b/DESIGN_SUMMARY.md deleted file mode 100644 index 35a7a69..0000000 --- a/DESIGN_SUMMARY.md +++ /dev/null @@ -1,387 +0,0 @@ -# Query Optimization & Hybrid Search Design — Complete - -## What Was Built - -### ✅ 1. Query Optimization Engine (`query_optimizer.rs` - 450 LOC) - -**6-stage pipeline for understanding queries:** - -1. **Normalization** — Lowercase, trim whitespace -2. **Tokenization** — Break into words -3. **Entity Extraction** — Find years, quoted phrases, tags -4. **Characteristic Analysis** — Detect dates, negation, special syntax -5. **Question Classification** — Procedural vs Factual vs Troubleshooting, etc -6. **Search Strategy Routing** — Choose optimal retrieval method - -**Output:** `QueryContext` + `SearchStrategy` + `Confidence` - -```rust -pub enum SearchStrategy { - Hybrid, // Both pgvector + OpenSearch (best accuracy) - SemanticOnly, // pgvector only (fallback) - LexicalOnly, // OpenSearch only (fallback) - LexicalFirst, // OpenSearch narrow → pgvector rerank (fastest) -} -``` - -**Key Features:** -- ✅ RRF (Reciprocal Rank Fusion) algorithm — no parameter tuning -- ✅ Cascading strategy support — multi-stage retrieval -- ✅ 15+ unit tests -- ✅ Zero external dependencies (pure logic) - ---- - -### ✅ 2. Hybrid Query Worker (`hybrid_query_worker.rs` - 380 LOC) - -**Orchestrates parallel retrieval across engines:** - -- **Stage 1**: Route query using QueryOptimizer -- **Stage 2**: Generate embedding (LLM) -- **Stage 3**: Execute parallel queries - - pgvector semantic (top-50) - - OpenSearch lexical (top-50) with JWT auth -- **Stage 4**: Fuse results using RRF -- **Stage 5**: Build rich response with score breakdown - -**Output:** `HybridQueryResponse` with: -- Top-10 results -- Score breakdown (semantic + lexical components) -- Metrics (latency, engine counts, fusion method) -- Retrieval engine used - -**Strategies Supported:** -- HYBRID: Parallel pgvector + OpenSearch → RRF fusion -- CASCADING: OpenSearch narrow (200) → pgvector rerank (10) -- SEMANTIC: pgvector only (fallback) -- LEXICAL: OpenSearch only (fallback) - ---- - -### ✅ 3. Design Documentation (5 comprehensive documents) - -#### **QUERY_OPTIMIZATION_ENGINE.md** (500+ LOC) -- **Executive summary** — Why Approach A (Parallel RRF) -- **Architecture overview** — Complete data flow -- **6-stage pipeline** — Detailed implementation of QueryOptimizer -- **Question classification** — Type detection + routing examples -- **Search strategy routing** — Decision tree with confidence scores -- **RRF algorithm** — Why RRF > Weighted Linear, formula, Rust code -- **Response format** — API contract with score breakdown -- **Integration path** — How to update /memory/query endpoint -- **4-phase implementation plan** — Week 1-4 deliverables -- **Testing checklist** — Unit + integration + A/B testing -- **Configuration reference** — Env vars + tuning parameters - -#### **HYBRID_SEARCH_DESIGN.md** (760+ LOC) -- 5-stage retrieval pipeline (normalize → parallel → normalize → fuse → rank) -- Index optimization for pgvector (HNSW, filtering, queries) -- Index optimization for OpenSearch (BM25, field boosts, analyzers) -- Accuracy metrics (MRR, NDCG@10, Precision@K, Recall@K) -- Query routing decision tree -- Weight tuning strategy (A/B testing framework) -- Indexing pipeline (write side) -- Testing strategy with fixtures - -#### **API_REVIEW.md** (400+ LOC) -- 10 endpoints reviewed (health, ingest, query, vault-*, etc) -- Distinction: Query APIs vs Retrieval APIs -- Current implementation gaps -- Recommended Phase 1-4 enhancements -- Architecture changes needed -- Implementation checklist - -#### **IMPLEMENTATION_NOTES.md** (280+ LOC) -- Compilation status (non-blocking API mismatches noted) -- VectorStore API corrections -- OpenSearchClient API fixes -- Phase 2 checklist (5-day implementation) -- Code diff preview -- Design validation matrix - -#### **memory-flow.md** (updated - 833 LOC) -- Complete retrieval pipeline diagram (5 stages) -- Query routing decision tree -- Index optimization details -- Pod infrastructure (now 8 core pods) -- Deployment checklist reorganized - ---- - -## Architecture Decision: Approach A (Parallel RRF) - -### Why This Approach? - -| Criterion | Score | Reasoning | -|-----------|-------|-----------| -| **Accuracy** | ⭐⭐⭐⭐⭐ | Semantic + Lexical covers all cases | -| **Fault Tolerance** | ⭐⭐⭐⭐⭐ | Fallback to semantic if OpenSearch down | -| **No False Negatives** | ⭐⭐⭐⭐⭐ | Semantic catches synonyms lexical misses | -| **Debugging** | ⭐⭐⭐⭐⭐ | Clear score breakdown for transparency | -| **Decoupled** | ⭐⭐⭐⭐⭐ | Embedding model changes don't break system | -| **Latency** | ⭐⭐⭐ | 150-250ms (parallel) vs 60-100ms (single engine) | -| **Complexity** | ⭐⭐⭐ | Moderate RRF logic + parallel orchestration | - -**Mission-critical for agent reasoning:** Agents make decisions based on retrieved context. Missing docs = wrong decisions. - ---- - -## Key Components - -### 1. QueryOptimizer (Pure Logic) - -```rust -optimizer.optimize_query("How do I fix kubernetes port 8080?") - → QueryContext { - raw_query: "How do I fix kubernetes port 8080?", - normalized: "how do i fix kubernetes port 8080?", - tokens: ["how", "do", "i", "fix", "kubernetes", "port", "8080"], - entities: {}, - token_count: 7, - has_special_syntax: false, - has_date_filters: false, - has_negation: false, - question_type: Procedural, - search_strategy: Hybrid, - confidence: 0.95, - } -``` - -### 2. HybridQueryWorker (Parallel Orchestration) - -```rust -worker.query("poimen", "How do I fix kubernetes port 8080?", 10, &jwt) - → HybridQueryResponse { - query: "How do I fix kubernetes port 8080?", - project: "poimen", - search_strategy: "Hybrid", - strategy_confidence: 0.95, - results: [ - { - id: "chunk-123", - rank: 1, - final_score: 0.0328, - semantic_score: 0.95, - lexical_score: 8.5, - fusion_method: "rrf", - text: "kubectl port-forward service port:8080...", - source: "runbooks/kubernetes/networking.md", - score_breakdown: { - semantic_rank: 1, - lexical_rank: 1, - rrf_components: {...} - } - }, - ... - ], - metrics: { - total_time_ms: 245, - semantic_time_ms: 120, - lexical_time_ms: 118, - fusion_time_ms: 7, - semantic_results_count: 50, - lexical_results_count: 50, - final_results_count: 10 - } - } -``` - -### 3. RRF Algorithm (No Parameter Tuning) - -```rust -// Input: two ranked lists -semantic: [(doc1, 0.95), (doc2, 0.88), (doc3, 0.82)] -lexical: [(doc1, 8.5), (doc4, 7.2), (doc2, 6.8)] - -// RRF formula: 1 / (k + rank) where k=60 -doc1: 1/(60+1) + 1/(60+1) = 0.0328 ← Top result -doc2: 1/(60+2) + 1/(60+3) = 0.0317 -doc4: 1/(60+2) = 0.0159 -doc3: 1/(60+3) = 0.0158 - -// Output: [doc1, doc2, doc4, doc3] (merged + ranked) -``` - -**Why RRF?** -- ✅ No parameter tuning (k=60 is academic standard) -- ✅ Robust to score distribution differences -- ✅ Works if embedding model changes -- ✅ Academic consensus for multi-engine fusion -- ❌ Loses score magnitudes (but transparency provided) - ---- - -## Implementation Phases - -### Phase 1: ✅ COMPLETE (This Session) - -**Deliverables:** -- ✅ QueryOptimizer (450 LOC, 15+ tests) -- ✅ HybridQueryWorker (380 LOC, stub with API fixes noted) -- ✅ RRF Fusion algorithm (no parameter tuning) -- ✅ Complete design documentation (2000+ LOC) -- ✅ Implementation notes + API corrections - -**Time: 4 hours of design + coding** - -### Phase 2: TODO (Week 2, 3-4 days) - -**Tasks:** -- [ ] Fix VectorStore API calls (15 min) -- [ ] Make OpenSearchClient::lexical_search public (5 min) -- [ ] Integrate HybridQueryWorker into /memory/query handler -- [ ] Add fallback strategy (hybrid → semantic → error) -- [ ] Update response format (include metrics + score breakdown) -- [ ] Write 10+ integration tests -- [ ] Measure latency (hybrid vs semantic vs cascading) - -### Phase 3: TODO (Week 3, 2-3 days) - -**Performance Optimization:** -- [ ] Benchmark all search strategies -- [ ] Optimize pgvector index (HNSW tuning) -- [ ] Optimize OpenSearch queries (field boosts) -- [ ] Add query result caching (1hr TTL) -- [ ] Profile parallel execution - -### Phase 4: TODO (Week 4, 2-3 days) - -**Testing & Validation:** -- [ ] Create test fixture dataset (50+ queries with ground truth) -- [ ] Measure NDCG@10, MRR, Precision@K -- [ ] A/B test: Hybrid vs Semantic-only -- [ ] A/B test: RRF vs Weighted Linear (0.6/0.4) -- [ ] Experiment with different question types -- [ ] Finalize configuration (env vars + defaults) - ---- - -## Files & Statistics - -### Code Files (830 LOC) - -``` -crates/mem-cli/src/ -├─ query_optimizer.rs (450 LOC, 15 tests) -│ ├─ QueryOptimizer (6-stage pipeline) -│ ├─ QueryContext (data structure) -│ ├─ QuestionType enum (6 types) -│ ├─ SearchStrategy enum (4 strategies) -│ ├─ RRFConfig (tuning parameters) -│ └─ RRFFusion (RRF algorithm) -│ -├─ hybrid_query_worker.rs (380 LOC, stub) -│ ├─ HybridQueryWorker (orchestrator) -│ ├─ retrieve_hybrid() (parallel) -│ ├─ retrieve_cascading() (2-stage) -│ ├─ fuse_results() (RRF) -│ └─ HybridQueryResponse (response type) -│ -└─ lib.rs - ├─ pub mod query_optimizer - └─ pub mod hybrid_query_worker -``` - -### Design Documents (2100+ LOC) - -``` -docs/ -├─ QUERY_OPTIMIZATION_ENGINE.md (500+ LOC) -│ ├─ Executive Summary -│ ├─ 6-Stage Pipeline Detailed -│ ├─ Question Classification -│ ├─ RRF Algorithm Explained -│ ├─ 4-Phase Implementation Plan -│ └─ Testing Checklist -│ -├─ HYBRID_SEARCH_DESIGN.md (760+ LOC) -│ ├─ 5-Stage Retrieval Pipeline -│ ├─ Index Optimization (pgvector + OpenSearch) -│ ├─ Accuracy Metrics -│ └─ Weight Tuning Strategy -│ -├─ API_REVIEW.md (400+ LOC) -│ ├─ 10 Endpoints Reviewed -│ ├─ Query vs Retrieval APIs -│ ├─ Current Gaps -│ └─ Phase 1-4 Enhancements -│ -├─ IMPLEMENTATION_NOTES.md (280+ LOC) -│ ├─ Compilation Status -│ ├─ API Corrections -│ └─ Phase 2 Checklist -│ -└─ memory-flow.md (updated, 833 LOC) - ├─ 5-Stage Hybrid Retrieval Pipeline - ├─ Query Routing Decision Tree - └─ Pod Infrastructure (8 core) -``` - ---- - -## Next Steps - -### Immediate (End of Session) - -✅ Review & approve design -✅ Commit code to repository -✅ Document in CLAUDE.md - -### Week 2 (Phase 2 Implementation) - -- [ ] Fix compilation errors (API mismatches) -- [ ] Integrate into /memory/query handler -- [ ] Add hybrid search tests -- [ ] Deploy to staging - -### Metrics to Track - -| Metric | Target | Notes | -|--------|--------|-------| -| Hybrid latency | 150-250ms | Parallel pgvector + OpenSearch | -| Cascading latency | 100-180ms | Lexical narrow → semantic rerank | -| NDCG@10 | ≥0.85 | Ranking quality | -| MRR | ≥0.8 | First correct result position | -| Precision@5 | ≥0.8 | Correct results in top-5 | -| Zero false negatives | 100% | Semantic catches synonyms | - ---- - -## Key Decisions - -✅ **Approach A: Parallel RRF** — Highest accuracy, fault tolerant -✅ **RRF over Weighted Linear** — No parameter tuning, robust -✅ **6-stage QueryOptimizer** — Understand query before retrieval -✅ **4 Search Strategies** — Hybrid/Semantic/Lexical/Cascading -✅ **JWT forwarding to OpenSearch** — Consistent auth -✅ **Fallback strategy** — Hybrid → Semantic → Error -✅ **Score breakdown in API** — Transparency + debugging - ---- - -## Success Criteria (Phase 1) - -✅ Design document complete and reviewed -✅ Code compiles (after API fixes) -✅ 15+ unit tests passing -✅ Architecture decisions documented -✅ Phase 2 implementation plan clear -✅ No architectural changes needed - -**All criteria met.** 🎉 - ---- - -## Summary - -We've designed and implemented a **production-grade Query Optimization Engine** for Poimen Memory: - -1. **QueryOptimizer** — 6-stage pipeline that understands queries -2. **HybridQueryWorker** — Parallel retrieval + RRF fusion -3. **4 Search Strategies** — Optimize for different query types -4. **Comprehensive Documentation** — 2100+ LOC covering architecture to testing - -**Approach:** Parallel RRF (Approach A) — highest accuracy for mission-critical agent reasoning. - -**Status:** Ready for Phase 2 implementation (3-4 day integration + testing). - diff --git a/QUICK-START.md b/QUICK-START.md deleted file mode 100644 index 68881c9..0000000 --- a/QUICK-START.md +++ /dev/null @@ -1,75 +0,0 @@ -# Quick Start — One-Time CI/CD Setup (3 minutes) - -## 🚀 Setup - -### 1. Add Secret to Repository -```bash -# Go to: https://git.riotpiao.com/rock/poimen-memory/settings/secrets -# Add: Name=REGISTRY_PAT, Value=bdf6a1d2317c28a332447083c61bb463d24defb7 -# Save - -# (This token is encrypted & managed in homelab via SOPS) -# See: CI-SETUP-WITH-KSOPS.md for details -``` - -### 3. Push to Trigger Build -```bash -cd ~/workplace/Poimen/memory -git commit --allow-empty -m "Trigger CI" -git push -``` - ---- - -## ✨ That's It! - -After these 3 steps, every push automatically: -- ✅ Runs all tests -- ✅ Builds Docker image -- ✅ Pushes to `forgejo.riotpiao.com/rock/poimen-memory:latest` -- ✅ ArgoCD deploys to K8s - ---- - -## 📊 Monitor - -```bash -# Watch build -https://git.riotpiao.com/rock/poimen-memory/actions - -# Watch deployment -kubectl get pods -n poimen -l app.kubernetes.io/name=poimen-memory -w -``` - ---- - -## 🧪 Test When Ready - -```bash -# Port forward -kubectl port-forward -n poimen svc/poimen-memory 8080:80 & - -# Health check -curl http://localhost:8080/health -``` - ---- - -## 🔄 Apply to Other Repos - -Same pattern for `poimen`, `poimen-workflows`, etc: - -1. Add `REGISTRY_PAT` secret -2. Copy `.forgejo/workflows/build.yaml` from this repo -3. Push - -See `CI-SETUP.md` for details. - ---- - -## Pattern Details - -- **Based on**: homelab-frontend (proven pattern) -- **Runner**: docker:27-cli (supports buildx) -- **Tags**: commit SHA + "latest" -- **No manual steps**: Fully automated diff --git a/crates/mem-cli/src/gateway_queue_adapter.rs b/crates/mem-cli/src/gateway_queue_adapter.rs index 5034a97..ee901bf 100644 --- a/crates/mem-cli/src/gateway_queue_adapter.rs +++ b/crates/mem-cli/src/gateway_queue_adapter.rs @@ -217,8 +217,8 @@ impl GatewayQueueAdapter { } } - fn queue_name(&self, project: &str) -> String { - format!("{}-{}", self.default_queue_prefix, project) + fn queue_name(&self, _project: &str) -> String { + self.default_queue_prefix.clone() } } @@ -505,7 +505,7 @@ mod tests { "test-token".to_string(), ); - assert_eq!(adapter.queue_name("myproject"), "poimen-chunks-myproject"); + assert_eq!(adapter.queue_name("myproject"), "poimen-chunks"); } #[test] diff --git a/k8s/infra/queue.yaml b/k8s/infra/queue.yaml new file mode 100644 index 0000000..31e3303 --- /dev/null +++ b/k8s/infra/queue.yaml @@ -0,0 +1,32 @@ +apiVersion: kmsvc.io/v1 +kind: Queue +metadata: + name: poimen-chunks + namespace: sqs +spec: + fifoQueue: false + visibilityTimeoutSeconds: 30 + messageRetentionPeriodSeconds: 86400 + maxReceiveCount: 3 + deadLetterTargetQueue: poimen-chunks-dlq + delaySeconds: 0 + partitionsPerShard: 2 + minShards: 1 + maxShards: 4 + shardSplitThresholdBytesPerSec: 1048576 + shardSplitCooldownSeconds: 300 +--- +apiVersion: kmsvc.io/v1 +kind: Queue +metadata: + name: poimen-chunks-dlq + namespace: sqs +spec: + fifoQueue: false + isDLQ: true + visibilityTimeoutSeconds: 30 + messageRetentionPeriodSeconds: 604800 + maxReceiveCount: 3 + partitionsPerShard: 1 + minShards: 1 + maxShards: 1