# Memory UI Flow - Complete Workflow ## Table of Contents 1. [Read Flow](#read-flow) 2. [Search Flow](#search-flow) 3. [Edit Flow (GRM Workflow)](#edit-flow-grm-workflow) 4. [Agent Context Flow](#agent-context-flow) 5. [System Architecture](#system-architecture) 6. [Pod Infrastructure](#pod-infrastructure) --- ## Read Flow Browse vault documents from the web UI. ``` ┌─────────────────────────────────────────────────────┐ │ User: memory.riotpiao.com │ │ (Browser, JWT token in localStorage) │ └────────────┬────────────────────────────────────────┘ │ │ GET /memory/vault?project=poimen │ Authorization: Bearer │ ↓ ┌─────────────────────────────────────────────────────┐ │ Memory Service Pod │ │ ├─ Load vault files from PVC │ │ ├─ Build file tree (directory structure) │ │ └─ Return JSON response │ └────────────┬────────────────────────────────────────┘ │ │ {files: [{path, title, updated_at}...]} │ ↓ ┌─────────────────────────────────────────────────────┐ │ UI: Render Vault Browser │ │ ├─ Project selector (dropdown) │ │ ├─ File tree (collapsible folders) │ │ ├─ Breadcrumb navigation │ │ └─ Preview panel (markdown rendering) │ └─────────────────────────────────────────────────────┘ ``` --- ## Search Flow Semantic search using embeddings and pgvector similarity. ``` ┌──────────────────────────────────────────────────────┐ │ User: Type query "deployment issues" │ │ Select filters (project=poimen, level=L1) │ └────────────┬─────────────────────────────────────────┘ │ │ POST /memory/query │ { │ "query": "deployment issues", │ "project": "poimen", │ "limit": 10 │ } │ ↓ ┌──────────────────────────────────────────────────────┐ │ Memory Service Pod │ │ ├─ Tokenize query │ │ ├─ Call LLM API for embedding │ │ └─ pgvector similarity search │ └────────────┬─────────────────────────────────────────┘ │ │ Query: similarity vector │ ↓ ┌──────────────────────────────────────────────────────┐ │ PostgreSQL Pod + pgvector │ │ ├─ Index: embedding <-> │ │ ├─ Query: │ │ │ SELECT chunks, similarity_score │ │ │ ORDER BY embedding <-> query_vec │ │ │ LIMIT 10 │ │ └─ Return ranked results │ └────────────┬─────────────────────────────────────────┘ │ │ [{chunk, score, source, level, breadcrumb}...] │ ↓ ┌──────────────────────────────────────────────────────┐ │ UI: Render Results │ │ ├─ Result cards with similarity badges (0.92) │ │ ├─ Syntax highlighted code blocks │ │ ├─ Link to source document │ │ └─ "Related" section (similarity graph) │ └──────────────────────────────────────────────────────┘ ``` --- ## Edit Flow (GRM Workflow) Full Git Review Merge workflow: create branch → MR → human approval → auto-sync vault. ``` ┌──────────────────────────────────────────────────────┐ │ User: Clicks "Edit" on document │ │ Example: runbook-deploy.md │ └────────────┬─────────────────────────────────────────┘ │ ↓ ┌──────────────────────────────────────────────────────┐ │ UI: Switch to Edit Mode │ │ ├─ Load document content from Memory Service │ │ ├─ Show markdown editor (CodeMirror) │ │ ├─ Disable Save button (drafts only) │ │ └─ Show "Submit for Review" button │ └────────────┬─────────────────────────────────────────┘ │ ├─ User makes edits (e.g., update deploy steps) │ ↓ ┌──────────────────────────────────────────────────────┐ │ User: Click "Submit for Review" │ └────────────┬─────────────────────────────────────────┘ │ │ ╔═══════╩═══════════════════════════════════════════╗ ║ STEP 1: CREATE BRANCH ║ ╚═══════╤═══════════════════════════════════════════╝ │ ↓ ┌──────────────────────────────────────────────────────┐ │ Frontend: POST /memory/grc/draft │ │ { │ │ "document_path": "vault/runbooks/deploy.md", │ │ "content": "", │ │ "message": "Update deploy steps", │ │ "user": "rock@riotpiao.com" │ │ } │ └────────────┬─────────────────────────────────────────┘ │ ↓ ┌──────────────────────────────────────────────────────┐ │ Memory Service Pod: GRC Handler │ │ ├─ Generate branch name: edit/rock/deploy- │ │ ├─ Call Forgejo API (create branch) │ │ ├─ Commit changes to branch │ │ └─ Return PR URL + branch name │ └────────────┬─────────────────────────────────────────┘ │ ↓ ┌──────────────────────────────────────────────────────┐ │ Forgejo Git Service │ │ ├─ Create branch: edit/rock/deploy- │ │ ├─ From: main │ │ ├─ Commit: "Update deploy steps" │ │ └─ Trigger CI checks (markdown lint) │ └────────────┬─────────────────────────────────────────┘ │ ↓ ╔═══════╩═══════════════════════════════════════════╗ ║ STEP 2: AUTO-CREATE MERGE REQUEST ║ ╚═══════╤═══════════════════════════════════════════╝ │ ↓ ┌──────────────────────────────────────────────────────┐ │ UI Feedback │ │ ✅ "Draft saved - Merge Request created" │ │ └─ Show clickable MR link │ └────────────┬─────────────────────────────────────────┘ │ │ ╔═══════╩═══════════════════════════════════════════╗ ║ STEP 3: HUMAN REVIEW (in Forgejo) ║ ╚═══════╤═══════════════════════════════════════════╝ │ ↓ ┌──────────────────────────────────────────────────────┐ │ Reviewer (e.g., lead engineer) │ │ ├─ Open MR in Forgejo web UI │ │ ├─ Review diff (before/after) │ │ ├─ Comment/suggest edits │ │ ├─ Approve or request changes │ │ └─ Click "Merge to main" │ └────────────┬─────────────────────────────────────────┘ │ ↓ ╔═══════╩═══════════════════════════════════════════╗ ║ STEP 4: AUTO-SYNC TO VAULT ║ ╚═══════╤═══════════════════════════════════════════╝ │ ↓ ┌──────────────────────────────────────────────────────┐ │ Forgejo: Merge Complete │ │ ├─ Branch merged to main │ │ ├─ Trigger webhook: pull_request_merged │ │ └─ Payload: {pr_id, merged_at, branch} │ └────────────┬─────────────────────────────────────────┘ │ │ Webhook trigger │ ↓ ┌──────────────────────────────────────────────────────┐ │ ArgoCD Application │ │ ├─ Webhook receiver │ │ ├─ Trigger sync of poimen-memory-app │ │ └─ Pull latest from git (main) │ └────────────┬─────────────────────────────────────────┘ │ ↓ ┌──────────────────────────────────────────────────────┐ │ Git-Sync Sidecar Pod (poimen namespace) │ │ ├─ Receive ArgoCD sync signal │ │ ├─ `git pull origin main` in vault/ │ │ ├─ File appears in PVC │ │ └─ Update complete │ └────────────┬─────────────────────────────────────────┘ │ ↓ ┌──────────────────────────────────────────────────────┐ │ Memory Service Pod: Indexing Job │ │ ├─ Detect vault file change │ │ ├─ Tokenize + embed new content │ │ ├─ Insert into pgvector index │ │ └─ Document now searchable │ └────────────┬─────────────────────────────────────────┘ │ ↓ ┌──────────────────────────────────────────────────────┐ │ UI Notification │ │ ✅ "Document published!" │ │ ├─ Document now visible to all │ │ ├─ Embeddings indexed │ │ └─ Available in search │ └──────────────────────────────────────────────────────┘ ``` --- ## Agent Context Flow Real-time agent execution with memory retrieval tracking. ``` ┌──────────────────────────────────────────────────────┐ │ User: Navigate to "Agent Workspace" tab │ │ (Shows live agent execution) │ └────────────┬─────────────────────────────────────────┘ │ │ Establish connection │ ↓ ┌──────────────────────────────────────────────────────┐ │ Frontend: WebSocket /memory/agents/stream │ │ (Fallback: HTTP polling) │ └────────────┬─────────────────────────────────────────┘ │ ↓ ┌──────────────────────────────────────────────────────┐ │ Memory Service Pod │ │ ├─ Tail agent execution log │ │ ├─ Emit events: │ │ │ - agent_started │ │ │ - memory_retrieved {query, chunks, scores} │ │ │ - tool_invoked {tool_name, args} │ │ │ - tool_result {result} │ │ │ - agent_decision {reasoning} │ │ │ - agent_complete │ │ └─ Stream as JSON events │ └────────────┬─────────────────────────────────────────┘ │ ↓ ┌──────────────────────────────────────────────────────┐ │ UI: Real-time Dashboard │ │ ├─ Timeline of agent actions (bottom-up) │ │ ├─ Memory chunks used (with similarity scores) │ │ ├─ Tool calls + outputs (expandable) │ │ ├─ Decision tree (branching logic) │ │ └─ Knowledge graph overlay (related docs) │ └──────────────────────────────────────────────────────┘ ``` --- ## System Architecture Complete deployment topology with all components. ``` ┌────────────────────────────────────────────────────────────────────────┐ │ EXTERNAL: User → memory.riotpiao.com (DNS A record) │ └────────────┬───────────────────────────────────────────────────────────┘ │ │ HTTPS │ ↓ ┌────────────────────────────────────────────────────────────────────────┐ │ K8s Ingress Controller (nginx-ingress) │ │ ├─ TLS termination (memory.riotpiao.com) │ │ ├─ Route to frontend Service (port 80) │ │ └─ Route to memory Service (port 8080) │ └────┬───────────────────────────────────┬───────────────────────────────┘ │ │ ↓ (frontend) ↓ (API) ┌──────────────────────────┐ ┌────────────────────────────────────┐ │ Frontend Service │ │ Memory Service (8080) │ │ (port 80) │ │ ├─ LoadBalancer type │ └────┬──────────────────────┘ └────┬───────────────────────────────┘ │ │ ↓ ↓ ┌──────────────────────────┐ ┌────────────────────────────────────┐ │ Frontend Pod (React SPA)│ │ Memory Pod 1 (poimen-memory-*) │ │ ├─ React app │ │ ├─ HTTP server (actix-web) │ │ ├─ Vite build │ │ ├─ JWT validation │ │ ├─ Static files │ │ ├─ GRC handler (Forgejo API) │ │ └─ API client │ │ ├─ Vault browser │ └──────────────────────────┘ │ ├─ Query (embedding) handler │ │ ├─ Skills handler │ ┌──────────────────────────┼─ Projects handler │ │ │ └─ Volume: /data/vault (PVC) │ │ │ │ │ └────┬───────────────────────────────┘ │ │ │ Memory Pod 2 (HA replica) │ │ (identical to Pod 1) │ │ │ ├───────────────────────────────┤ │ │ ↓ ↓ ┌─────────────────────────────────────────────────────────────┐ │ PostgreSQL StatefulSet (memory-db-0, memory-db-1) │ │ ├─ Primary: memory-db-0 (PVC: 20Gi) │ │ ├─ Replica: memory-db-1 (PVC: 20Gi) │ │ ├─ Service: memory-db (headless) │ │ ├─ Tables: │ │ │ ├─ chunks (id, text, project_id, embedding, source) │ │ │ ├─ skills (id, name, metadata) │ │ │ ├─ projects (id, name) │ │ │ └─ agent_logs (id, agent_id, action, timestamp) │ │ └─ Extension: pgvector (vector similarity) │ └─────────────────────────────────────────────────────────────┘ │ │ INDEX: embedding <-> vector[] │ └─ Used by: /memory/query (similarity search) ┌─────────────────────────────────────────────────────────────┐ │ Storage: PVC (poimen-memory-vault, 10Gi, Longhorn) │ │ ├─ Mount path: /data/vault │ │ ├─ Content: │ │ │ ├─ vault/skills/ │ │ │ ├─ vault/runbooks/ │ │ │ ├─ vault/evidence/ │ │ │ └─ .git/ (full git history) │ │ └─ Sync: git-sync sidecar (on file changes) │ └─────────────────────────────────────────────────────────────┘ │ │ ArgoCD monitors + syncs │ ↓ ┌─────────────────────────────────────────────────────────────┐ │ Git-Sync Sidecar (runs in Memory Pod) │ │ ├─ Watches: https://forgejo.riotpiao.com/.../memory.git │ │ ├─ Branch: main │ │ ├─ Sync interval: 30s │ │ ├─ On merge: pulls to /data/vault │ │ └─ Triggers indexing │ └─────────────────────────────────────────────────────────────┘ │ │ Webhooks │ ↓ ┌─────────────────────────────────────────────────────────────┐ │ External: Forgejo + ArgoCD │ │ ├─ Forgejo webhook: pr_merged → ArgoCD │ │ ├─ ArgoCD watches: poimen-memory-app (in git) │ │ ├─ Auto-sync enabled (prune + selfHeal) │ │ └─ Revision tracking │ └─────────────────────────────────────────────────────────────┘ ┌─────────────────────────────────────────────────────────────┐ │ External: Authentik (OIDC) │ │ ├─ Issuer: https://authentik.riotpiao.com/.../ │ │ ├─ JWKS: .../jwks/ │ │ ├─ OAuth2 App: poimen-memory │ │ └─ Used by: JWT validation in Memory Service │ └─────────────────────────────────────────────────────────────┘ ┌─────────────────────────────────────────────────────────────┐ │ External: LLM Service (Embeddings) │ │ ├─ Provider: Vertex AI / Hugging Face / etc │ │ ├─ Used by: /memory/query (tokenize + embed) │ │ └─ Cached results (1hr TTL) │ └─────────────────────────────────────────────────────────────┘ ``` --- ## Pod Infrastructure Complete pod inventory deployed in `poimen` namespace. ### Production Pods | Pod Name | Role | Replicas | PVC | Purpose | |----------|------|----------|-----|---------| | **poimen-memory-\*** | API Server | 2 | 10Gi vault | HTTP server, JWT auth, GRC, Query handler | | **memory-db-0** | PostgreSQL Primary | 1 | 20Gi | Main database, chunks + metadata | | **memory-db-1** | PostgreSQL Replica | 1 | 20Gi | High availability, read replicas | | **frontend-\*** | React SPA | 1+ | — | Web UI (memory.riotpiao.com) | ### Supporting Infrastructure (External) | Component | Role | Location | |-----------|------|----------| | **Git-Sync Sidecar** | Auto-pull vault | Embedded in memory pod | | **ArgoCD Application** | CD orchestration | argocd namespace | | **Ingress Controller** | Reverse proxy | ingress-nginx namespace | | **Longhorn** | Storage provider | Storage layer | ### Total Pod Count: **6 Production Pods** ``` Namespace: poimen ├─ poimen-memory (ReplicaSet) × 2 pods ...................... (2) ├─ memory-db-0 (StatefulSet) ............................... (1) ├─ memory-db-1 (StatefulSet) ............................... (1) └─ frontend (Deployment) × 1-2 pods ......................... (1-2) Namespace: argocd └─ argocd-server, argocd-repo-server, etc .................. (5+) Namespace: ingress-nginx └─ nginx-ingress-controller ................................ (1) TOTAL: 12-14 pods (6 core + 6-8 supporting) ``` ### Pod Responsibilities #### Memory Service Pod (×2, HA) - **Listen**: 0.0.0.0:8080 - **Endpoints**: - `GET /memory/vault` — Read vault files - `POST /memory/query` — Semantic search (embeddings) - `GET /memory/skills` — List skills - `POST /memory/grc/draft` — Create branch + MR - `GET /memory/grc/status` — Check MR status - `GET /memory/agents/logs` — Stream agent events - **Auth**: JWT (Authentik) - **Connections**: - PostgreSQL (query + index) - Forgejo API (GRC) - LLM service (embeddings) - PVC (vault files) #### PostgreSQL Pod (×2, Primary + Replica) - **Listen**: 5432 - **Service**: memory-db (headless for StatefulSet) - **Storage**: 20Gi per pod (PVC) - **Replication**: Streaming replication (primary → replica) - **Extensions**: pgvector - **Data**: - chunks table (with vector index) - skills table - projects table - agent_logs table #### Frontend Pod (×1-2) - **Listen**: 80 - **Serve**: React SPA static files - **Endpoints**: - `/` — App shell - `/api/*` — Proxy to Memory Service (8080) - **Auth**: JWT (localStorage) - **Build**: Vite (production bundle) #### Git-Sync Sidecar (embedded in Memory Pod) - **Runs**: As a init container + background process - **Watch**: Forgejo main branch - **Sync interval**: 30 seconds - **Action on merge**: `git pull` → trigger re-index --- ## Traffic Flow Diagram ``` Internet (Users) │ │ HTTPS │ ↓ ┌─────────────────────────┐ │ Ingress Controller │ │ (nginx) │ └──┬────────────┬─────────┘ │ │ │ port 80 │ port 8080 │ │ ↓ ↓ ┌──────────┐ ┌──────────────────┐ │ Frontend │ │ Memory Service │ │ (React) │ │ (Rust + Actix) │ └────┬─────┘ └──┬───────────┬────┘ │ │ │ │ ┌──┘ └──┐ │ │ │ ↓ ↓ ↓ ┌───────────────────┐ ┌──────────────┐ │ PostgreSQL │ │ Vault PVC │ │ (pgvector index) │ │ (git files) │ └───────────────────┘ └──────────────┘ ``` --- ## Deployment Checklist - [x] Memory Service (2 pods) deployed in `poimen` namespace - [x] PostgreSQL StatefulSet (2 pods) deployed - [x] PVC: poimen-memory-vault (10Gi) attached - [x] Git-sync sidecar configured (auto-pull on merge) - [x] JWT auth enabled (Authentik integration) - [x] Ingress configured (memory.riotpiao.com) - [ ] Frontend pod deployment (TODO) - [ ] GRC endpoints tested (TODO) - [ ] Agent logging endpoints tested (TODO)