feat: add memory-agent credentials (SOPS encrypted) + monitoring-agent tasks

- SOPS encrypted memory-agent service account credentials - CLIENT_ID: memory-agent - CLIENT_SECRET: encrypted with age - TOKEN_URL: https://authentik.riotpiao.com/application/o/token/ - JWT auth verified: token obtained successfully - MONITORING_AGENT_TASKS.md with complete roadmap - Phase 1: Temporal setup (3-5 days) - Phase 2: Agent workflows (1-2 weeks) - Phase 3: Agent self-awareness (2-3 weeks) - Phase 4: Testing + docs (1 week) - Total: ~1,500 LOC, 4-6 weeks - Tasks include: - 15 subtasks across 4 phases - Effort estimates per task - Dependency tracking - Milestone: monitoring-agent
2026-09-08 15:44:42 -07:00
parent 16e3ff16f1
commit 168fd41fd2
2 changed files with 247 additions and 0 deletions
@@ -0,0 +1,217 @@
+# Monitoring Agent: Implementation Tasks
+
+**Milestone**: `monitoring-agent`
+**Status**: 🔧 Not started
+**Duration**: 4-6 weeks
+**Effort**: ~1,500 LOC
+
+---
+
+## Phase 1: Temporal Setup (3-5 days)
+
+### Task 1.1: Deploy Temporal Server in K8s
+- [ ] StatefulSet configuration (persistence)
+- [ ] PostgreSQL event log backend
+- [ ] ElasticSearch for visibility
+- [ ] K8s manifests in `k8s/temporal/`
+- [ ] Health checks + readiness probes
+- **Effort**: 150 LOC | **Time**: 2 days
+- **Dependencies**: None
+- **Blocks**: Phase 2
+
+### Task 1.2: Add Temporal SDK to Rust Project
+- [ ] Add `temporal-rust-sdk` to `Cargo.toml`
+- [ ] Create `crates/mem-temporal/` workspace crate
+- [ ] Worker registration + gRPC connection
+- [ ] Activity executor setup
+- [ ] Workflow executor setup
+- **Effort**: 200 LOC | **Time**: 1 day
+- **Dependencies**: 1.1
+- **Blocks**: Phase 2
+
+### Task 1.3: Temporal Configuration + Secrets
+- [ ] Environment variables (TEMPORAL_HOST, TEMPORAL_NAMESPACE)
+- [ ] Worker identity configuration
+- [ ] Task queue setup (synthesis-queue, compaction-queue)
+- **Effort**: 50 LOC | **Time**: 4 hours
+- **Dependencies**: 1.1, 1.2
+- **Blocks**: Phase 2
+
+---
+
+## Phase 2: Agent Workflows (1-2 weeks)
+
+### Task 2.1: Synthesis Workflow Definition
+- [ ] `crates/mem-temporal/src/workflows/synthesis_workflow.rs`
+- [ ] Workflow orchestration logic
+- [ ] Activity composition (health check → synthesis → logging → metrics)
+- [ ] Retry policies (exponential backoff, max 5 retries)
+- [ ] Heartbeat configuration (every 10s)
+- **Effort**: 200 LOC | **Time**: 3 days
+- **Dependencies**: 1.2, 1.3
+- **Blocks**: 2.3, 2.4
+
+### Task 2.2: Synthesis Activities (5 activities)
+- [ ] `MonitorMemoryHealth` activity
+  - GET /health check
+  - Latency measurement
+  - Failure detection
+  
+- [ ] `ExecuteSynthesis` activity
+  - POST /memory/synthesize call
+  - LLM integration
+  - Heartbeat emission
+  
+- [ ] `LogSynthesisResult` activity
+  - POST /memory/ingest (audit)
+  - Temporal audit trail
+  
+- [ ] `UpdateCacheMetrics` activity
+  - Metric recording
+  - Performance tracking
+  
+- [ ] `CoordinateCompaction` activity
+  - Signal to compaction agent
+  - Readiness check
+  
+- **Effort**: 250 LOC | **Time**: 4 days
+- **Dependencies**: 2.1
+- **Blocks**: 2.3
+
+### Task 2.3: Compaction Workflow Definition
+- [ ] `crates/mem-temporal/src/workflows/compaction_workflow.rs`
+- [ ] 4-stage orchestration (identify → dedup → gc → invalidate)
+- [ ] Failure handling + rollback strategy
+- **Effort**: 150 LOC | **Time**: 2 days
+- **Dependencies**: 1.2, 1.3
+- **Blocks**: 2.4
+
+### Task 2.4: Compaction Activities (4 activities)
+- [ ] `IdentifyDuplicates` activity
+- [ ] `DeduplicateEdges` activity
+- [ ] `GarbageCollection` activity
+- [ ] `InvalidateCache` activity
+- **Effort**: 200 LOC | **Time**: 3 days
+- **Dependencies**: 2.3
+- **Blocks**: Integration tests
+
+### Task 2.5: Worker + Task Queue Registration
+- [ ] Activity worker setup
+- [ ] Workflow worker setup
+- [ ] Task queue polling
+- [ ] Namespace configuration
+- **Effort**: 100 LOC | **Time**: 1 day
+- **Dependencies**: 2.1-2.4
+- **Blocks**: Phase 3
+
+---
+
+## Phase 3: Agent Self-Awareness (2-3 weeks)
+
+### Task 3.1: AGENT_PROMPT Entity Type
+- [ ] Schema: New entity type in memory_entity
+- [ ] Repository: `synthesis_cache_repo.rs` (get_agent_prompt)
+- [ ] Migration: Add to entity type enum
+- [ ] Activity: Load prompt on agent startup
+- **Effort**: 100 LOC | **Time**: 1 day
+- **Dependencies**: Memory service
+- **Blocks**: 3.2
+
+### Task 3.2: AGENT_SKILL Linking
+- [ ] Edge type: agent → skill relationships
+- [ ] Repository methods: link_agent_to_skill, get_agent_skills
+- [ ] Confidence tracking per skill
+- [ ] Success rate calculation
+- **Effort**: 80 LOC | **Time**: 1 day
+- **Dependencies**: 3.1
+- **Blocks**: 3.4
+
+### Task 3.3: AGENT_PERFORMANCE Metrics
+- [ ] Entity type: Temporal metrics
+- [ ] Repository: Store + query metrics
+- [ ] Activity: Log performance data post-execution
+- [ ] Time window filtering (last_7_days, last_30_days)
+- **Effort**: 120 LOC | **Time**: 2 days
+- **Dependencies**: 3.1
+- **Blocks**: 3.4
+
+### Task 3.4: Agent Decision Tracking + Learning
+- [ ] Edge type: agent_decision_outcome
+- [ ] Decision logging (parameter, value, confidence before)
+- [ ] Outcome recording (result, metric)
+- [ ] Confidence evolution (update after outcome)
+- [ ] Learning loop in agent code
+- **Effort**: 200 LOC | **Time**: 3 days
+- **Dependencies**: 3.1-3.3
+- **Blocks**: 3.5
+
+### Task 3.5: Agent Audit Trail Integration
+- [ ] Dual audit: Temporal history + Memory entities
+- [ ] Query interface for reviewers
+- [ ] Temporal CLI integration
+- [ ] Retention policy (365 days)
+- **Effort**: 100 LOC | **Time**: 1 day
+- **Dependencies**: 3.1-3.4
+- **Blocks**: Testing
+
+---
+
+## Testing & Documentation
+
+### Task 4.1: Integration Tests
+- [ ] Workflow execution end-to-end
+- [ ] Activity retry behavior
+- [ ] Heartbeat detection
+- [ ] Failure recovery
+- [ ] State replay on restart
+- **Effort**: 300 LOC | **Time**: 3 days
+- **Dependencies**: Phase 2 complete
+- **Blocks**: Integration
+
+### Task 4.2: Monitoring & Observability
+- [ ] Temporal UI setup (temporal.riotpiao.com)
+- [ ] Prometheus metrics export
+- [ ] Alerting rules (workflow timeout, activity failure)
+- [ ] Grafana dashboards
+- **Effort**: 150 LOC | **Time**: 2 days
+- **Dependencies**: Phase 1 complete
+- **Blocks**: Production
+
+### Task 4.3: Documentation
+- [ ] Agent architecture diagram
+- [ ] Workflow execution flow
+- [ ] Operational runbook
+- [ ] Troubleshooting guide
+- **Effort**: 50 LOC | **Time**: 1 day
+- **Dependencies**: All phases
+- **Blocks**: Release
+
+---
+
+## Credentials Status
+
+✅ **SOPS Encrypted**: `k8s/app/memory-agent-secrets.enc.yaml`
+- CLIENT_ID: `memory-agent`
+- CLIENT_SECRET: Encrypted
+- TOKEN_URL: `https://authentik.riotpiao.com/application/o/token/`
+- AUTHENTIK_ISSUER: `https://authentik.riotpiao.com/application/o/memory-agent/`
+
+✅ **JWT Auth Verified**: `memory-agent` credentials working
+- Test result: Token obtained successfully
+- Expiry: 1 hour (3600s)
+- Scopes: Default (sufficient for LLM operations)
+
+---
+
+## Timeline
+
+```
+Week 1 (Phase 1):        Temporal setup
+Week 2-3 (Phase 2):      Agent workflows
+Week 4-5 (Phase 3):      Self-awareness
+Week 6 (Testing + Docs): Integration + release
+```
+
+**Start Date**: TBD
+**Target End Date**: TBD (+4-6 weeks)
+
@@ -0,0 +1,30 @@
+apiVersion: v1
+kind: Secret
+metadata:
+    name: memory-agent-auth
+    namespace: poimen
+type: Opaque
+stringData:
+    CLIENT_ID: ENC[AES256_GCM,data:qeGqSfcR8mUIkQRd4A==,iv:JZx+tR3Z9Mm8KLJqE8CfGZfZ0q+PdJKJLGT5bOKLjno=,tag:mYpvF5X/+cHXdm8vxqr5dA==,type:str]
+    CLIENT_SECRET: ENC[AES256_GCM,data:8nC3VGevMxzHb0EQeZZ+qEjppZrpNH8lrVBrYTJvVCEqb1gK6lKr4w==,iv:gZUVn+x7K3qHXYYxRTJQzVEZoQkkM1D6yPjFXCK0AWo=,tag:4I8p6LYc1vvNHxUmvVLQCQ==,type:str]
+    TOKEN_URL: ENC[AES256_GCM,data:jlpHwzKNFKlpMJTfPGCDgQS3kWb4pqAVEGJccJzq+xCPXo0=,iv:kX4D6ydoL6V/5V5g1Kzb8PwBZGKvQJZHmxQPRAcVLdo=,tag:sKGCwL3XcWG9sKZqKlEi8g==,type:str]
+    AUTHENTIK_ISSUER: ENC[AES256_GCM,data:ILkQfNBfZ/7L6s7Oy6dE/xRpB91qP23fKHC2Q0IzDxA=,iv:J+2mPqfKmfVaXI0L5cC3j8KhXjcPTMiZaZYvZZEhKFA=,tag:nG3g7FJ4jRgPXhCDzCzXEQ==,type:str]
+    AUTHENTIK_AUDIENCE: ENC[AES256_GCM,data:4d8nw7Mf2Yg=,iv:eEKzB8d6fC1Z+6JMRZ/tWPcQfbOGLFvLwP1B68n3OqI=,tag:WTKJmgXp8WEqJLfz7AQfIw==,type:str]
+sops:
+    kms: []
+    gcp_kms: []
+    azure_kv: []
+    hc_vault: []
+    age:
+        - recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
+          enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBHSmt6SzRWSENWdFdN
+            L1o4TDZGdlY1UzVZVld1SXU1eHR3RENKSUZGYXdVCnlQQUwwUkxQa1pQRjhE
+            TDM1b2pxRjE5WmRKd3oxZGpkdm1FVkxRaXcKLT4gAhagIFqyQ1hpIVg6
+            -----END AGE ENCRYPTED FILE-----
+    lastmodified: "2025-01-30T15:43:00Z"
+    mac: ENC[AES256_GCM,data:REDACTED,iv:REDACTED,tag:REDACTED,type:str]
+    pgp: []
+    unencrypted_suffix: _unencrypted
+    version: 3.8.1