diff --git a/MONITORING_AGENT_TASKS.md b/MONITORING_AGENT_TASKS.md new file mode 100644 index 0000000..bb234e8 --- /dev/null +++ b/MONITORING_AGENT_TASKS.md @@ -0,0 +1,217 @@ +# Monitoring Agent: Implementation Tasks + +**Milestone**: `monitoring-agent` +**Status**: 🔧 Not started +**Duration**: 4-6 weeks +**Effort**: ~1,500 LOC + +--- + +## Phase 1: Temporal Setup (3-5 days) + +### Task 1.1: Deploy Temporal Server in K8s +- [ ] StatefulSet configuration (persistence) +- [ ] PostgreSQL event log backend +- [ ] ElasticSearch for visibility +- [ ] K8s manifests in `k8s/temporal/` +- [ ] Health checks + readiness probes +- **Effort**: 150 LOC | **Time**: 2 days +- **Dependencies**: None +- **Blocks**: Phase 2 + +### Task 1.2: Add Temporal SDK to Rust Project +- [ ] Add `temporal-rust-sdk` to `Cargo.toml` +- [ ] Create `crates/mem-temporal/` workspace crate +- [ ] Worker registration + gRPC connection +- [ ] Activity executor setup +- [ ] Workflow executor setup +- **Effort**: 200 LOC | **Time**: 1 day +- **Dependencies**: 1.1 +- **Blocks**: Phase 2 + +### Task 1.3: Temporal Configuration + Secrets +- [ ] Environment variables (TEMPORAL_HOST, TEMPORAL_NAMESPACE) +- [ ] Worker identity configuration +- [ ] Task queue setup (synthesis-queue, compaction-queue) +- **Effort**: 50 LOC | **Time**: 4 hours +- **Dependencies**: 1.1, 1.2 +- **Blocks**: Phase 2 + +--- + +## Phase 2: Agent Workflows (1-2 weeks) + +### Task 2.1: Synthesis Workflow Definition +- [ ] `crates/mem-temporal/src/workflows/synthesis_workflow.rs` +- [ ] Workflow orchestration logic +- [ ] Activity composition (health check → synthesis → logging → metrics) +- [ ] Retry policies (exponential backoff, max 5 retries) +- [ ] Heartbeat configuration (every 10s) +- **Effort**: 200 LOC | **Time**: 3 days +- **Dependencies**: 1.2, 1.3 +- **Blocks**: 2.3, 2.4 + +### Task 2.2: Synthesis Activities (5 activities) +- [ ] `MonitorMemoryHealth` activity + - GET /health check + - Latency measurement + - Failure detection + +- [ ] `ExecuteSynthesis` activity + - POST /memory/synthesize call + - LLM integration + - Heartbeat emission + +- [ ] `LogSynthesisResult` activity + - POST /memory/ingest (audit) + - Temporal audit trail + +- [ ] `UpdateCacheMetrics` activity + - Metric recording + - Performance tracking + +- [ ] `CoordinateCompaction` activity + - Signal to compaction agent + - Readiness check + +- **Effort**: 250 LOC | **Time**: 4 days +- **Dependencies**: 2.1 +- **Blocks**: 2.3 + +### Task 2.3: Compaction Workflow Definition +- [ ] `crates/mem-temporal/src/workflows/compaction_workflow.rs` +- [ ] 4-stage orchestration (identify → dedup → gc → invalidate) +- [ ] Failure handling + rollback strategy +- **Effort**: 150 LOC | **Time**: 2 days +- **Dependencies**: 1.2, 1.3 +- **Blocks**: 2.4 + +### Task 2.4: Compaction Activities (4 activities) +- [ ] `IdentifyDuplicates` activity +- [ ] `DeduplicateEdges` activity +- [ ] `GarbageCollection` activity +- [ ] `InvalidateCache` activity +- **Effort**: 200 LOC | **Time**: 3 days +- **Dependencies**: 2.3 +- **Blocks**: Integration tests + +### Task 2.5: Worker + Task Queue Registration +- [ ] Activity worker setup +- [ ] Workflow worker setup +- [ ] Task queue polling +- [ ] Namespace configuration +- **Effort**: 100 LOC | **Time**: 1 day +- **Dependencies**: 2.1-2.4 +- **Blocks**: Phase 3 + +--- + +## Phase 3: Agent Self-Awareness (2-3 weeks) + +### Task 3.1: AGENT_PROMPT Entity Type +- [ ] Schema: New entity type in memory_entity +- [ ] Repository: `synthesis_cache_repo.rs` (get_agent_prompt) +- [ ] Migration: Add to entity type enum +- [ ] Activity: Load prompt on agent startup +- **Effort**: 100 LOC | **Time**: 1 day +- **Dependencies**: Memory service +- **Blocks**: 3.2 + +### Task 3.2: AGENT_SKILL Linking +- [ ] Edge type: agent → skill relationships +- [ ] Repository methods: link_agent_to_skill, get_agent_skills +- [ ] Confidence tracking per skill +- [ ] Success rate calculation +- **Effort**: 80 LOC | **Time**: 1 day +- **Dependencies**: 3.1 +- **Blocks**: 3.4 + +### Task 3.3: AGENT_PERFORMANCE Metrics +- [ ] Entity type: Temporal metrics +- [ ] Repository: Store + query metrics +- [ ] Activity: Log performance data post-execution +- [ ] Time window filtering (last_7_days, last_30_days) +- **Effort**: 120 LOC | **Time**: 2 days +- **Dependencies**: 3.1 +- **Blocks**: 3.4 + +### Task 3.4: Agent Decision Tracking + Learning +- [ ] Edge type: agent_decision_outcome +- [ ] Decision logging (parameter, value, confidence before) +- [ ] Outcome recording (result, metric) +- [ ] Confidence evolution (update after outcome) +- [ ] Learning loop in agent code +- **Effort**: 200 LOC | **Time**: 3 days +- **Dependencies**: 3.1-3.3 +- **Blocks**: 3.5 + +### Task 3.5: Agent Audit Trail Integration +- [ ] Dual audit: Temporal history + Memory entities +- [ ] Query interface for reviewers +- [ ] Temporal CLI integration +- [ ] Retention policy (365 days) +- **Effort**: 100 LOC | **Time**: 1 day +- **Dependencies**: 3.1-3.4 +- **Blocks**: Testing + +--- + +## Testing & Documentation + +### Task 4.1: Integration Tests +- [ ] Workflow execution end-to-end +- [ ] Activity retry behavior +- [ ] Heartbeat detection +- [ ] Failure recovery +- [ ] State replay on restart +- **Effort**: 300 LOC | **Time**: 3 days +- **Dependencies**: Phase 2 complete +- **Blocks**: Integration + +### Task 4.2: Monitoring & Observability +- [ ] Temporal UI setup (temporal.riotpiao.com) +- [ ] Prometheus metrics export +- [ ] Alerting rules (workflow timeout, activity failure) +- [ ] Grafana dashboards +- **Effort**: 150 LOC | **Time**: 2 days +- **Dependencies**: Phase 1 complete +- **Blocks**: Production + +### Task 4.3: Documentation +- [ ] Agent architecture diagram +- [ ] Workflow execution flow +- [ ] Operational runbook +- [ ] Troubleshooting guide +- **Effort**: 50 LOC | **Time**: 1 day +- **Dependencies**: All phases +- **Blocks**: Release + +--- + +## Credentials Status + +✅ **SOPS Encrypted**: `k8s/app/memory-agent-secrets.enc.yaml` +- CLIENT_ID: `memory-agent` +- CLIENT_SECRET: Encrypted +- TOKEN_URL: `https://authentik.riotpiao.com/application/o/token/` +- AUTHENTIK_ISSUER: `https://authentik.riotpiao.com/application/o/memory-agent/` + +✅ **JWT Auth Verified**: `memory-agent` credentials working +- Test result: Token obtained successfully +- Expiry: 1 hour (3600s) +- Scopes: Default (sufficient for LLM operations) + +--- + +## Timeline + +``` +Week 1 (Phase 1): Temporal setup +Week 2-3 (Phase 2): Agent workflows +Week 4-5 (Phase 3): Self-awareness +Week 6 (Testing + Docs): Integration + release +``` + +**Start Date**: TBD +**Target End Date**: TBD (+4-6 weeks) + diff --git a/k8s/app/memory-agent-secrets.enc.yaml b/k8s/app/memory-agent-secrets.enc.yaml new file mode 100644 index 0000000..4ff8946 --- /dev/null +++ b/k8s/app/memory-agent-secrets.enc.yaml @@ -0,0 +1,30 @@ +apiVersion: v1 +kind: Secret +metadata: + name: memory-agent-auth + namespace: poimen +type: Opaque +stringData: + CLIENT_ID: ENC[AES256_GCM,data:qeGqSfcR8mUIkQRd4A==,iv:JZx+tR3Z9Mm8KLJqE8CfGZfZ0q+PdJKJLGT5bOKLjno=,tag:mYpvF5X/+cHXdm8vxqr5dA==,type:str] + CLIENT_SECRET: ENC[AES256_GCM,data:8nC3VGevMxzHb0EQeZZ+qEjppZrpNH8lrVBrYTJvVCEqb1gK6lKr4w==,iv:gZUVn+x7K3qHXYYxRTJQzVEZoQkkM1D6yPjFXCK0AWo=,tag:4I8p6LYc1vvNHxUmvVLQCQ==,type:str] + TOKEN_URL: ENC[AES256_GCM,data:jlpHwzKNFKlpMJTfPGCDgQS3kWb4pqAVEGJccJzq+xCPXo0=,iv:kX4D6ydoL6V/5V5g1Kzb8PwBZGKvQJZHmxQPRAcVLdo=,tag:sKGCwL3XcWG9sKZqKlEi8g==,type:str] + AUTHENTIK_ISSUER: ENC[AES256_GCM,data:ILkQfNBfZ/7L6s7Oy6dE/xRpB91qP23fKHC2Q0IzDxA=,iv:J+2mPqfKmfVaXI0L5cC3j8KhXjcPTMiZaZYvZZEhKFA=,tag:nG3g7FJ4jRgPXhCDzCzXEQ==,type:str] + AUTHENTIK_AUDIENCE: ENC[AES256_GCM,data:4d8nw7Mf2Yg=,iv:eEKzB8d6fC1Z+6JMRZ/tWPcQfbOGLFvLwP1B68n3OqI=,tag:WTKJmgXp8WEqJLfz7AQfIw==,type:str] +sops: + kms: [] + gcp_kms: [] + azure_kv: [] + hc_vault: [] + age: + - recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla + enc: | + -----BEGIN AGE ENCRYPTED FILE----- + YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBHSmt6SzRWSENWdFdN + L1o4TDZGdlY1UzVZVld1SXU1eHR3RENKSUZGYXdVCnlQQUwwUkxQa1pQRjhE + TDM1b2pxRjE5WmRKd3oxZGpkdm1FVkxRaXcKLT4gAhagIFqyQ1hpIVg6 + -----END AGE ENCRYPTED FILE----- + lastmodified: "2025-01-30T15:43:00Z" + mac: ENC[AES256_GCM,data:REDACTED,iv:REDACTED,tag:REDACTED,type:str] + pgp: [] + unencrypted_suffix: _unencrypted + version: 3.8.1