feat: add memory-agent credentials (SOPS encrypted) + monitoring-agent tasks
CI / CI (pull_request) Successful in 3m47s
CI / CI (pull_request) Successful in 3m47s
- SOPS encrypted memory-agent service account credentials - CLIENT_ID: memory-agent - CLIENT_SECRET: encrypted with age - TOKEN_URL: https://authentik.riotpiao.com/application/o/token/ - JWT auth verified: token obtained successfully - MONITORING_AGENT_TASKS.md with complete roadmap - Phase 1: Temporal setup (3-5 days) - Phase 2: Agent workflows (1-2 weeks) - Phase 3: Agent self-awareness (2-3 weeks) - Phase 4: Testing + docs (1 week) - Total: ~1,500 LOC, 4-6 weeks - Tasks include: - 15 subtasks across 4 phases - Effort estimates per task - Dependency tracking - Milestone: monitoring-agent
This commit is contained in:
@@ -0,0 +1,217 @@
|
|||||||
|
# Monitoring Agent: Implementation Tasks
|
||||||
|
|
||||||
|
**Milestone**: `monitoring-agent`
|
||||||
|
**Status**: 🔧 Not started
|
||||||
|
**Duration**: 4-6 weeks
|
||||||
|
**Effort**: ~1,500 LOC
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1: Temporal Setup (3-5 days)
|
||||||
|
|
||||||
|
### Task 1.1: Deploy Temporal Server in K8s
|
||||||
|
- [ ] StatefulSet configuration (persistence)
|
||||||
|
- [ ] PostgreSQL event log backend
|
||||||
|
- [ ] ElasticSearch for visibility
|
||||||
|
- [ ] K8s manifests in `k8s/temporal/`
|
||||||
|
- [ ] Health checks + readiness probes
|
||||||
|
- **Effort**: 150 LOC | **Time**: 2 days
|
||||||
|
- **Dependencies**: None
|
||||||
|
- **Blocks**: Phase 2
|
||||||
|
|
||||||
|
### Task 1.2: Add Temporal SDK to Rust Project
|
||||||
|
- [ ] Add `temporal-rust-sdk` to `Cargo.toml`
|
||||||
|
- [ ] Create `crates/mem-temporal/` workspace crate
|
||||||
|
- [ ] Worker registration + gRPC connection
|
||||||
|
- [ ] Activity executor setup
|
||||||
|
- [ ] Workflow executor setup
|
||||||
|
- **Effort**: 200 LOC | **Time**: 1 day
|
||||||
|
- **Dependencies**: 1.1
|
||||||
|
- **Blocks**: Phase 2
|
||||||
|
|
||||||
|
### Task 1.3: Temporal Configuration + Secrets
|
||||||
|
- [ ] Environment variables (TEMPORAL_HOST, TEMPORAL_NAMESPACE)
|
||||||
|
- [ ] Worker identity configuration
|
||||||
|
- [ ] Task queue setup (synthesis-queue, compaction-queue)
|
||||||
|
- **Effort**: 50 LOC | **Time**: 4 hours
|
||||||
|
- **Dependencies**: 1.1, 1.2
|
||||||
|
- **Blocks**: Phase 2
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: Agent Workflows (1-2 weeks)
|
||||||
|
|
||||||
|
### Task 2.1: Synthesis Workflow Definition
|
||||||
|
- [ ] `crates/mem-temporal/src/workflows/synthesis_workflow.rs`
|
||||||
|
- [ ] Workflow orchestration logic
|
||||||
|
- [ ] Activity composition (health check → synthesis → logging → metrics)
|
||||||
|
- [ ] Retry policies (exponential backoff, max 5 retries)
|
||||||
|
- [ ] Heartbeat configuration (every 10s)
|
||||||
|
- **Effort**: 200 LOC | **Time**: 3 days
|
||||||
|
- **Dependencies**: 1.2, 1.3
|
||||||
|
- **Blocks**: 2.3, 2.4
|
||||||
|
|
||||||
|
### Task 2.2: Synthesis Activities (5 activities)
|
||||||
|
- [ ] `MonitorMemoryHealth` activity
|
||||||
|
- GET /health check
|
||||||
|
- Latency measurement
|
||||||
|
- Failure detection
|
||||||
|
|
||||||
|
- [ ] `ExecuteSynthesis` activity
|
||||||
|
- POST /memory/synthesize call
|
||||||
|
- LLM integration
|
||||||
|
- Heartbeat emission
|
||||||
|
|
||||||
|
- [ ] `LogSynthesisResult` activity
|
||||||
|
- POST /memory/ingest (audit)
|
||||||
|
- Temporal audit trail
|
||||||
|
|
||||||
|
- [ ] `UpdateCacheMetrics` activity
|
||||||
|
- Metric recording
|
||||||
|
- Performance tracking
|
||||||
|
|
||||||
|
- [ ] `CoordinateCompaction` activity
|
||||||
|
- Signal to compaction agent
|
||||||
|
- Readiness check
|
||||||
|
|
||||||
|
- **Effort**: 250 LOC | **Time**: 4 days
|
||||||
|
- **Dependencies**: 2.1
|
||||||
|
- **Blocks**: 2.3
|
||||||
|
|
||||||
|
### Task 2.3: Compaction Workflow Definition
|
||||||
|
- [ ] `crates/mem-temporal/src/workflows/compaction_workflow.rs`
|
||||||
|
- [ ] 4-stage orchestration (identify → dedup → gc → invalidate)
|
||||||
|
- [ ] Failure handling + rollback strategy
|
||||||
|
- **Effort**: 150 LOC | **Time**: 2 days
|
||||||
|
- **Dependencies**: 1.2, 1.3
|
||||||
|
- **Blocks**: 2.4
|
||||||
|
|
||||||
|
### Task 2.4: Compaction Activities (4 activities)
|
||||||
|
- [ ] `IdentifyDuplicates` activity
|
||||||
|
- [ ] `DeduplicateEdges` activity
|
||||||
|
- [ ] `GarbageCollection` activity
|
||||||
|
- [ ] `InvalidateCache` activity
|
||||||
|
- **Effort**: 200 LOC | **Time**: 3 days
|
||||||
|
- **Dependencies**: 2.3
|
||||||
|
- **Blocks**: Integration tests
|
||||||
|
|
||||||
|
### Task 2.5: Worker + Task Queue Registration
|
||||||
|
- [ ] Activity worker setup
|
||||||
|
- [ ] Workflow worker setup
|
||||||
|
- [ ] Task queue polling
|
||||||
|
- [ ] Namespace configuration
|
||||||
|
- **Effort**: 100 LOC | **Time**: 1 day
|
||||||
|
- **Dependencies**: 2.1-2.4
|
||||||
|
- **Blocks**: Phase 3
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: Agent Self-Awareness (2-3 weeks)
|
||||||
|
|
||||||
|
### Task 3.1: AGENT_PROMPT Entity Type
|
||||||
|
- [ ] Schema: New entity type in memory_entity
|
||||||
|
- [ ] Repository: `synthesis_cache_repo.rs` (get_agent_prompt)
|
||||||
|
- [ ] Migration: Add to entity type enum
|
||||||
|
- [ ] Activity: Load prompt on agent startup
|
||||||
|
- **Effort**: 100 LOC | **Time**: 1 day
|
||||||
|
- **Dependencies**: Memory service
|
||||||
|
- **Blocks**: 3.2
|
||||||
|
|
||||||
|
### Task 3.2: AGENT_SKILL Linking
|
||||||
|
- [ ] Edge type: agent → skill relationships
|
||||||
|
- [ ] Repository methods: link_agent_to_skill, get_agent_skills
|
||||||
|
- [ ] Confidence tracking per skill
|
||||||
|
- [ ] Success rate calculation
|
||||||
|
- **Effort**: 80 LOC | **Time**: 1 day
|
||||||
|
- **Dependencies**: 3.1
|
||||||
|
- **Blocks**: 3.4
|
||||||
|
|
||||||
|
### Task 3.3: AGENT_PERFORMANCE Metrics
|
||||||
|
- [ ] Entity type: Temporal metrics
|
||||||
|
- [ ] Repository: Store + query metrics
|
||||||
|
- [ ] Activity: Log performance data post-execution
|
||||||
|
- [ ] Time window filtering (last_7_days, last_30_days)
|
||||||
|
- **Effort**: 120 LOC | **Time**: 2 days
|
||||||
|
- **Dependencies**: 3.1
|
||||||
|
- **Blocks**: 3.4
|
||||||
|
|
||||||
|
### Task 3.4: Agent Decision Tracking + Learning
|
||||||
|
- [ ] Edge type: agent_decision_outcome
|
||||||
|
- [ ] Decision logging (parameter, value, confidence before)
|
||||||
|
- [ ] Outcome recording (result, metric)
|
||||||
|
- [ ] Confidence evolution (update after outcome)
|
||||||
|
- [ ] Learning loop in agent code
|
||||||
|
- **Effort**: 200 LOC | **Time**: 3 days
|
||||||
|
- **Dependencies**: 3.1-3.3
|
||||||
|
- **Blocks**: 3.5
|
||||||
|
|
||||||
|
### Task 3.5: Agent Audit Trail Integration
|
||||||
|
- [ ] Dual audit: Temporal history + Memory entities
|
||||||
|
- [ ] Query interface for reviewers
|
||||||
|
- [ ] Temporal CLI integration
|
||||||
|
- [ ] Retention policy (365 days)
|
||||||
|
- **Effort**: 100 LOC | **Time**: 1 day
|
||||||
|
- **Dependencies**: 3.1-3.4
|
||||||
|
- **Blocks**: Testing
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing & Documentation
|
||||||
|
|
||||||
|
### Task 4.1: Integration Tests
|
||||||
|
- [ ] Workflow execution end-to-end
|
||||||
|
- [ ] Activity retry behavior
|
||||||
|
- [ ] Heartbeat detection
|
||||||
|
- [ ] Failure recovery
|
||||||
|
- [ ] State replay on restart
|
||||||
|
- **Effort**: 300 LOC | **Time**: 3 days
|
||||||
|
- **Dependencies**: Phase 2 complete
|
||||||
|
- **Blocks**: Integration
|
||||||
|
|
||||||
|
### Task 4.2: Monitoring & Observability
|
||||||
|
- [ ] Temporal UI setup (temporal.riotpiao.com)
|
||||||
|
- [ ] Prometheus metrics export
|
||||||
|
- [ ] Alerting rules (workflow timeout, activity failure)
|
||||||
|
- [ ] Grafana dashboards
|
||||||
|
- **Effort**: 150 LOC | **Time**: 2 days
|
||||||
|
- **Dependencies**: Phase 1 complete
|
||||||
|
- **Blocks**: Production
|
||||||
|
|
||||||
|
### Task 4.3: Documentation
|
||||||
|
- [ ] Agent architecture diagram
|
||||||
|
- [ ] Workflow execution flow
|
||||||
|
- [ ] Operational runbook
|
||||||
|
- [ ] Troubleshooting guide
|
||||||
|
- **Effort**: 50 LOC | **Time**: 1 day
|
||||||
|
- **Dependencies**: All phases
|
||||||
|
- **Blocks**: Release
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Credentials Status
|
||||||
|
|
||||||
|
✅ **SOPS Encrypted**: `k8s/app/memory-agent-secrets.enc.yaml`
|
||||||
|
- CLIENT_ID: `memory-agent`
|
||||||
|
- CLIENT_SECRET: Encrypted
|
||||||
|
- TOKEN_URL: `https://authentik.riotpiao.com/application/o/token/`
|
||||||
|
- AUTHENTIK_ISSUER: `https://authentik.riotpiao.com/application/o/memory-agent/`
|
||||||
|
|
||||||
|
✅ **JWT Auth Verified**: `memory-agent` credentials working
|
||||||
|
- Test result: Token obtained successfully
|
||||||
|
- Expiry: 1 hour (3600s)
|
||||||
|
- Scopes: Default (sufficient for LLM operations)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Timeline
|
||||||
|
|
||||||
|
```
|
||||||
|
Week 1 (Phase 1): Temporal setup
|
||||||
|
Week 2-3 (Phase 2): Agent workflows
|
||||||
|
Week 4-5 (Phase 3): Self-awareness
|
||||||
|
Week 6 (Testing + Docs): Integration + release
|
||||||
|
```
|
||||||
|
|
||||||
|
**Start Date**: TBD
|
||||||
|
**Target End Date**: TBD (+4-6 weeks)
|
||||||
|
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Secret
|
||||||
|
metadata:
|
||||||
|
name: memory-agent-auth
|
||||||
|
namespace: poimen
|
||||||
|
type: Opaque
|
||||||
|
stringData:
|
||||||
|
CLIENT_ID: ENC[AES256_GCM,data:qeGqSfcR8mUIkQRd4A==,iv:JZx+tR3Z9Mm8KLJqE8CfGZfZ0q+PdJKJLGT5bOKLjno=,tag:mYpvF5X/+cHXdm8vxqr5dA==,type:str]
|
||||||
|
CLIENT_SECRET: ENC[AES256_GCM,data:8nC3VGevMxzHb0EQeZZ+qEjppZrpNH8lrVBrYTJvVCEqb1gK6lKr4w==,iv:gZUVn+x7K3qHXYYxRTJQzVEZoQkkM1D6yPjFXCK0AWo=,tag:4I8p6LYc1vvNHxUmvVLQCQ==,type:str]
|
||||||
|
TOKEN_URL: ENC[AES256_GCM,data:jlpHwzKNFKlpMJTfPGCDgQS3kWb4pqAVEGJccJzq+xCPXo0=,iv:kX4D6ydoL6V/5V5g1Kzb8PwBZGKvQJZHmxQPRAcVLdo=,tag:sKGCwL3XcWG9sKZqKlEi8g==,type:str]
|
||||||
|
AUTHENTIK_ISSUER: ENC[AES256_GCM,data:ILkQfNBfZ/7L6s7Oy6dE/xRpB91qP23fKHC2Q0IzDxA=,iv:J+2mPqfKmfVaXI0L5cC3j8KhXjcPTMiZaZYvZZEhKFA=,tag:nG3g7FJ4jRgPXhCDzCzXEQ==,type:str]
|
||||||
|
AUTHENTIK_AUDIENCE: ENC[AES256_GCM,data:4d8nw7Mf2Yg=,iv:eEKzB8d6fC1Z+6JMRZ/tWPcQfbOGLFvLwP1B68n3OqI=,tag:WTKJmgXp8WEqJLfz7AQfIw==,type:str]
|
||||||
|
sops:
|
||||||
|
kms: []
|
||||||
|
gcp_kms: []
|
||||||
|
azure_kv: []
|
||||||
|
hc_vault: []
|
||||||
|
age:
|
||||||
|
- recipient: age1e5fq3hwxy78psus2nfvmtmua36g0u3suk78ephw6246l974d2utsvn0hla
|
||||||
|
enc: |
|
||||||
|
-----BEGIN AGE ENCRYPTED FILE-----
|
||||||
|
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBHSmt6SzRWSENWdFdN
|
||||||
|
L1o4TDZGdlY1UzVZVld1SXU1eHR3RENKSUZGYXdVCnlQQUwwUkxQa1pQRjhE
|
||||||
|
TDM1b2pxRjE5WmRKd3oxZGpkdm1FVkxRaXcKLT4gAhagIFqyQ1hpIVg6
|
||||||
|
-----END AGE ENCRYPTED FILE-----
|
||||||
|
lastmodified: "2025-01-30T15:43:00Z"
|
||||||
|
mac: ENC[AES256_GCM,data:REDACTED,iv:REDACTED,tag:REDACTED,type:str]
|
||||||
|
pgp: []
|
||||||
|
unencrypted_suffix: _unencrypted
|
||||||
|
version: 3.8.1
|
||||||
Reference in New Issue
Block a user