Test
f69295db6a
Planning documents for memory service integration: MEMORY_DRIVEN_ARCHITECTURE.md: - Current state machine architecture (10 phases, 80 tasks) - Memory service integration points & flow diagrams - Activity usage per phase (T0-T10) - Prompt optimization with memory context - Retry policy enhancement via memory - Complete flow diagrams & context hierarchy - Skills and context consumption model TOOL_USAGE_AND_SKILLS.md: - Poimen tool landscape (6 categories) - WorkflowDef builder, event log, executor patterns - Verifier/judge/model provider integration - Storage abstraction (EventLog + BlobStore) - Skills ingestion strategy (4 phases) - YAML skills registry example - Tool-skill dependency matrix - End-to-end execution scenario with memory Both docs include: - Flow diagrams - Code examples - Integration patterns - Next steps for implementation
684 lines
24 KiB
Markdown
684 lines
24 KiB
Markdown
# Memory-Driven Architecture for Poimen Workflows
|
|
|
|
## Executive Summary
|
|
|
|
Poimen state machine (10 phases, 80 tasks, 10 composition gates) will consume Memory Service context & skills to:
|
|
- **Learn** from execution attempts (L1 knowledge)
|
|
- **Diagnose** failures using memory (three-tier retrieval)
|
|
- **Document** decisions for future runs (L2 knowledge)
|
|
- **Optimize** prompts with relevant context before agent execution
|
|
- **Track** tools, skills, and pattern usage across the harness lifecycle
|
|
|
|
This document outlines how Temporal activities integrate with the existing state machine to create a memory-driven, self-improving workflow system.
|
|
|
|
---
|
|
|
|
## Current State Machine Architecture
|
|
|
|
```
|
|
Poimen Harness (Rust + JSON-RPC)
|
|
├─ Kernel (Event Log + State Machine)
|
|
├─ 10 Phases (T0-T10)
|
|
├─ 80 Tasks (70 build + 10 composition gates)
|
|
├─ WorkflowDef IR (YAML + Rust builder)
|
|
└─ 3 Ports (Verifier, Judge, ModelProvider)
|
|
```
|
|
|
|
### Key Components
|
|
|
|
**WorkflowDef (IR)**: Canonical hash of workflow definition
|
|
- YAML declares: steps, transitions, retry policy, budgets
|
|
- Rust implements: verifier logic, judge logic, model behavior
|
|
|
|
**State Machine**: Event-sourced, immutable audit trail
|
|
- Events: WorkerEvent enum
|
|
- Attempts: AttemptState with context partition capture
|
|
- Folds: re-derive state from event log
|
|
|
|
**Run Executor**: Poll-based with:
|
|
- Retry policy per step
|
|
- Budget tracking (attempts, tokens, time)
|
|
- Context partition per attempt
|
|
|
|
---
|
|
|
|
## Memory Service Integration Points
|
|
|
|
### Architecture Diagram
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Poimen Workflow │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ T1-T10: Task Execution Loop │ │
|
|
│ │ │ │
|
|
│ │ ┌─────────────────────────────────────────────────────┐ │ │
|
|
│ │ │ For each step in workflow: │ │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ 1. RetrieveContext (Memory Service) │ │ │
|
|
│ │ │ ├─ Tool: executor type (planner/judge/impl) │ │ │
|
|
│ │ │ ├─ Task: step name │ │ │
|
|
│ │ │ └─ Returns: tier-1 (signature) + tier-2 (ML) │ │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ 2. OptimizePrompt (with context) │ │ │
|
|
│ │ │ ├─ Add learned facts from memory │ │ │
|
|
│ │ │ ├─ Include skill usage examples │ │ │
|
|
│ │ │ └─ Attach budget constraints │ │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ 3. ExecuteStep (ModelProvider) │ │ │
|
|
│ │ │ └─ Agent uses optimized prompt │ │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ 4. OnStepComplete: │ │ │
|
|
│ │ │ ├─ Success? → LearnFromExecution │ │ │
|
|
│ │ │ ├─ Failure? → AnalyzeError │ │ │
|
|
│ │ │ └─ DocumentDecision (all paths) │ │ │
|
|
│ │ │ │ │ │
|
|
│ │ └─────────────────────────────────────────────────────┘ │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ ↕ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ Memory Service (PostgreSQL + OpenSearch + Vault) │ │
|
|
│ │ │ │
|
|
│ │ ├─ L1 Knowledge: Task execution results │ │
|
|
│ │ ├─ L2 Knowledge: Verified patterns & decisions │ │
|
|
│ │ ├─ R (Reference): Docs, skill examples, guides │ │
|
|
│ │ └─ Vault: Organized facts by tool/phase/domain │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### Memory Service Activities Flow
|
|
|
|
```
|
|
Workflow Step Execution → Memory Activities → Response
|
|
|
|
1. PRE-EXECUTION (Before step runs)
|
|
┌─────────────────────────────────┐
|
|
│ ExecuteGetContext Activity │
|
|
│ ├─ Input: tool, task, budget │
|
|
│ ├─ Retrieval: 3-tier │
|
|
│ │ ├─ Tier 1: Signature match │
|
|
│ │ │ (exact failure patterns) │
|
|
│ │ ├─ Tier 2: Vector search │
|
|
│ │ │ (learned from similar) │
|
|
│ │ └─ Tier 3: References │
|
|
│ │ (docs, skill guides) │
|
|
│ └─ Returns: Lessons + Skills │
|
|
└─────────────────────────────────┘
|
|
↓
|
|
┌─────────────────────────────────┐
|
|
│ Prompt Optimization │
|
|
│ ├─ Add context lessons │
|
|
│ ├─ Inject skill examples │
|
|
│ └─ Set budget constraints │
|
|
└─────────────────────────────────┘
|
|
|
|
2. EXECUTION
|
|
┌─────────────────────────────────┐
|
|
│ Agent executes with context │
|
|
│ (planner/judge/implementer) │
|
|
└─────────────────────────────────┘
|
|
|
|
3. POST-EXECUTION (After step completes)
|
|
┌─────────────────────────────────┐
|
|
│ if SUCCESS: │
|
|
│ ExecuteLearnFromExecution │
|
|
│ ├─ taskID: step name │
|
|
│ ├─ result: output │
|
|
│ ├─ tags: [tool, phase] │
|
|
│ └─ Returns: knowledgeID │
|
|
├──────────────────────────────────┤
|
|
│ if FAILURE: │
|
|
│ ExecuteAnalyzeError │
|
|
│ ├─ errorMsg: failure message │
|
|
│ ├─ Returns: recovery steps │
|
|
│ └─ (helps with retry) │
|
|
├──────────────────────────────────┤
|
|
│ ALWAYS: │
|
|
│ ExecuteDocumentDecision │
|
|
│ ├─ Type: phase milestone │
|
|
│ ├─ Decision: action taken │
|
|
│ └─ Reasoning: why chosen │
|
|
└─────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Skills and Context in State Machine
|
|
|
|
### Skill Types
|
|
|
|
**Tool Skills** (Skill Category 1):
|
|
```
|
|
┌──────────────────────────────────────┐
|
|
│ Tool Skills (Executor capabilities) │
|
|
├──────────────────────────────────────┤
|
|
│ • planner-best-practices │ (T1.3: Plan generation)
|
|
│ • judge-evaluation-patterns │ (T1.5: Rubric application)
|
|
│ • implementer-code-patterns │ (T2.1: Code generation)
|
|
│ • verifier-logic-chains │ (T1.4: Verification)
|
|
└──────────────────────────────────────┘
|
|
```
|
|
|
|
**Domain Skills** (Skill Category 2):
|
|
```
|
|
┌──────────────────────────────────────┐
|
|
│ Domain Skills (Phase-specific) │
|
|
├──────────────────────────────────────┤
|
|
│ • T0: State machine kernel │ Event log, fork, rewind
|
|
│ • T1: Workflow execution │ Attempt lifecycle, budgets
|
|
│ • T2: Error recovery │ Crash matrix, checkpoints
|
|
│ • T3: IR canonicalization │ YAML ↔ Rust equivalence
|
|
│ • T4-T10: Specialization │ Phase-specific patterns
|
|
└──────────────────────────────────────┘
|
|
```
|
|
|
|
**Pattern Skills** (Skill Category 3):
|
|
```
|
|
┌──────────────────────────────────────┐
|
|
│ Pattern Skills (Cross-cutting) │
|
|
├──────────────────────────────────────┤
|
|
│ • retry-strategy │ Exponential backoff
|
|
│ • budget-tracking │ Token/attempt/time limits
|
|
│ • composition-gates │ Phase completion criteria
|
|
│ • schema-evolution │ Backward compatibility
|
|
└──────────────────────────────────────┘
|
|
```
|
|
|
|
### Context Hierarchy
|
|
|
|
```
|
|
WorkflowContext (L0 - Always available)
|
|
├─ WorkflowDef (IR + hash)
|
|
├─ PhaseId (T0-T10)
|
|
├─ StepId (current step)
|
|
└─ AttemptState (attempt #, budget)
|
|
├─ Attempt context (attempt-scoped)
|
|
├─ Decision points (retry/abort)
|
|
└─ Cost ledger (tokens spent)
|
|
|
|
TaskContext (L1 - Learned from execution)
|
|
├─ Tool type (planner/judge/impl)
|
|
├─ Execution results (input/output)
|
|
├─ Failure patterns (error signatures)
|
|
└─ Retry outcomes (success rates)
|
|
|
|
ReferenceContext (L2 - From vault)
|
|
├─ Skill documentation
|
|
├─ Best practices (YAML-level)
|
|
├─ Code patterns (Rust-level)
|
|
└─ Design rationale
|
|
```
|
|
|
|
---
|
|
|
|
## Activity Usage Per Phase
|
|
|
|
### Phase 0-2 (Kernel & Execution Foundation)
|
|
|
|
```
|
|
T0: State Machine Kernel
|
|
├─ GetContextActivity
|
|
│ └─ Retrieve lessons on event log patterns
|
|
├─ LearnFromExecutionActivity
|
|
│ └─ Record fold/rewind operations
|
|
└─ DocumentDecisionActivity
|
|
└─ Track checkpointing decisions
|
|
|
|
T1: Attempt Lifecycle
|
|
├─ GetContextActivity
|
|
│ ├─ Tier 1: Known retry patterns
|
|
│ └─ Tier 2: Attempt budget tracking
|
|
├─ DiagnoseIssueActivity (on failure)
|
|
│ ├─ Search for "budget exhausted" patterns
|
|
│ └─ Find recovery step limits
|
|
└─ LearnFromExecutionActivity
|
|
└─ Record successful attempt patterns
|
|
|
|
T2: Error Recovery
|
|
├─ GetContextActivity
|
|
│ └─ Crash matrix lessons
|
|
├─ AnalyzeErrorActivity
|
|
│ ├─ Match against crash patterns
|
|
│ └─ Return recovery procedure
|
|
└─ DocumentDecisionActivity
|
|
└─ Log recovery action chosen
|
|
```
|
|
|
|
### Phase 3-5 (IR & Canonicalization)
|
|
|
|
```
|
|
T3: WorkflowDef IR
|
|
├─ GetContextActivity
|
|
│ └─ Tier 1: Canonical hash failures
|
|
├─ SearchKnowledgeActivity
|
|
│ └─ "YAML builder equivalence" patterns
|
|
└─ DocumentDecisionActivity
|
|
└─ IR versioning decisions
|
|
|
|
T4: Schema Evolution
|
|
├─ GetContextActivity
|
|
│ └─ Backward compatibility lessons
|
|
├─ DiagnoseIssueActivity
|
|
│ └─ Upcaster failure diagnosis
|
|
└─ LearnFromExecutionActivity
|
|
└─ Schema migration successes
|
|
|
|
T5: Storage Abstraction
|
|
├─ SearchKnowledgeActivity
|
|
│ └─ DB migration patterns
|
|
└─ DocumentDecisionActivity
|
|
└─ Storage backend selection
|
|
```
|
|
|
|
### Phase 6-8 (Orchestration & APIs)
|
|
|
|
```
|
|
T6: Orchestrator
|
|
├─ GetContextActivity
|
|
│ ├─ Tool: orchestrator
|
|
│ ├─ Task: workflow step dispatch
|
|
│ └─ Returns: step ordering lessons
|
|
├─ SearchKnowledgeActivity
|
|
│ └─ Query ordering patterns
|
|
└─ LearnFromExecutionActivity
|
|
└─ Successful step sequences
|
|
|
|
T7: HTTP API
|
|
├─ DiagnoseIssueActivity (on API error)
|
|
│ └─ Match error codes to recovery
|
|
└─ DocumentDecisionActivity
|
|
└─ Rate limit/timeout decisions
|
|
|
|
T8: Observability
|
|
├─ SearchKnowledgeActivity
|
|
│ └─ Logging pattern queries
|
|
└─ RefreshMemoryActivity
|
|
└─ Periodic metric snapshots
|
|
```
|
|
|
|
### Phase 9-10 (Delivery & Completion)
|
|
|
|
```
|
|
T9: Deployment
|
|
├─ GetContextActivity
|
|
│ ├─ Tool: deployment executor
|
|
│ └─ Task: artifact rollout
|
|
├─ DiagnoseIssueActivity (on deployment failure)
|
|
│ └─ Canary issues, rollback strategies
|
|
└─ AnalyzeErrorActivity
|
|
└─ Find deployment-specific solutions
|
|
|
|
T10: CLI & Metrics
|
|
├─ SearchKnowledgeActivity
|
|
│ └─ Transcript formatting patterns
|
|
├─ LearnFromExecutionActivity
|
|
│ └─ User interaction patterns
|
|
└─ DocumentDecisionActivity
|
|
└─ Metric collection decisions
|
|
```
|
|
|
|
---
|
|
|
|
## Prompt Optimization with Memory Context
|
|
|
|
### Before (Current)
|
|
|
|
```go
|
|
prompt := fmt.Sprintf(`
|
|
Execute step: %s
|
|
Workflow: %s
|
|
Budget: %d tokens
|
|
|
|
Task: %s
|
|
`)
|
|
```
|
|
|
|
### After (Memory-Optimized)
|
|
|
|
```go
|
|
// 1. Get context from memory
|
|
ctx, err := ExecuteGetContext(
|
|
wfCtx,
|
|
"planner", // tool type
|
|
"T1.3-run-executor", // task name
|
|
4096, // budget
|
|
)
|
|
if err != nil {
|
|
log.Warn("memory unavailable, continue without context")
|
|
ctx = nil
|
|
}
|
|
|
|
// 2. Build prompt with lessons
|
|
lessons := ""
|
|
if ctx != nil && len(ctx.Lessons) > 0 {
|
|
// Add tier-1 (signature matches)
|
|
for _, lesson := range ctx.Lessons {
|
|
if lesson.Tier == 1 {
|
|
lessons += fmt.Sprintf("Known pattern: %s\n", lesson.Text)
|
|
}
|
|
}
|
|
}
|
|
|
|
// 3. Inject skills
|
|
skills := ""
|
|
if ctx != nil && len(ctx.Skills) > 0 {
|
|
for _, skill := range ctx.Skills {
|
|
skills += fmt.Sprintf("Skill %s: %s\n", skill.Name, skill.Why)
|
|
}
|
|
}
|
|
|
|
// 4. Build optimized prompt
|
|
prompt := fmt.Sprintf(`
|
|
Execute step: %s
|
|
Workflow: %s
|
|
Budget: %d tokens
|
|
|
|
# Learned Patterns
|
|
%s
|
|
|
|
# Skills to Apply
|
|
%s
|
|
|
|
# Instructions
|
|
%s
|
|
`, stepName, workflowId, budget, lessons, skills, instructions)
|
|
|
|
// 5. Send to agent with enriched context
|
|
response := agent.Execute(prompt)
|
|
|
|
// 6. Learn from result
|
|
ExecuteLearnFromExecution(
|
|
wfCtx,
|
|
stepName,
|
|
response.Text,
|
|
[]string{"phase", "tool", "status"},
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
## Tool Usage Summary
|
|
|
|
### Basic Tools
|
|
|
|
**Core Workflow Tools**:
|
|
- `State Machine Events`: Insert events, compute state
|
|
- `WorkflowDef Builder`: Create IR programmatically
|
|
- `Run Executor`: Poll and execute steps
|
|
- `Attempt Lifecycle`: Retry, checkpoint, rewind
|
|
|
|
**Testing Tools**:
|
|
- `Harness`: Verification framework
|
|
- `Integration Tests`: Phase composition gates
|
|
- `Verify Script`: Assertion + diff runner
|
|
|
|
### Memory-Integrated Tools
|
|
|
|
**New with Memory Service**:
|
|
- `ExecuteGetContext`: Retrieve 3-tier context
|
|
- `ExecuteLearnFromExecution`: Capture task results
|
|
- `ExecuteAnalyzeError`: Diagnosis on failure
|
|
- `ExecuteDocumentDecision`: Log milestones
|
|
- `ExecuteSearchKnowledge`: Find patterns
|
|
- `ExecuteHealthCheck`: Verify service readiness
|
|
|
|
**Memory Vault Organization**:
|
|
```
|
|
vault/
|
|
├─ tools/
|
|
│ ├─ planner/
|
|
│ │ └─ best-practices.md
|
|
│ ├─ judge/
|
|
│ │ └─ rubric-patterns.md
|
|
│ └─ verifier/
|
|
│ └─ logic-chains.md
|
|
├─ phases/
|
|
│ ├─ T0-kernel/
|
|
│ ├─ T1-execution/
|
|
│ └─ T2-recovery/
|
|
├─ patterns/
|
|
│ ├─ retry-strategies.md
|
|
│ ├─ budget-tracking.md
|
|
│ └─ error-signatures.md
|
|
└─ skills/
|
|
├─ schema-evolution.md
|
|
├─ composition-gates.md
|
|
└─ ir-canonicalization.md
|
|
```
|
|
|
|
---
|
|
|
|
## State Machine Consumption Model
|
|
|
|
### Step Execution with Memory
|
|
|
|
```rust
|
|
// In RunExecutor::execute_step()
|
|
|
|
fn execute_step(
|
|
&self,
|
|
workflow: &WorkflowDef,
|
|
step: &StepId,
|
|
attempt: &AttemptState,
|
|
) -> Result<StepOutput> {
|
|
// 1. Pre-execution: Retrieve context
|
|
let context = self.memory_svc
|
|
.retrieve_context(
|
|
"tool_type", // planner/judge/implementer
|
|
format!("{:?}", step), // step name
|
|
attempt.budget.remaining_tokens,
|
|
)
|
|
.await
|
|
.ok(); // Fail gracefully if memory unavailable
|
|
|
|
// 2. Optimize prompt with memory lessons
|
|
let prompt = self.optimize_prompt(
|
|
&workflow.def,
|
|
step,
|
|
context.as_ref(), // Lessons + skills
|
|
);
|
|
|
|
// 3. Execute step with agent
|
|
let output = self.model_provider.run(
|
|
&self.model_id,
|
|
&prompt,
|
|
&attempt.budget,
|
|
).await?;
|
|
|
|
// 4. Post-execution: Learn or diagnose
|
|
if output.status == StepStatus::Success {
|
|
self.memory_svc
|
|
.learn_from_execution(
|
|
format!("{:?}", step),
|
|
output.text.clone(),
|
|
vec!["tool", "phase"],
|
|
)
|
|
.await
|
|
.ok(); // Non-blocking
|
|
} else {
|
|
self.memory_svc
|
|
.analyze_error(
|
|
&output.error_message,
|
|
)
|
|
.await
|
|
.ok(); // Returns recovery suggestions
|
|
}
|
|
|
|
// 5. Document decision
|
|
self.memory_svc
|
|
.document_decision(
|
|
"step_completion",
|
|
output.text.clone(),
|
|
format!("Attempt {}", attempt.number),
|
|
)
|
|
.await
|
|
.ok();
|
|
|
|
Ok(output)
|
|
}
|
|
```
|
|
|
|
### Retry Policy Integration
|
|
|
|
```rust
|
|
// In AttemptState::should_retry()
|
|
|
|
fn should_retry(&self, error: &Error) -> bool {
|
|
// 1. Check budget first
|
|
if self.budget.attempts_remaining == 0 {
|
|
return false;
|
|
}
|
|
|
|
// 2. Consult memory for pattern
|
|
let recovery = self.memory_svc
|
|
.analyze_error(&error.message)
|
|
.await
|
|
.ok();
|
|
|
|
// 3. If memory suggests retry strategy, use it
|
|
if let Some(recovery_steps) = recovery {
|
|
for step in recovery_steps {
|
|
if step.level == "L1" { // High confidence
|
|
return step.suggests_retry();
|
|
}
|
|
}
|
|
}
|
|
|
|
// 4. Fall back to default policy
|
|
self.retry_policy.should_retry(self.number, error)
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Flow Diagram: Memory-Driven Lifecycle
|
|
|
|
```
|
|
Workflow Initiated
|
|
│
|
|
↓
|
|
┌───────────────┐
|
|
│ Phase T0-T10 │
|
|
└───────┬───────┘
|
|
│
|
|
┌─────────────┼─────────────┐
|
|
↓ ↓ ↓
|
|
┌─────────────┐ ┌──────────┐ ┌─────────┐
|
|
│ Get Context │ │ Execute │ │ Analyze │
|
|
│ (Pre-exec) │ │ Step │ │ Result │
|
|
└──────┬──────┘ └────┬─────┘ └────┬────┘
|
|
│ │ │
|
|
├─────────────→ │ (optimize) │
|
|
│ │ │
|
|
│ ┌──────────→│◄────────────┤
|
|
│ │ ↓ │
|
|
│ │ ┌─────────────┐ │
|
|
│ │ │ Memory Tier │ │
|
|
│ │ │ 1/2/3 │ │
|
|
│ │ └─────────────┘ │
|
|
│ │ │
|
|
└───┴────────────────────────┴────→ Learn/Document
|
|
│
|
|
↓
|
|
┌───────────────────┐
|
|
│ Continue or Retry?│
|
|
└─────┬─────────────┘
|
|
│
|
|
┌───────────┴────────────┐
|
|
↓ ↓
|
|
Next Step Attempt Retry
|
|
│ (with memory
|
|
│ guidance)
|
|
│ │
|
|
└──────────┬─────────────┘
|
|
↓
|
|
Phase Complete?
|
|
│ │
|
|
Yes ↓ No ↓
|
|
│ Return to
|
|
Composition Step Loop
|
|
Gate
|
|
│
|
|
↓
|
|
All Phases Done?
|
|
│
|
|
Yes ↓ No
|
|
│ └─→ Next Phase
|
|
Workflow
|
|
Complete ──→ DocumentDecision
|
|
(Final)
|
|
```
|
|
|
|
---
|
|
|
|
## Memory-Skills Matrix
|
|
|
|
### Which Activities for Which Tools
|
|
|
|
```
|
|
│ Planner │ Judge │ Impl │ Verifier │ Executor
|
|
─────────┼─────────┼───────┼──────┼──────────┼─────────
|
|
Create │ ✓ │ ✓ │ ✓ │ ✓ │ ✓
|
|
Update │ ✓ │ ✓ │ ✓ │ ✓ │ ✓
|
|
Search │ ✓ │ ✓ │ ✓ │ ✓ │ ✓
|
|
Context │ ✓ │ ✓ │ ✓ │ ✓ │ ✓
|
|
Learn │ ✓ │ ✓ │ ✓ │ ✓ │ ✓
|
|
Diagnose │ ✓ │ ✓ │ ✓ │ ✓ │ ✓
|
|
Document │ ✓ │ ✓ │ ✓ │ ✓ │ ✓
|
|
Vault │ ✓ │ ✓ │ ✓ │ ✓ │ ✓
|
|
Health │ ✓ │ ✓ │ ✓ │ ✓ │ ✓
|
|
Analyze │ ✓ │ ✓ │ ✓ │ ✓ │ ✓
|
|
```
|
|
|
|
### Context Availability by Phase
|
|
|
|
```
|
|
Phase │ L0 (Workflow) │ L1 (Task) │ L2 (Reference)
|
|
──────┼───────────────┼───────────┼────────────────
|
|
T0-1 │ High │ Growing │ Available
|
|
T2-3 │ High │ High │ High
|
|
T4-6 │ High │ High │ Very High
|
|
T7-10 │ High │ Very High│ Very High
|
|
```
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
### Phase 1: Integration (This Sprint)
|
|
- ✅ Memory activities implemented (12 activities)
|
|
- ✅ Temporal test suite passing (23/23 tests)
|
|
- 🔄 Wire activities into RunExecutor
|
|
- 🔄 Add memory pre/post-execution hooks
|
|
- 🔄 Ingest skill YAML → memory vault
|
|
|
|
### Phase 2: Optimization (Next Sprint)
|
|
- 🔄 Prompt optimization with context
|
|
- 🔄 Retry policy enhancement via memory
|
|
- 🔄 Budget tracking with learned limits
|
|
- 🔄 Phase composition gate improvements
|
|
|
|
### Phase 3: Observability (2 Sprints)
|
|
- 🔄 Memory usage metrics per phase
|
|
- 🔄 Context relevance scoring
|
|
- 🔄 Skill suggestion effectiveness tracking
|
|
- 🔄 Orchestrator dashboard with memory stats
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
Memory-driven architecture enables Poimen to:
|
|
|
|
1. **Learn** from every execution (Tier 1 knowledge)
|
|
2. **Improve** prompts with context (Tier 2/3 lessons)
|
|
3. **Recover** from failures faster (diagnose + suggest)
|
|
4. **Document** decisions for compliance (audit trail)
|
|
5. **Organize** skills and patterns (vault by domain)
|
|
6. **Scale** across phases (cross-phase pattern reuse)
|
|
|
|
The state machine becomes a **learning system**, not just an executor—every run improves future runs.
|