rock/poimen-workflows
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(architecture): add memory-driven architecture & tool usage planning

Browse Source 
Planning documents for memory service integration:

MEMORY_DRIVEN_ARCHITECTURE.md:
- Current state machine architecture (10 phases, 80 tasks)
- Memory service integration points & flow diagrams
- Activity usage per phase (T0-T10)
- Prompt optimization with memory context
- Retry policy enhancement via memory
- Complete flow diagrams & context hierarchy
- Skills and context consumption model

TOOL_USAGE_AND_SKILLS.md:
- Poimen tool landscape (6 categories)
- WorkflowDef builder, event log, executor patterns
- Verifier/judge/model provider integration
- Storage abstraction (EventLog + BlobStore)
- Skills ingestion strategy (4 phases)
- YAML skills registry example
- Tool-skill dependency matrix
- End-to-end execution scenario with memory

Both docs include:
- Flow diagrams
- Code examples
- Integration patterns
- Next steps for implementation
This commit is contained in:
Test
2026-08-29 21:52:13 -07:00
parent 5ef14ad5ec
commit f69295db6a
4 changed files with 1270 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TOOL_USAGE_AND_SKILLS.md
+585
View File
@@ -0,0 +1,585 @@
# Tool Usage & Skills Ingestion Strategy
## Poimen Tool Landscape
### Category 1: Workflow Definition Tools
**Tool**: `WorkflowDef Builder` (Rust)
```rust
let workflow = WorkflowDef::builder()
.name("poimen")
.phase(T0::phases())?
.step(StepId::from("T0.1-identity"))?
.transition_to(StepId::from("T0.2-kernel"))?
.build()?;
```
**Skill Usage**:
- Know when to use builder vs YAML
- Understand phase dependencies
- Handle schema version mismatches
**Memory Integration**:
```
IngestActivity {
level: "L2",
title: "WorkflowDef Builder Pattern",
content: "Use builder for Rust workflows. YAML for runtime customization.",
tags: ["T3-canonicalization", "IR"],
}
```
---
### Category 2: State Machine Tools
**Tool**: `Event Log` (immutable JSONL)
```
{"attempt_id": "1", "step": "T0.1", "event": "WorkerEvent::Started"}
{"attempt_id": "1", "step": "T0.1", "event": "WorkerEvent::Completed"}
{"attempt_id": "1", "step": "T0.2", "event": "WorkerEvent::Attempted"}
```
**Skills**:
- Event log format and ordering
- Atomic commit protocol for writes
- Fold + re-derive pattern
**Memory Integration**:
```
SearchActivity {
query: "event log corruption recovery",
returns: ["Verify checksum", "Replay from marker", "Fork + rewind"]
}
```
**Tool**: `Fold & Re-derive`
```rust
fn fold_state(state: &mut AttemptState, event: &WorkerEvent) {
match event {
WorkerEvent::Started => state.status = Running,
WorkerEvent::Completed => state.status = Success,
// ...
}
}
```
**Skills**:
- Deterministic state transitions
- No side effects in fold
- Time-ordered replay
**Memory Integration**:
```
DiagnoseIssueActivity {
issue: "state divergence after event log replay",
returns: [
"Tier 1: Check for non-deterministic fold",
"Tier 2: Verify event order",
"Tier 3: See fold/re-derive docs"
]
}
```
---
### Category 3: Execution Tools
**Tool**: `Run Executor` (polling)
```rust
loop {
let task = queue.wait_for_task(timeout)?;
let output = executor.execute_step(&task)?;
queue.mark_complete(&task, &output)?;
}
```
**Skills**:
- Long-poll timeouts
- Task queue semantics
- Backpressure handling
**Memory Integration**:
```
IngestActivity {
level: "L1",
title: "Executor Timeout Pattern",
content: "20s task queue poll, 30s step timeout, exponential backoff",
tags: ["executor", "T1-execution"],
}
```
**Tool**: `Attempt Lifecycle`
```rust
pub struct AttemptState {
number: u32, // 1st, 2nd, 3rd attempt
started_at: SystemTime,
budget: Budget, // tokens, attempts, time
context: PartitionedContext, // input for this attempt
retry_policy: RetryPolicy,
}
```
**Skills**:
- Budget exhaustion detection
- Retry condition evaluation
- Context capture per attempt
**Memory Integration**:
```
ContextActivity {
tool: "executor",
task: "attempt-lifecycle",
returns: {
tier_1: "Known budget limits per phase",
tier_2: "Learned attempt success rates",
tier_3: "Docs on RetryPolicy tuning",
}
}
```
---
### Category 4: Verification Tools
**Tool**: `Verifier Port` (pluggable)
```rust
pub trait Verifier {
fn verify(&self, output: &Output, rubric: &Rubric) -> Result<bool>;
}
```
**Skills**:
- Rubric definition (JSON/YAML)
- Verification logic chains
- Failure categorization
**Memory Integration**:
```
SearchActivity {
query: "rubric evaluation patterns",
returns: [
"Multi-level rubric structure",
"Failure classification system",
"Score aggregation methods"
]
}
```
**Tool**: `Judge Port` (decision logic)
```rust
pub trait Judge {
fn decide(&self, attempt: &AttemptState) -> Decision;
// → Approve | Reject | RequestRevision | Retry
}
```
**Skills**:
- Decision thresholds
- Evidence combination
- Feedback injection
**Memory Integration**:
```
DiagnoseIssueActivity {
issue: "judge consistently rejects step output",
returns: [
"Tier 1: Check rubric alignment",
"Tier 2: Review judge logic history",
"Tier 3: See judge tuning guide"
]
}
```
---
### Category 5: Model Provider Tools
**Tool**: `ModelProvider Port`
```rust
pub trait ModelProvider {
fn run(&self, model_id: &str, prompt: &str, budget: &Budget) -> Result<Output>;
}
```
**Skills**:
- Model selection (when to use which model)
- Prompt engineering
- Token budgeting
- Error handling per model
**Memory Integration - Prompt Optimization**:
```
GetContextActivity {
tool: "model-provider",
task: "planner-step-generation",
returns: {
tier_1: "Known failure patterns for this step",
tier_2: "Successful prompt patterns",
tier_3: "Model capability guide",
}
}
// Use returned context to optimize prompt:
optimized_prompt = inject_learned_lessons(
base_prompt,
context.lessons, // "Always include edge cases for T1.3"
context.skills, // "Skill: planning-with-constraints"
)
```
**Skill Example: Prompt Template**:
```yaml
title: "Planner Step with Constraint Handling"
level: "L2"
content: |
You are a step planner for workflow execution.
# Constraints (learned):
- Never generate steps without verification steps
- Include retry limits in plan
- Budget awareness required
# Examples from memory (tier-2):
- Previous successful T1.3 outputs show pattern X
- Failed attempts shared pattern Y to avoid
# Instructions:
Generate plan with these considerations...
```
---
### Category 6: Storage Tools
**Tool**: `EventLog Port` (redb implementation)
```rust
pub trait EventLog {
fn append(&mut self, event: WorkerEvent) -> Result<u64>;
fn read(&self, range: Range<u64>) -> Result<Vec<WorkerEvent>>;
}
```
**Skills**:
- Event serialization format
- Atomic writes
- Recovery from incomplete commits
**Memory Integration**:
```
LearnFromExecutionActivity {
taskID: "T0.5-eventlog-persistence",
result: "Redb backend successfully persisted 10K events",
tags: ["storage", "T0", "persistence"]
}
```
**Tool**: `BlobStore Port` (prompt/output capture)
```rust
pub trait BlobStore {
fn write(&self, path: &str, data: &[u8]) -> Result<()>;
fn read(&self, path: &str) -> Result<Vec<u8>>;
}
```
**Skills**:
- Path conventions (/{attempt_id}/{step_id}/prompt.txt)
- Compression strategies
- Retention policies
**Memory Integration**:
```
DocumentDecisionActivity {
decisionType: "blob-retention",
decision: "Archive attempts > 30 days to cold storage",
reasoning: "Balance audit trail with cost"
}
```
---
## Skills Ingestion Strategy
### Phase 1: YAML Skills Registry
**File**: `prompts/skills.yaml`
```yaml
skills:
- id: "kernel-state-machine"
category: "T0-kernel"
level: "L2"
title: "State Machine Kernel Patterns"
content: |
Key patterns for T0:
- Event log append-only design
- Atomic commit with 2PC
- Fold determinism for state derivation
- Fork/rewind for attempt recovery
- id: "attempt-lifecycle"
category: "T1-execution"
level: "L2"
title: "Attempt Lifecycle Management"
content: |
Execution loop patterns:
- Poll-based task queue
- Budget tracking (tokens, attempts, time)
- Retry policy evaluation
- Context capture per attempt
- id: "prompt-optimization"
category: "model-provider"
level: "L2"
title: "Memory-Based Prompt Optimization"
content: |
Best practices:
- Retrieve 3-tier context before execution
- Inject learned facts from tier-1 (exact matches)
- Include tier-2 patterns (ML-similar)
- Reference tier-3 docs (general guidance)
- Set budget constraints from experience
```
### Phase 2: Ingest Skills on Startup
```go
// In cmd/starter/main.go
func ingestSkills(memSvc *memory.Service) error {
skillsYAML, err := ioutil.ReadFile("prompts/skills.yaml")
if err != nil {
return err
}
var skillsConfig struct {
Skills []struct {
ID string `yaml:"id"`
Category string `yaml:"category"`
Level string `yaml:"level"`
Title string `yaml:"title"`
Content string `yaml:"content"`
} `yaml:"skills"`
}
if err := yaml.Unmarshal(skillsYAML, &skillsConfig); err != nil {
return err
}
for _, skill := range skillsConfig.Skills {
_, err := memSvc.CreateKnowledge(ctx, &memory.KnowledgeRecord{
Level: skill.Level,
Title: skill.Title,
Content: skill.Content,
Source: fmt.Sprintf("skills:///%s", skill.ID),
Metadata: map[string]interface{}{
"skill_id": skill.ID,
"category": skill.Category,
"type": "skill",
},
})
if err != nil {
log.Warn(fmt.Sprintf("Failed to ingest skill %s: %v", skill.ID, err))
continue
}
log.Info(fmt.Sprintf("Ingested skill: %s", skill.Title))
}
return nil
}
```
### Phase 3: Reference Docs Ingestion
**File**: `poimen/crates/doc/` (Rust doc comments)
```rust
/// # Attempt Lifecycle Pattern
///
/// Every step execution follows this sequence:
/// 1. Check budget (tokens, attempts, time remaining)
/// 2. Retrieve context from memory (3-tier)
/// 3. Optimize prompt with lessons & skills
/// 4. Execute with ModelProvider
/// 5. Evaluate with Verifier
/// 6. Decide with Judge
/// 7. Learn (success) or Diagnose (failure)
/// 8. Retry or proceed to next step
///
/// # Budget Tracking
/// - Tokens: Count LLM input/output tokens
/// - Attempts: Number of retries allowed
/// - Time: Wall-clock timeout per step
///
/// # Retry Policy
/// - Exponential backoff: 1s → 2s → 4s
/// - Max attempts: 3 (configurable)
/// - Non-retryable: Syntax errors, auth failures
pub struct AttemptState { ... }
```
**Ingest Docs**:
```go
// Extract doc comments and ingest as L2 knowledge
// Run during build/startup:
// $ cargo doc --extract-comments | memory-ingest --level L2
```
### Phase 4: Execution Pattern Capture
```go
// In RunExecutor::execute_step()
func (e *Executor) execute_step(ctx *WorkflowContext, step *StepId) error {
// ... execution logic ...
// Capture pattern on success
if output.status == Success {
memSvc.CreateKnowledge(ctx, &memory.KnowledgeRecord{
Level: "L1",
Title: fmt.Sprintf("Successful %s execution", step),
Content: fmt.Sprintf(
"Step %s completed with output:\n%s",
step, output.text,
),
Source: fmt.Sprintf("workflow://execution/%s", step),
Metadata: map[string]interface{}{
"step_id": step.String(),
"phase": ctx.PhaseId,
"attempt": ctx.AttemptState.Number,
"tokens_used": output.tokens,
},
})
}
}
```
---
## Tool-Skill Mapping Matrix
```
┌────────────────────────────────────────────────────────────────┐
│ Tool → Skill Dependencies │
├──────────────────────┬──────────────────────────────────────────┤
│ Tool │ Skills Needed (from memory) │
├──────────────────────┼──────────────────────────────────────────┤
│ WorkflowDef Builder │ • Phase dependencies │
│ │ • IR canonicalization rules │
│ │ • Schema versioning │
├──────────────────────┼──────────────────────────────────────────┤
│ Event Log │ • Event ordering guarantees │
│ │ • Atomic commit protocol │
│ │ • Checksum validation │
├──────────────────────┼──────────────────────────────────────────┤
│ Run Executor │ • Attempt lifecycle patterns │
│ │ • Budget exhaustion detection │
│ │ • Retry policy evaluation │
├──────────────────────┼──────────────────────────────────────────┤
│ Verifier Port │ • Rubric structure design │
│ │ • Failure categorization │
│ │ • Score aggregation rules │
├──────────────────────┼──────────────────────────────────────────┤
│ Judge Port │ • Decision thresholds │
│ │ • Evidence combination logic │
│ │ • Feedback injection patterns │
├──────────────────────┼──────────────────────────────────────────┤
│ ModelProvider │ • Prompt engineering best practices │
│ │ • Token budget awareness │
│ │ • Model-specific quirks │
├──────────────────────┼──────────────────────────────────────────┤
│ EventLog Storage │ • Serialization format choices │
│ │ • Compression strategies │
│ │ • Recovery procedures │
├──────────────────────┼──────────────────────────────────────────┤
│ BlobStore │ • Path naming conventions │
│ │ • Retention policies │
│ │ • Archive triggers │
└──────────────────────┴──────────────────────────────────────────┘
```
---
## Basic Tool Usage Example
### Scenario: Planner Step Fails Repeatedly
**User Command**:
```bash
poimen plan my-workflow.yaml --phase T1 --retry-with-memory
```
**Tool Execution Chain**:
```
1. LOAD WORKFLOW
WorkflowDefBuilder.from_yaml("my-workflow.yaml")
→ Memory: Retrieve "IR-canonicalization" skills
→ Validate against stored L2 knowledge
2. INIT EXECUTOR
RunExecutor.new()
→ Memory: Get "attempt-lifecycle" context
→ Load retry policy from memory lessons
3. EXECUTE PLANNER STEP
for attempt in 1..max_attempts:
a) GetContextActivity
- Tool: "planner"
- Task: "step-generation"
- Returns: lessons + skills
b) OptimizePrompt
- Inject learned facts (tier-1)
- Add pattern examples (tier-2)
- Set budget from history
c) ModelProvider.run(optimized_prompt)
- Send to planner agent
- Wait for output
d) Verifier.verify(output)
- Check against rubric
- Score output quality
e) Judge.decide(output)
- Approve | Retry | Reject
f) On Success: LearnFromExecutionActivity
- Store successful output pattern (L1)
g) On Failure: AnalyzeErrorActivity
- Search for similar failures
- Return recovery suggestions
h) DocumentDecisionActivity
- Log decision and reasoning
4. COMPLETED
✅ Plan generated (or user feedback required)
→ Memory: Ingest execution pattern
→ Next phase starts
```
---
## Summary: Tool & Skill Flow
```
Workflow Execution
↓
Tools Used ────────────────→ Skills Retrieved from Memory
├─ WorkflowDefBuilder ├─ IR canonicalization rules
├─ EventLog ├─ State machine patterns
├─ RunExecutor ├─ Attempt lifecycle
├─ Verifier Port ├─ Rubric design
├─ Judge Port ├─ Decision logic
├─ ModelProvider ├─ Prompt optimization
└─ Storage Ports └─ Retention policies
Skills Guide Execution ──────→ Results Learned
├─ Success patterns (L1)
├─ Failure recovery (L1)
├─ Verified practices (L2)
└─ Vault enriched for next run
```
This creates a **virtuous cycle**: Each execution improves the memory, which improves the next execution.