poimen-workflows/TOOL_USAGE_AND_SKILLS.md

Tool Usage & Skills Ingestion Strategy

Poimen Tool Landscape

Category 1: Workflow Definition Tools

Tool: WorkflowDef Builder (Rust)

let workflow = WorkflowDef::builder()
    .name("poimen")
    .phase(T0::phases())?
    .step(StepId::from("T0.1-identity"))?
    .transition_to(StepId::from("T0.2-kernel"))?
    .build()?;

Skill Usage:

• Know when to use builder vs YAML
• Understand phase dependencies
• Handle schema version mismatches

Memory Integration:

IngestActivity {
    level: "L2",
    title: "WorkflowDef Builder Pattern",
    content: "Use builder for Rust workflows. YAML for runtime customization.",
    tags: ["T3-canonicalization", "IR"],
}

---

Category 2: State Machine Tools

Tool: Event Log (immutable JSONL)

{"attempt_id": "1", "step": "T0.1", "event": "WorkerEvent::Started"}
{"attempt_id": "1", "step": "T0.1", "event": "WorkerEvent::Completed"}
{"attempt_id": "1", "step": "T0.2", "event": "WorkerEvent::Attempted"}

Skills:

• Event log format and ordering
• Atomic commit protocol for writes
• Fold + re-derive pattern

Memory Integration:

SearchActivity {
    query: "event log corruption recovery",
    returns: ["Verify checksum", "Replay from marker", "Fork + rewind"]
}

Tool: Fold & Re-derive

fn fold_state(state: &mut AttemptState, event: &WorkerEvent) {
    match event {
        WorkerEvent::Started => state.status = Running,
        WorkerEvent::Completed => state.status = Success,
        // ...
    }
}

Skills:

• Deterministic state transitions
• No side effects in fold
• Time-ordered replay

Memory Integration:

DiagnoseIssueActivity {
    issue: "state divergence after event log replay",
    returns: [
        "Tier 1: Check for non-deterministic fold",
        "Tier 2: Verify event order",
        "Tier 3: See fold/re-derive docs"
    ]
}

---

Category 3: Execution Tools

Tool: Run Executor (polling)

loop {
    let task = queue.wait_for_task(timeout)?;
    let output = executor.execute_step(&task)?;
    queue.mark_complete(&task, &output)?;
}

Skills:

• Long-poll timeouts
• Task queue semantics
• Backpressure handling

Memory Integration:

IngestActivity {
    level: "L1",
    title: "Executor Timeout Pattern",
    content: "20s task queue poll, 30s step timeout, exponential backoff",
    tags: ["executor", "T1-execution"],
}

Tool: Attempt Lifecycle

pub struct AttemptState {
    number: u32,                    // 1st, 2nd, 3rd attempt
    started_at: SystemTime,
    budget: Budget,                 // tokens, attempts, time
    context: PartitionedContext,    // input for this attempt
    retry_policy: RetryPolicy,
}

Skills:

• Budget exhaustion detection
• Retry condition evaluation
• Context capture per attempt

Memory Integration:

ContextActivity {
    tool: "executor",
    task: "attempt-lifecycle",
    returns: {
        tier_1: "Known budget limits per phase",
        tier_2: "Learned attempt success rates",
        tier_3: "Docs on RetryPolicy tuning",
    }
}

---

Category 4: Verification Tools

Tool: Verifier Port (pluggable)

pub trait Verifier {
    fn verify(&self, output: &Output, rubric: &Rubric) -> Result<bool>;
}

Skills:

• Rubric definition (JSON/YAML)
• Verification logic chains
• Failure categorization

Memory Integration:

SearchActivity {
    query: "rubric evaluation patterns",
    returns: [
        "Multi-level rubric structure",
        "Failure classification system",
        "Score aggregation methods"
    ]
}

Tool: Judge Port (decision logic)

pub trait Judge {
    fn decide(&self, attempt: &AttemptState) -> Decision;
    // → Approve | Reject | RequestRevision | Retry
}

Skills:

• Decision thresholds
• Evidence combination
• Feedback injection

Memory Integration:

DiagnoseIssueActivity {
    issue: "judge consistently rejects step output",
    returns: [
        "Tier 1: Check rubric alignment",
        "Tier 2: Review judge logic history",
        "Tier 3: See judge tuning guide"
    ]
}

---

Category 5: Model Provider Tools

Tool: ModelProvider Port

pub trait ModelProvider {
    fn run(&self, model_id: &str, prompt: &str, budget: &Budget) -> Result<Output>;
}

Skills:

• Model selection (when to use which model)
• Prompt engineering
• Token budgeting
• Error handling per model

Memory Integration - Prompt Optimization:

GetContextActivity {
    tool: "model-provider",
    task: "planner-step-generation",
    returns: {
        tier_1: "Known failure patterns for this step",
        tier_2: "Successful prompt patterns",
        tier_3: "Model capability guide",
    }
}
// Use returned context to optimize prompt:
optimized_prompt = inject_learned_lessons(
    base_prompt,
    context.lessons,  // "Always include edge cases for T1.3"
    context.skills,   // "Skill: planning-with-constraints"
)

Skill Example: Prompt Template:

title: "Planner Step with Constraint Handling"
level: "L2"
content: |
  You are a step planner for workflow execution.
  
  # Constraints (learned):
  - Never generate steps without verification steps
  - Include retry limits in plan
  - Budget awareness required
  
  # Examples from memory (tier-2):
  - Previous successful T1.3 outputs show pattern X
  - Failed attempts shared pattern Y to avoid
  
  # Instructions:
  Generate plan with these considerations...

---

Category 6: Storage Tools

Tool: EventLog Port (redb implementation)

pub trait EventLog {
    fn append(&mut self, event: WorkerEvent) -> Result<u64>;
    fn read(&self, range: Range<u64>) -> Result<Vec<WorkerEvent>>;
}

Skills:

• Event serialization format
• Atomic writes
• Recovery from incomplete commits

Memory Integration:

LearnFromExecutionActivity {
    taskID: "T0.5-eventlog-persistence",
    result: "Redb backend successfully persisted 10K events",
    tags: ["storage", "T0", "persistence"]
}

Tool: BlobStore Port (prompt/output capture)

pub trait BlobStore {
    fn write(&self, path: &str, data: &[u8]) -> Result<()>;
    fn read(&self, path: &str) -> Result<Vec<u8>>;
}

Skills:

• Path conventions (/{attempt_id}/{step_id}/prompt.txt)
• Compression strategies
• Retention policies

Memory Integration:

DocumentDecisionActivity {
    decisionType: "blob-retention",
    decision: "Archive attempts > 30 days to cold storage",
    reasoning: "Balance audit trail with cost"
}

---

Skills Ingestion Strategy

Phase 1: YAML Skills Registry

File: prompts/skills.yaml

skills:
  - id: "kernel-state-machine"
    category: "T0-kernel"
    level: "L2"
    title: "State Machine Kernel Patterns"
    content: |
      Key patterns for T0:
      - Event log append-only design
      - Atomic commit with 2PC
      - Fold determinism for state derivation
      - Fork/rewind for attempt recovery
    
  - id: "attempt-lifecycle"
    category: "T1-execution"
    level: "L2"
    title: "Attempt Lifecycle Management"
    content: |
      Execution loop patterns:
      - Poll-based task queue
      - Budget tracking (tokens, attempts, time)
      - Retry policy evaluation
      - Context capture per attempt
    
  - id: "prompt-optimization"
    category: "model-provider"
    level: "L2"
    title: "Memory-Based Prompt Optimization"
    content: |
      Best practices:
      - Retrieve 3-tier context before execution
      - Inject learned facts from tier-1 (exact matches)
      - Include tier-2 patterns (ML-similar)
      - Reference tier-3 docs (general guidance)
      - Set budget constraints from experience

Phase 2: Ingest Skills on Startup

// In cmd/starter/main.go

func ingestSkills(memSvc *memory.Service) error {
    skillsYAML, err := ioutil.ReadFile("prompts/skills.yaml")
    if err != nil {
        return err
    }
    
    var skillsConfig struct {
        Skills []struct {
            ID       string `yaml:"id"`
            Category string `yaml:"category"`
            Level    string `yaml:"level"`
            Title    string `yaml:"title"`
            Content  string `yaml:"content"`
        } `yaml:"skills"`
    }
    
    if err := yaml.Unmarshal(skillsYAML, &skillsConfig); err != nil {
        return err
    }
    
    for _, skill := range skillsConfig.Skills {
        _, err := memSvc.CreateKnowledge(ctx, &memory.KnowledgeRecord{
            Level:   skill.Level,
            Title:   skill.Title,
            Content: skill.Content,
            Source:  fmt.Sprintf("skills:///%s", skill.ID),
            Metadata: map[string]interface{}{
                "skill_id": skill.ID,
                "category": skill.Category,
                "type":     "skill",
            },
        })
        if err != nil {
            log.Warn(fmt.Sprintf("Failed to ingest skill %s: %v", skill.ID, err))
            continue
        }
        log.Info(fmt.Sprintf("Ingested skill: %s", skill.Title))
    }
    
    return nil
}

Phase 3: Reference Docs Ingestion

File: poimen/crates/doc/ (Rust doc comments)

/// # Attempt Lifecycle Pattern
///
/// Every step execution follows this sequence:
/// 1. Check budget (tokens, attempts, time remaining)
/// 2. Retrieve context from memory (3-tier)
/// 3. Optimize prompt with lessons & skills
/// 4. Execute with ModelProvider
/// 5. Evaluate with Verifier
/// 6. Decide with Judge
/// 7. Learn (success) or Diagnose (failure)
/// 8. Retry or proceed to next step
///
/// # Budget Tracking
/// - Tokens: Count LLM input/output tokens
/// - Attempts: Number of retries allowed
/// - Time: Wall-clock timeout per step
///
/// # Retry Policy
/// - Exponential backoff: 1s → 2s → 4s
/// - Max attempts: 3 (configurable)
/// - Non-retryable: Syntax errors, auth failures
pub struct AttemptState { ... }

Ingest Docs:

// Extract doc comments and ingest as L2 knowledge
// Run during build/startup:
// $ cargo doc --extract-comments | memory-ingest --level L2

Phase 4: Execution Pattern Capture

// In RunExecutor::execute_step()

func (e *Executor) execute_step(ctx *WorkflowContext, step *StepId) error {
    // ... execution logic ...
    
    // Capture pattern on success
    if output.status == Success {
        memSvc.CreateKnowledge(ctx, &memory.KnowledgeRecord{
            Level:   "L1",
            Title:   fmt.Sprintf("Successful %s execution", step),
            Content: fmt.Sprintf(
                "Step %s completed with output:\n%s",
                step, output.text,
            ),
            Source:   fmt.Sprintf("workflow://execution/%s", step),
            Metadata: map[string]interface{}{
                "step_id": step.String(),
                "phase": ctx.PhaseId,
                "attempt": ctx.AttemptState.Number,
                "tokens_used": output.tokens,
            },
        })
    }
}

---

Tool-Skill Mapping Matrix

┌────────────────────────────────────────────────────────────────┐
│ Tool → Skill Dependencies                                      │
├──────────────────────┬──────────────────────────────────────────┤
│ Tool                 │ Skills Needed (from memory)              │
├──────────────────────┼──────────────────────────────────────────┤
│ WorkflowDef Builder  │ • Phase dependencies                     │
│                      │ • IR canonicalization rules              │
│                      │ • Schema versioning                      │
├──────────────────────┼──────────────────────────────────────────┤
│ Event Log            │ • Event ordering guarantees              │
│                      │ • Atomic commit protocol                 │
│                      │ • Checksum validation                    │
├──────────────────────┼──────────────────────────────────────────┤
│ Run Executor         │ • Attempt lifecycle patterns             │
│                      │ • Budget exhaustion detection            │
│                      │ • Retry policy evaluation                │
├──────────────────────┼──────────────────────────────────────────┤
│ Verifier Port        │ • Rubric structure design                │
│                      │ • Failure categorization                 │
│                      │ • Score aggregation rules                │
├──────────────────────┼──────────────────────────────────────────┤
│ Judge Port           │ • Decision thresholds                    │
│                      │ • Evidence combination logic             │
│                      │ • Feedback injection patterns            │
├──────────────────────┼──────────────────────────────────────────┤
│ ModelProvider        │ • Prompt engineering best practices      │
│                      │ • Token budget awareness                 │
│                      │ • Model-specific quirks                  │
├──────────────────────┼──────────────────────────────────────────┤
│ EventLog Storage     │ • Serialization format choices           │
│                      │ • Compression strategies                 │
│                      │ • Recovery procedures                    │
├──────────────────────┼──────────────────────────────────────────┤
│ BlobStore            │ • Path naming conventions                │
│                      │ • Retention policies                     │
│                      │ • Archive triggers                       │
└──────────────────────┴──────────────────────────────────────────┘

---

Basic Tool Usage Example

Scenario: Planner Step Fails Repeatedly

User Command:

poimen plan my-workflow.yaml --phase T1 --retry-with-memory

Tool Execution Chain:

1. LOAD WORKFLOW
   WorkflowDefBuilder.from_yaml("my-workflow.yaml")
   → Memory: Retrieve "IR-canonicalization" skills
   → Validate against stored L2 knowledge
   
2. INIT EXECUTOR
   RunExecutor.new()
   → Memory: Get "attempt-lifecycle" context
   → Load retry policy from memory lessons
   
3. EXECUTE PLANNER STEP
   for attempt in 1..max_attempts:
       a) GetContextActivity
          - Tool: "planner"
          - Task: "step-generation"
          - Returns: lessons + skills
       
       b) OptimizePrompt
          - Inject learned facts (tier-1)
          - Add pattern examples (tier-2)
          - Set budget from history
       
       c) ModelProvider.run(optimized_prompt)
          - Send to planner agent
          - Wait for output
       
       d) Verifier.verify(output)
          - Check against rubric
          - Score output quality
       
       e) Judge.decide(output)
          - Approve | Retry | Reject
       
       f) On Success: LearnFromExecutionActivity
          - Store successful output pattern (L1)
       
       g) On Failure: AnalyzeErrorActivity
          - Search for similar failures
          - Return recovery suggestions
       
       h) DocumentDecisionActivity
          - Log decision and reasoning

4. COMPLETED
   ✅ Plan generated (or user feedback required)
   → Memory: Ingest execution pattern
   → Next phase starts

---

Summary: Tool & Skill Flow

Workflow Execution
      ↓
   Tools Used ────────────────→ Skills Retrieved from Memory
      ├─ WorkflowDefBuilder       ├─ IR canonicalization rules
      ├─ EventLog                 ├─ State machine patterns
      ├─ RunExecutor              ├─ Attempt lifecycle
      ├─ Verifier Port            ├─ Rubric design
      ├─ Judge Port               ├─ Decision logic
      ├─ ModelProvider            ├─ Prompt optimization
      └─ Storage Ports            └─ Retention policies
      
      Skills Guide Execution ──────→ Results Learned
                                    ├─ Success patterns (L1)
                                    ├─ Failure recovery (L1)
                                    ├─ Verified practices (L2)
                                    └─ Vault enriched for next run

This creates a virtuous cycle: Each execution improves the memory, which improves the next execution.