poimen-workflows/IMPLEMENTATION_PLAN.md

Poimen Declarative Workflow Implementation Plan

Executive Summary

Current State: Hardcoded workflows (OrchestratorWorkflow, TaskUnitWorkflow)

• Fixed flow: Clone → Plan → Implement → Judge → Commit
• New flows require recompilation
• No customer-specified sequencing

Target State: API-driven dynamic workflows

• Customer sends JSON specifying activity sequence
• Routing engine executes dynamically
• Support 3 patterns: Sequential, Await-Task-Complete, Retry
• Backward compatible with existing cmd/starter

Timeline: 4 weeks | Effort: ~80 story points | Team: 2-3 engineers

---

Phase 1: Core Routing Engine (Week 1) — 16 hours

Goal: Build engine that parses WorkflowSpec and executes activity sequences dynamically

Files to Create:

• internal/routing/types.go — Data structures for WorkflowSpec, ActivityStep, RetryConfig
• internal/routing/jsonpath_evaluator.go — Parameter interpolation using JSONPath
• internal/routing/execution_context.go — Execution state management
• internal/routing/activity_registry.go — Activity name → implementation mapper
• statemachine/routing_workflow.go — Core RoutingWorkflow that ties everything together
• Tests: *_test.go files for each module

Key Implementation Details:

• RoutingWorkflow is the main Temporal workflow
• It parses activity steps and executes them sequentially
• Uses JSONPath to interpolate parameters: ${step_1.output.path}
• Supports retry logic, error catching, and conditional jumps
• All activities are pre-registered in worker pool

Backward Compatibility: ✅ Keep OrchestratorWorkflow, TaskUnitWorkflow unchanged

---

Phase 2: API Server (Week 2) — 10 hours

Goal: HTTP API that accepts WorkflowSpec JSON and manages workflow execution

Files to Create:

• cmd/api-server/main.go — Server bootstrap, Temporal client connection
• cmd/api-server/handlers.go — POST /api/v1/workflows, GET /api/v1/workflows/{id}/status
• cmd/api-server/validation.go — WorkflowSpec validation (references, timeouts, etc.)
• cmd/api-server/server.go — Server config and lifecycle
• Tests: handlers, validation, integration tests

Endpoints:

POST /api/v1/workflows
  → Submit WorkflowSpec
  → Returns: {workflow_id, status, polling_url}

GET /api/v1/workflows/{workflow_id}/status
  → Poll execution status
  → Returns: {status, steps_completed, results, final_output}

Key Details:

• Validates WorkflowSpec before submission
• Submits to RoutingWorkflow (not OrchestratorWorkflow)
• Caches workflow metadata locally
• Polls Temporal for status

---

Phase 3: Homelab Frontend Integration (Week 3) — 5 hours

Goal: Expose API through ingress, connect to gateway

Files to Create:

• k8s/api-server-deployment.yaml — K8s Deployment, 2 replicas
• k8s/api-server-service.yaml — K8s Service for api-server
• k8s/api-server-networkpolicy.yaml — Allow ingress traffic, deny by default
• homelab-frontend/k8s/gateway-routes.yaml — Route /api/v1/workflows to api-server

Key Details:

• api-server connects to Temporal: temporal-frontend.temporal:7233
• Ingress routes external traffic to api-server:8080
• NetworkPolicy allows only ingress-nginx → api-server traffic

---

Phase 4: Migration & Docs (Week 4) — 7 hours

Files to Create:

• docs/API.md — API reference (endpoints, request/response examples)
• docs/MIGRATION.md — How to migrate from cmd/starter to API
• docs/examples/sequential-workflow.json — Example: Clone → Analyze → Implement → Judge
• docs/examples/retry-workflow.json — Example: with retry + error handling
• docs/examples/conditional-workflow.json — Example: with catch blocks

Key Details:

• Comprehensive examples for all 3 patterns
• cmd/starter continues to work (backward compatible)
• Documentation emphasizes JSONPath parameter passing

---

Three Composite Patterns

Pattern A: Sequential (Request-Response)

{
  "activities": [
    {"name": "Step1", "resource": "ActivityA", "parameters": {...}, "next": "Step2"},
    {"name": "Step2", "resource": "ActivityB", "parameters": {"input": "${Step1.output}"}, "next": "Step3"},
    {"name": "Step3", "resource": "ActivityC", "parameters": {...}, "end": true}
  ]
}

Flow: Step1 → Step2 (receives Step1's output) → Step3

---

Pattern B: Await-Task-Complete (Callback)

{
  "activities": [
    {"name": "LaunchJob", "type": "await-callback", "resource": "LaunchActivity", "heartbeat_timeout": "5m", "next": "ProcessResult"},
    {"name": "ProcessResult", "resource": "ProcessActivity", "parameters": {"result": "${LaunchJob.callback}"}, "end": true}
  ]
}

Flow: LaunchJob (waits for callback) → ProcessResult (receives callback data)

---

Pattern C: Retry with Error Handling

{
  "activities": [
    {"name": "Generate", "resource": "ImplementerActivity", "retry": {"maxAttempts": 3, "backoffRate": 2.0}, "catch": [{"errorEquals": ["Timeout"], "next": "HandleTimeout"}], "next": "Verify"},
    {"name": "Verify", "resource": "JudgeActivity", "end": true},
    {"name": "HandleTimeout", "type": "pass", "result": {"status": "timeout"}, "end": true}
  ]
}

Flow: Generate (retry 3x) → if success: Verify, else: HandleTimeout

---

Success Criteria

• ✅ RoutingWorkflow executes dynamic sequences correctly
• ✅ JSONPath parameter passing works (nested references, arrays)
• ✅ Error handling and catch blocks work
• ✅ Retry logic respects backoff intervals
• ✅ API validates WorkflowSpec (no invalid refs)
• ✅ Status polling returns step-by-step results
• ✅ All tests pass (>90% coverage)
• ✅ Backward compatible (cmd/starter still works)
• ✅ API exposed via ingress at /api/v1/workflows
• ✅ Performance: <200ms submit, <100ms poll

---

Architecture Diagram

┌─ Customer/Frontend ─────────────────────────┐
│  POST /api/v1/workflows (JSON WorkflowSpec) │
└────────────────────┬────────────────────────┘
                     ↓
          ┌──────────────────────┐
          │   API Server         │
          │ • Validate spec      │
          │ • Submit to Temporal │
          │ • Cache metadata     │
          └────────────┬─────────┘
                       ↓
      ┌────────────────────────────────┐
      │    Temporal Server             │
      │  RoutingWorkflow               │
      │  ├─ Parse activities           │
      │  ├─ Evaluate JSONPath          │
      │  ├─ Execute sequence           │
      │  └─ Handle errors/retries      │
      └────────────┬───────────────────┘
                   ↓
      ┌────────────────────────────────┐
      │  Temporal Worker Pool          │
      │  • Poll queue                  │
      │  • Execute activities          │
      │  • Return results              │
      └────────────────────────────────┘

---

Risk Mitigation

Risk	Mitigation
JSONPath bugs	Unit tests + property-based testing
Activity registry mismatch	Validate registered activities match spec
Temporal network issues	Retry logic + timeouts in API server
Cache staleness	Event-driven cache invalidation

---

Rollback Plan

Phase	Rollback Strategy
Phase 1	Keep old workflows, don't use RoutingWorkflow
Phase 2	Disable api-server, fall back to cmd/starter
Phase 3	Remove ingress, access api-server directly
Phase 4	Continue supporting both APIs (cmd/starter + API)

---

Next Steps

1. Week 1: Start Phase 1 implementation (RoutingWorkflow)
2. Week 2: Start Phase 2 (API Server)
3. Week 3: Deploy to K8s (Phase 3)
4. Week 4: Document, test, release (Phase 4)
5. Post-Release: Monitor, gather feedback, plan Phase 2 enhancements (parallel, conditional branching)