refactor: rename await-kmsvc pattern to awaitTaskComplete

- Better describes the intent: awaiting task completion via queue - More descriptive than 'await-kmsvc' - Applied to all workflow design documents - Reflects the actual operation: wait for external task to complete Updated files: - DESIGN_MASTER_REVIEW.md - SERVICE_INTEGRATION.md - IMPLEMENTATION_PLAN.md - CLI_DESIGN.md - FINAL_COMPREHENSIVE_DESIGN.md
2026-08-31 18:30:11 -07:00
parent db71919207
commit c32f3e7077
7 changed files with 1769 additions and 2 deletions
@@ -0,0 +1,81 @@
 # Poimen CLI — Dynamic JSON-Based Design
 ## Overview
 New CLI (`cmd/cli`) that accepts JSON WorkflowSpecs and dynamically understands activity sequences—same mechanism as HTTP API, but command-line interface.
 ## Key Commands
 ```bash
 # Submit workflow from JSON
 poimen-cli submit workflow.json
 poimen-cli submit workflow.json --wait
 poimen-cli submit workflow.json --watch
 # Check status
 poimen-cli status wf-abc123
 poimen-cli status wf-abc123 --wait
 # Template management
 poimen-cli template list
 poimen-cli template show code-review-v1
 # Execute immediately (sync)
 poimen-cli execute template code-review-v1 --input-file input.json
 ```
 ## Workflow JSON Example
 ```json
 {
  "name": "code-review-pipeline",
  "input": {"repo": "https://github.com/..."},
  "activities": [
    {
      "name": "Clone",
      "resource": "CloneRepoActivity",
      "parameters": {"repo": "${input.repo}"},
      "timeout": "5m",
      "next": "Analyze"
    },
    {
      "name": "Analyze",
      "resource": "AnalyzeCodeActivity",
      "parameters": {"path": "${Clone.output.path}"},
      "timeout": "10m",
      "retry": {"maxAttempts": 3},
      "catch": [{"errorEquals": ["Timeout"], "next": "HandleTimeout"}],
      "next": "Judge"
    }
  ]
 }
 ```
 ## File Structure
 ```
 cmd/
  ├─ cli/                  (NEW)
  │  ├─ main.go
  │  └─ commands/
  │     ├─ submit.go
  │     ├─ status.go
  │     ├─ template.go
  │     ├─ execute.go
  │     └─ history.go
  ├─ starter/              (OLD: keep for backward compat)
  └─ worker/
 ```
 ## Three Entry Points (Same Core)
 | Interface | Endpoint | Input | Output |
 |-----------|----------|-------|--------|
 | **CLI** | `submit workflow.json` | JSON file | workflow_id |
 | **CLI** | `execute template T` | Template name | Results (sync) |
 | **API** | `POST /api/v1/workflows` | JSON body | workflow_id |
 | **API** | `POST /api/v1/execute` | JSON body | Results (sync) |
 All use same `RoutingWorkflow` internally—just different interfaces.
 See CLI_DESIGN.md in repo for full implementation details.
@@ -0,0 +1,889 @@
 # Poimen Service Integration & Dynamic Workflows — MASTER REVIEW DOCUMENT
 **Status**: Ready for Approval  
 **Version**: 1.0  
 **Date**: 2025-01-31  
 **Duration**: 52-58 hours (4 weeks)
 ---
 ## EXECUTIVE SUMMARY (5 MINUTES)
 ### Problem
 Current workflows are **hardcoded in Go**. Changing activity sequences requires:
 - Code modification
 - Recompilation
 - K8s deployment
 - Pod restart
 - **Total: 5-10 minutes**
 ### Solution
 New **RoutingWorkflow** reads JSON WorkflowSpec and executes dynamically:
 - **Three entry points**: CLI + HTTP API + Legacy CLI (backward compatible)
 - **Three patterns**: Sequential, Await-Task-Complete (KMSvc queue), Retry with error handling
 - **JSONPath parameters**: `${step1.output.path}` instead of hardcoded values
 - **Result**: Workflow changes in **seconds** (API call only)
 ### Impact
 ✅ Instant changes | ✅ Three interfaces | ✅ Full compatibility | ❌ +KMSvc complexity
 ---
 ## PART 1: THE THREE PATTERNS
 ### Pattern 1: Sequential (A → B → C)
 Activities execute in sequence. Each step passes output to next via JSONPath.
 ```json
 {
  "activities": [
    {
      "name": "Clone",
      "resource": "CloneRepoActivity",
      "parameters": {"repo": "${input.repo}"},
      "timeout": "5m",
      "next": "Analyze"
    },
    {
      "name": "Analyze",
      "resource": "AnalyzeCodeActivity",
      "parameters": {"path": "${Clone.output.path}"},
      "timeout": "10m",
      "next": "Judge"
    },
    {
      "name": "Judge",
      "resource": "JudgeActivity",
      "parameters": {"code": "${Analyze.output.code}"},
      "timeout": "5m",
      "end": true
    }
  ]
 }
 ```
 **Execution Flow**:
 ```
 Clone ──(output: {path: /tmp/repo})──> Analyze ──(output: {code: ...})──> Judge ──(final result)
 ```
 **Use Case**: Code review pipeline (clone → analyze → judge)
 ---
 ### Pattern 2: Await-Task-Complete (Launch → Poll Queue → Process)
 Launch long-running job, poll KMSvc queue for result with correlation ID matching, then proceed.
 ```json
 {
  "name": "LaunchJob",
  "type": "awaitTaskComplete",
  "resource": "LaunchJobActivity",
  "parameters": {"job_id": "job-123"},
  "queue": "job-completions",
  "correlation_key": "${LaunchJob.output.correlation_id}",
  "timeout": "5m",
  "next": "ProcessResult"
 }
 ```
 **Execution Flow**:
 ```
 LaunchJobActivity ──(returns {correlation_id: "xyz"})──> Poll Queue "job-completions"
                                                            ↓
                                                    [External system completes job]
                                                    [Publishes {correlation_id: "xyz", result: ...}]
                                                            ↓
                                                    [Workflow receives message]
                                                    [Proceeds to ProcessResult]
 ```
 **Use Case**: Long-running batch jobs (model training, data processing)
 ---
 ### Pattern 3: Retry with Error Handling
 Activity retries N times with exponential backoff. On failure, jumps to catch block.
 ```json
 {
  "name": "Generate",
  "resource": "ImplementerActivity",
  "parameters": {"task": "${input.task}"},
  "timeout": "30m",
  "retry": {
    "maxAttempts": 3,
    "backoffRate": 2.0,
    "initialInterval": "1s"
  },
  "catch": [
    {"errorEquals": ["Timeout"], "next": "HandleTimeout"}
  ],
  "next": "Verify"
 }
 ```
 **Execution Flow**:
 ```
 Generate (attempt 1) ──TIMEOUT──> [Wait 1s]
 Generate (attempt 2) ──TIMEOUT──> [Wait 2s]
 Generate (attempt 3) ──TIMEOUT──> [Wait 4s]
 Max retries exceeded ──> Jump to HandleTimeout
 ```
 **Use Case**: Code generation (retry on timeout, handle gracefully)
 ---
 ## PART 2: THE THREE ENTRY POINTS
 ### Entry Point 1: CLI (New Command-Line)
 ```bash
 # Submit workflow from JSON file
 $ poimen-cli submit workflow.json
 Workflow submitted: wf-abc123
 Poll status: poimen-cli status wf-abc123
 # Submit and wait for results
 $ poimen-cli submit workflow.json --wait
 Waiting for workflow to complete...
 ✅ Workflow completed
 Output: {...}
 # Execute template immediately (sync, no polling)
 $ poimen-cli execute template code-review-v1 --input-file input.json
 Executing template...
 Step 1/3: Clone ... OK (45s)
 Step 2/3: Analyze ... OK (120s)
 Step 3/3: Judge ... OK (30s)
 ✅ COMPLETED
 Output: {...}
 # Check status later
 $ poimen-cli status wf-abc123
 Workflow ID: wf-abc123
 Status: COMPLETED
 Started: 2025-01-31 15:04:05
 Completed: 2025-01-31 15:05:00
 # List templates
 $ poimen-cli template list
 Available Templates:
  - code-review-v1
  - clone-analyze-v1
 ```
 **Implementation**: `cmd/cli/` (new directory)
 ---
 ### Entry Point 2: HTTP API (New)
 #### Route 1: Submit Workflow (Async)
 ```bash
 curl -X POST http://api.example.com/api/v1/workflows \
  -H "Content-Type: application/json" \
  -d @workflow.json
 ```
 **Response**:
 ```json
 {
  "workflow_id": "wf-abc123",
  "status": "RUNNING",
  "polling_url": "/api/v1/workflows/wf-abc123/status"
 }
 ```
 #### Check Status
 ```bash
 curl http://api.example.com/api/v1/workflows/wf-abc123/status
 ```
 **Response**:
 ```json
 {
  "workflow_id": "wf-abc123",
  "status": "COMPLETED",
  "steps_completed": 3,
  "results": {
    "Clone": { "status": "COMPLETED", "output": {...} },
    "Analyze": { "status": "COMPLETED", "output": {...} },
    "Judge": { "status": "COMPLETED", "output": {...} }
  },
  "final_output": {...}
 }
 ```
 #### Route 2: Execute Template (Sync)
 ```bash
 curl -X POST http://api.example.com/api/v1/execute \
  -H "Content-Type: application/json" \
  -d '{
    "template": "code-review-v1",
    "input": {"repo": "https://github.com/...", "branch": "feature/x"},
    "timeout": "2m"
  }'
 ```
 **Response** (immediate):
 ```json
 {
  "status": "COMPLETED",
  "results": {...},
  "final_output": {...}
 }
 ```
 **Implementation**: `cmd/api-server/` (new directory)
 ---
 ### Entry Point 3: Legacy CLI (Unchanged)
 ```bash
 # Old way still works, completely backward compatible
 $ go run ./cmd/starter \
  --repo https://github.com/rockliang/poimen \
  --remote file:///tmp/poimen-output \
  --milestone T0 \
  --planner-model ornith \
  --judge-model ornith \
  --implementer-model claude-sonnet-5
 Workflow submitted: orch-poimen
 Status: RUNNING
 ```
 **Implementation**: `cmd/starter/` (existing, unchanged)
 ---
 ## PART 3: ARCHITECTURE OVERVIEW
 ```
 ┌─────────────────────────────────────────────────────────────────┐
 │                   THREE ENTRY POINTS                             │
 ├─────────────────────────────────────────────────────────────────┤
 │                                                                   │
 │  CLI (JSON file)           HTTP API (JSON POST)    Legacy CLI   │
 │  ┌──────────────┐          ┌──────────────┐        ┌─────────┐  │
 │  │ poimen-cli   │          │ /api/v1/     │        │ cmd/    │  │
 │  │ submit       │          │ workflows    │        │ starter │  │
 │  └──────┬───────┘          └──────┬───────┘        └────┬────┘  │
 │         │                         │                     │        │
 └─────────┼─────────────────────────┼─────────────────────┼────────┘
          │                         │                     │
          └─────────────────────────┼─────────────────────┘
                                    ↓
                    ┌───────────────────────────┐
                    │   RoutingWorkflow         │
                    │   (NEW - generic)         │
                    │                           │
                    │ 1. Parse WorkflowSpec     │
                    │ 2. For each activity:     │
                    │    - Resolve JSONPath     │
                    │    - Execute activity     │
                    │    - Handle errors        │
                    │    - Store result         │
                    │ 3. Return results         │
                    └───────────┬───────────────┘
                                ↓
                    ┌───────────────────────────┐
                    │  Temporal Server          │
                    │                           │
                    │  Worker Pool:             │
                    │  - Executes activities    │
                    │  - Returns results        │
                    │  - Retries with backoff   │
                    └───────────────────────────┘
 ```
 ---
 ## PART 4: BEFORE vs AFTER
 ### Before (Hardcoded Workflows)
 ```bash
 # To change activity sequence:
 1. Modify Go code (OrchestratorWorkflow in statemachine/)
 2. Compile: go build ./cmd/worker
 3. Push docker image
 4. Deploy to K8s: kubectl set image deployment/poimen-worker
 5. Wait for rollout
 Time: 5-10 minutes
 ```
 **Workflow definition**:
 ```go
 // Hard to change, tight coupling to specific sequence
 if err := workflow.ExecuteActivity(ctx, "CloneRepoActivity", params).Get(ctx, nil); err != nil {
    return err
 }
 if err := workflow.ExecuteActivity(ctx, "PlanningActivity", params2).Get(ctx, nil); err != nil {
    return err
 }
 // ... more activities hardcoded
 ```
 ---
 ### After (Dynamic Workflows)
 ```bash
 # To change activity sequence:
 1. Edit workflow.json (define sequence)
 2. Call API or CLI
 3. Done
 Time: Seconds
 ```
 **Workflow definition**:
 ```json
 {
  "activities": [
    {"name": "Clone", "resource": "CloneRepoActivity", "parameters": {...}, "next": "Analyze"},
    {"name": "Analyze", "resource": "AnalyzeCodeActivity", "parameters": {...}, "next": "Judge"},
    {"name": "Judge", "resource": "JudgeActivity", "parameters": {...}, "end": true}
  ]
 }
 ```
 ---
 ## PART 5: IMPLEMENTATION PHASES
 ### Phase 1: Routing Engine (Week 1) — 20-22 Hours
 **Goal**: Core RoutingWorkflow that executes JSON sequences dynamically
 **Files to Create**:
 ```
 internal/routing/
  ├─ types.go                    (2h)  - WorkflowSpec, ActivityStep
  ├─ jsonpath_evaluator.go       (3h)  - ${step1.output.path} resolution
  ├─ execution_context.go        (2h)  - State tracking (vars, step results)
  ├─ activity_registry.go        (2h)  - Activity name → function mapping
  ├─ kmsvc_client.go             (2-3h)- KMSvc queue polling (await-continue)
  └─ *_test.go                   (tests)
 statemachine/
  ├─ routing_workflow.go         (4-5h)- ⭐ Main workflow logic
  └─ routing_workflow_test.go    (2h)  - Tests
 cmd/worker/main.go: Register RoutingWorkflow (1h)
 ```
 **Deliverable**: RoutingWorkflow executes activity sequences with:
 - ✅ Sequential execution (A → B → C)
 - ✅ Await-continue pattern (KMSvc queue polling)
 - ✅ Retry with exponential backoff
 - ✅ Error catch blocks
 - ✅ JSONPath parameter resolution
 **Risk**: Low-Medium (KMSvc queue polling adds complexity)
 ---
 ### Phase 2: API Server (Week 2) — 12-14 Hours
 **Goal**: HTTP endpoints for workflow submission and status polling
 **Files to Create**:
 ```
 cmd/api-server/
  ├─ main.go                     (2h)  - Server bootstrap
  ├─ handlers.go                 (4h)  - Routes: /workflows, /execute, status
  ├─ validation.go               (2h)  - Validate WorkflowSpec
  ├─ template_loader.go          (2-3h)- Load predefined templates
  └─ *_test.go                   (2h)  - Tests
 ```
 **Deliverable**:
 - `POST /api/v1/workflows` — Submit workflow (async)
 - `GET /api/v1/workflows/{id}/status` — Check status
 - `POST /api/v1/execute` — Execute template (sync)
 **Risk**: Low (isolated, no breaking changes)
 ---
 ### Phase 3: CLI (Week 2) — 8-10 Hours
 **Goal**: Command-line interface for workflow submission
 **Files to Create**:
 ```
 cmd/cli/
  ├─ main.go                     (2h)  - Entry point, command routing
  └─ commands/
     ├─ submit.go                (2h)  - poimen-cli submit workflow.json
     ├─ status.go                (2h)  - poimen-cli status wf-id
     ├─ template.go              (1h)  - poimen-cli template list/show
     ├─ execute.go               (1h)  - poimen-cli execute template
     └─ history.go               (1h)  - poimen-cli history wf-id
 ```
 **Deliverable**:
 - `poimen-cli submit workflow.json [--wait] [--watch]`
 - `poimen-cli status wf-id [--wait]`
 - `poimen-cli execute template name --input-file input.json`
 - `poimen-cli template list/show`
 **Risk**: Low
 ---
 ### Phase 4: Kubernetes Integration (Week 3) — 5 Hours
 **Goal**: Deploy api-server to K8s, expose via ingress
 **Files to Create**:
 ```
 k8s/
  ├─ api-server-deployment.yaml  (2h)  - 2 replicas, Temporal connection
  ├─ api-server-service.yaml     (1h)  - Port 8080
  └─ api-server-networkpolicy.yaml(1h) - Allow ingress traffic
 homelab-frontend/k8s/
  └─ gateway-routes.yaml         (1h)  - Route /api/v1/workflows to api-server
 ```
 **Deliverable**: API exposed at `https://api.riotpiao.com/api/v1/workflows`
 **Risk**: Medium (network policy changes)
 ---
 ### Phase 5: Documentation (Week 4) — 7 Hours
 **Files to Create**:
 ```
 docs/
  ├─ API.md                      (2h)  - Complete API reference
  ├─ CLI.md                      (2h)  - CLI commands + examples
  ├─ TEMPLATES.md                (1h)  - How to create/manage templates
  ├─ MIGRATION.md                (1h)  - Migration from cmd/starter
  └─ examples/
     ├─ sequential-workflow.json
     ├─ retry-workflow.json
     └─ template-execute.json
 ```
 **Deliverable**: Complete documentation with working examples
 **Risk**: Low
 ---
 ### Timeline Summary
 ```
 Phase 1: Routing Engine      20-22 hours  (Week 1)
 Phase 2: API Server          12-14 hours  (Week 2)
 Phase 3: CLI                  8-10 hours  (Week 2)
 Phase 4: K8s Integration       5 hours    (Week 3)
 Phase 5: Documentation         7 hours    (Week 4)
 ─────────────────────────────────────────────────
 TOTAL                         52-58 hours (~1.5 weeks)
 Team: 2-3 engineers
 Duration: 4 weeks (with overlapping phases)
 ```
 ---
 ## PART 6: CRITICAL KMSVC QUESTIONS
 These answers **block implementation** of Phase 1:
 ### Q1: What is KMSvc?
 - Is it Kafka? Redis? AWS SQS? Custom system?
 - What Go client library should we use?
 - Example: `github.com/segmentio/kafka-go`?
 ### Q2: Message Format
 - Is it JSON?
 - What fields are required? (e.g., `correlation_id`, `result`, `status`?)
 - Example:
 ```json
 {
  "correlation_id": "xyz123",
  "status": "completed",
  "result": {...}
 }
 ```
 ### Q3: Correlation ID Management
 - Who generates the correlation_id? (LaunchJobActivity? Or RoutingWorkflow?)
 - Is it returned in LaunchJobActivity output?
 - Example: LaunchJobActivity returns:
 ```json
 {
  "correlation_id": "abc123-generated-by-activity",
  "job_id": "job-123"
 }
 ```
 ### Q4: Queue Polling Strategy
 - Active polling loop (pull messages every N seconds)? OR
 - Temporal Signal (push-based, external system sends signal to workflow)?
 - If polling: What interval? (1s, 5s, 10s?)
 ### Q5: Message Timeout Behavior
 - If no message arrives within timeout (e.g., 5m), what happens?
  - Option A: Jump to catch block (error handler)
  - Option B: Fail entire workflow
  - Option C: Infinite wait (only human intervention can stop)
 ### Q6: Consumer Group Strategy
 - Single shared consumer (all workflows share one connection)?
 - Per-workflow consumer (each workflow gets own connection)?
 - Dead-letter queue for unmatched messages?
 ---
 ## PART 7: KEY DESIGN DECISIONS
 ### Decision 1: Three Patterns (Sequential, Await-Task-Complete, Retry)
 **Rationale**: Covers 90% of use cases. Retry critical for production. Await-Task-Complete enables long-running jobs.
 **Alternative Rejected**: Just Sequential (too limited)
 ### Decision 2: KMSvc Queue for Await-Task-Complete
 **Rationale**: Decouples external systems from Temporal. External systems only need queue client, not Temporal SDK.
 **Alternative Rejected**: Temporal callback/signal (tight coupling)
 ### Decision 3: JSONPath for Parameters
 **Rationale**: Standard notation (AWS Step Functions, Kubernetes). Supports nested refs and arrays.
 **Alternative Rejected**: Go templating (overkill, security risks)
 ### Decision 4: RoutingWorkflow (not Activity)
 **Rationale**: Workflows can call activities with retries. Durable execution + replay guarantee.
 **Alternative Rejected**: RoutingActivity (can't call other activities, no retries)
 ### Decision 5: Static Activity Registry
 **Rationale**: Type-safe, simpler. Worker pod must compile all activities anyway.
 **Alternative Rejected**: Dynamic registration (complex, unsafe)
 ### Decision 6: Keep cmd/starter Forever
 **Rationale**: No need to break existing deployments. Let customers choose.
 **Alternative Rejected**: Deprecate (breaking change)
 ### Decision 7: Shared api-server (2 replicas)
 **Rationale**: Simpler, resource-efficient, easier to scale.
 **Alternative Rejected**: Per-namespace servers (overkill)
 ---
 ## PART 8: RISKS & MITIGATIONS
 | Risk | Severity | Mitigation |
 |------|----------|-----------|
 | JSONPath evaluation bugs | Medium | Unit tests (>90% coverage), property-based testing |
 | **KMSvc queue reliability** | **High** | Dead-letter queue, timeouts, persistence checks |
 | **Correlation ID mismatch** | **Medium** | Strict matching, schema validation, detailed logging |
 | **Consumer group conflicts** | **Medium** | Clear strategy, rebalancing, message redelivery |
 | Activity registry mismatch | Low | Validate at spec submission time |
 | Temporal network issues | Low | Retry + timeout in api-server |
 | Cache staleness (api-server) | Low | 24h TTL, event-driven invalidation |
 | Performance regression | Low | Measure baseline: target <200ms submit, <100ms poll |
 **Total Risk Score**: Medium-High (up from Low-Medium due to KMSvc)
 ---
 ## PART 9: SUCCESS CRITERIA
 ### Functional ✅
 - RoutingWorkflow executes activity sequences dynamically
 - JSONPath parameters resolve correctly (nested, arrays)
 - Error catch blocks jump to handlers
 - Retry logic with exponential backoff (1s → 2s → 4s)
 - API validates WorkflowSpec (no dangling refs)
 - CLI reads JSON (no hardcoded flags)
 - Status polling returns step-by-step results
 - Backward compatible (cmd/starter unchanged)
 ### Non-Functional ✅
 - Performance: <200ms workflow submit, <100ms poll
 - Uptime: 99.9% (2 replicas, rolling updates)
 - Test coverage: >90%
 - Documentation: Complete + 3 working examples
 ---
 ## PART 10: BACKWARD COMPATIBILITY
 ### What Stays Unchanged ✅
 - `cmd/starter/` (old CLI works exactly as before)
 - All existing activities
 - All existing workflows (OrchestratorWorkflow, TaskUnitWorkflow)
 - Temporal worker registration
 ### What's New ✅
 - `cmd/cli/` (new CLI)
 - `cmd/api-server/` (new HTTP API)
 - `internal/routing/` (new routing engine)
 - `statemachine/routing_workflow.go` (new workflow)
 ### Migration Path ✅
 - Customers can use OLD way or NEW way
 - No forced upgrades
 - No breaking changes
 ---
 ## PART 11: FILES SUMMARY
 ### New Files (Phase 1-5)
 **Routing Engine** (Phase 1):
 ```
 internal/routing/types.go
 internal/routing/jsonpath_evaluator.go
 internal/routing/execution_context.go
 internal/routing/activity_registry.go
 internal/routing/kmsvc_client.go
 statemachine/routing_workflow.go
 ```
 **API Server** (Phase 2):
 ```
 cmd/api-server/main.go
 cmd/api-server/handlers.go
 cmd/api-server/validation.go
 cmd/api-server/template_loader.go
 ```
 **CLI** (Phase 3):
 ```
 cmd/cli/main.go
 cmd/cli/commands/submit.go
 cmd/cli/commands/status.go
 cmd/cli/commands/template.go
 cmd/cli/commands/execute.go
 cmd/cli/commands/history.go
 ```
 **K8s** (Phase 4):
 ```
 k8s/api-server-deployment.yaml
 k8s/api-server-service.yaml
 k8s/api-server-networkpolicy.yaml
 homelab-frontend/k8s/gateway-routes.yaml
 ```
 **Docs** (Phase 5):
 ```
 docs/API.md
 docs/CLI.md
 docs/TEMPLATES.md
 docs/MIGRATION.md
 docs/examples/*.json
 ```
 ### Modified Files
 - `cmd/worker/main.go` (register RoutingWorkflow)
 - `homelab-frontend/k8s/network-policy.yaml` (already done ✅)
 ---
 ## PART 12: COMPARISON TABLE
 | Aspect | Before (Hardcoded) | After (Dynamic) |
 |--------|---|---|
 | **Workflow Definition** | Go code (OrchestratorWorkflow) | JSON (customer-specified) |
 | **Sequencing** | Fixed (Clone→Plan→Impl→Judge) | Any (customer defines in JSON) |
 | **Parameters** | Direct: `in.RemoteURL` | JSONPath: `${input.repo}` |
 | **Change Time** | 5-10 min (recompile+deploy) | Seconds (API call) |
 | **Entry Points** | CLI only (hardcoded flags) | CLI + API + Legacy CLI |
 | **Error Handling** | Try-catch (per-activity) | Catch blocks (conditional jumps) |
 | **Retry Policy** | Temporal default | Configurable per-activity |
 | **Backward Compat** | N/A | 100% (cmd/starter unchanged) |
 | **Code Coupling** | Tight (to specific sequence) | Loose (generic RoutingWorkflow) |
 ---
 ## PART 13: APPROVAL CHECKLIST
 ### Must Answer (Blocks Phase 1)
 - [ ] Q1: What is KMSvc system?
 - [ ] Q2: Message format for KMSvc?
 - [ ] Q3: Who generates correlation_id?
 - [ ] Q4: Polling strategy (active or signal)?
 - [ ] Q5: Timeout behavior?
 - [ ] Q6: Consumer group strategy?
 ### Should Approve
 - [ ] Timeline OK? (52-58 hours)
 - [ ] Architecture OK? (RoutingWorkflow + 3 entry points)
 - [ ] Risk level acceptable? (Medium-High)
 - [ ] Backward compatibility OK? (Keep cmd/starter)
 ### Sign-Off
 - [ ] Tech Lead approval
 - [ ] Stakeholder sign-off
 - [ ] Ready to start Phase 1
 ---
 ## PART 14: NEXT STEPS
 1. **Review This Document** (1-2 hours)
   - Read Parts 1-7 (understanding)
   - Read Parts 8-9 (risks & success)
   - Read Part 13 (approval checklist)
 2. **Answer Critical Questions** (Q1-Q6)
   - Email or Slack responses required
   - Needed before Phase 1 can start
 3. **Stakeholder Approval**
   - Architecture review meeting
   - Risk acknowledgment
   - Sign-off on timeline
 4. **Start Implementation**
   - Week 1: Phase 1 (RoutingWorkflow)
   - Week 2: Phase 2 (API) + Phase 3 (CLI)
   - Week 3: Phase 4 (K8s deploy)
   - Week 4: Phase 5 (Docs)
 ---
 ## APPENDIX: EXAMPLE WORKFLOW JSON
 ```json
 {
  "name": "code-review-pipeline",
  "input": {
    "repo": "https://github.com/rockliang/poimen",
    "branch": "feature/new-cli"
  },
  "activities": [
    {
      "name": "Clone",
      "resource": "CloneRepoActivity",
      "parameters": {
        "repo": "${input.repo}",
        "branch": "${input.branch}"
      },
      "timeout": "5m",
      "next": "Analyze"
    },
    {
      "name": "Analyze",
      "resource": "AnalyzeCodeActivity",
      "parameters": {
        "path": "${Clone.output.path}"
      },
      "timeout": "10m",
      "retry": {
        "maxAttempts": 3,
        "backoffRate": 2.0,
        "initialInterval": "1s"
      },
      "catch": [
        {
          "errorEquals": ["Timeout"],
          "next": "HandleTimeout"
        }
      ],
      "next": "Judge"
    },
    {
      "name": "Judge",
      "resource": "JudgeActivity",
      "parameters": {
        "code": "${Analyze.output.code}",
        "analysis": "${Analyze.output.analysis}"
      },
      "timeout": "5m",
      "end": true
    },
    {
      "name": "HandleTimeout",
      "type": "pass",
      "result": {
        "status": "failed",
        "reason": "Analysis timed out after 10m"
      },
      "end": true
    }
  ]
 }
 ```
 ---
 ## APPENDIX: CLI USAGE EXAMPLES
 ```bash
 # Submit workflow from JSON file
 $ poimen-cli submit workflow.json
 Workflow submitted: wf-abc123
 Poll status: poimen-cli status wf-abc123
 # Submit and wait for results
 $ poimen-cli submit workflow.json --wait
 Waiting for workflow to complete...
 ✅ Workflow completed
 Output:
 {
  "verdict": "approved",
  "score": 0.95,
  "issues": []
 }
 # Check status
 $ poimen-cli status wf-abc123
 Workflow ID: wf-abc123
 Status: COMPLETED
 Started: 2025-01-31 15:04:05
 Completed: 2025-01-31 15:05:00
 # Execute template immediately (sync)
 $ poimen-cli execute template code-review-v1 --input-file input.json
 Executing template: code-review-v1
 [15:04:05] Step 1/3: Clone ... OK (45s)
 [15:04:50] Step 2/3: Analyze ... OK (120s)
 [15:05:50] Step 3/3: Judge ... OK (30s)
 ✅ COMPLETED
 Output: {...}
 # List templates
 $ poimen-cli template list
 Available Templates:
  - code-review-v1
  - clone-analyze-v1
  - simple-test-v1
 # Show template
 $ poimen-cli template show code-review-v1
 Template: code-review-v1
 Activities: 3
 1. Clone (CloneRepoActivity)
   Timeout: 5m
 2. Analyze (AnalyzeCodeActivity)
   Timeout: 10m
   Retry: 3 attempts, backoff 2.0
 3. Judge (JudgeActivity)
   Timeout: 5m
 ```
 ---
 **END OF MASTER REVIEW DOCUMENT**
 For detailed technical documentation, see:
 - SERVICE_INTEGRATION.md (patterns + API spec)
 - IMPLEMENTATION_PLAN.md (phase breakdown)
 - CLI_DESIGN.md (CLI implementation)
 - FINAL_COMPREHENSIVE_DESIGN.md (1-2 hour read)
@@ -0,0 +1,119 @@
 # Poimen Service Integration & Dynamic Workflows — FINAL DESIGN
 **Status**: Ready for Implementation Review  
 **Updated**: 2025-01-31  
 **Scope**: Replace hardcoded workflows with API-driven, JSON-based, customer-specified sequences
 ---
 ## 1-Minute Summary
 ### Problem
 Hardcoded workflows require code changes → recompile → deploy (5-10 min) to change activity sequences.
 ### Solution
 - New **RoutingWorkflow** reads JSON WorkflowSpec and executes dynamically
 - Three interfaces: **CLI** + **HTTP API** + **legacy CLI** (backward compat)
 - Three patterns: Sequential, Await-Task-Complete (KMSvc), Retry with error handling
 - JSONPath parameters: `${step1.output.path}`
 ### Result
 Workflow changes in **seconds** (API call) instead of 5-10 min (recompile)
 ---
 ## The Three Patterns
 ### 1. Sequential (A → B → C)
 Activities execute in sequence, each step passes output to next via JSONPath.
 ### 2. Await-Task-Complete (Launch → Poll Queue → Process)
 Launch long-running job, poll KMSvc queue for result with correlation ID matching, then proceed.
 ### 3. Retry with Error Handling
 Activity retry N times with exponential backoff, jump to catch block on failure.
 ---
 ## The Three Entry Points
 ### CLI (New)
 ```bash
 poimen-cli submit workflow.json --wait
 poimen-cli execute template code-review-v1 --input-file input.json
 poimen-cli status wf-abc123
 ```
 ### HTTP API (New)
 ```bash
 curl -X POST /api/v1/workflows -d @workflow.json
 curl -X POST /api/v1/execute -d '{"template": "...", "input": {...}}'
 curl /api/v1/workflows/wf-abc123/status
 ```
 ### Legacy CLI (Unchanged)
 ```bash
 go run ./cmd/starter --repo ... --milestone T0 (still works)
 ```
 ---
 ## Implementation: 5 Phases
 | Phase | Files | Hours | Week |
 |-------|-------|-------|------|
 | 1. Routing Engine | `internal/routing/`, `statemachine/routing_workflow.go` | 20-22 | 1 |
 | 2. API Server | `cmd/api-server/` | 12-14 | 2 |
 | 3. CLI | `cmd/cli/` | 8-10 | 2 |
 | 4. K8s Deploy | `k8s/`, `homelab-frontend/k8s/` | 5 | 3 |
 | 5. Docs | `docs/`, examples | 7 | 4 |
 | **TOTAL** | **~15 files** | **52-58** | **4 weeks** |
 ---
 ## Critical KMSvc Questions
 1. **What is KMSvc?** (Kafka/Redis/SQS/custom?)
 2. **Message format?** (JSON with correlation_id field?)
 3. **Who generates correlation_id?** (Activity or Workflow?)
 4. **Polling strategy?** (Active loop or Temporal Signal?)
 5. **Timeout behavior?** (Catch block, fail, or infinite wait?)
 6. **Consumer group?** (Shared or per-workflow?)
 ---
 ## Success Criteria
 ✅ RoutingWorkflow executes sequences dynamically  
 ✅ JSONPath parameters resolve correctly  
 ✅ Error catch blocks work  
 ✅ Retry with backoff works  
 ✅ CLI reads JSON (no flags)  
 ✅ API validates specs  
 ✅ Backward compatible  
 ✅ Performance: <200ms submit  
 ✅ Test coverage: >90%  
 ---
 ## Key Files
 See full design doc (FINAL_COMPREHENSIVE_DESIGN.md in repo) for:
 - Detailed phase breakdown
 - Code examples
 - Risk assessment
 - Design decisions
 - Complete file list
 - Timeline details
 ---
 ## Next Steps
 1. Answer KMSvc Q1-Q6
 2. Review 52-58 hour estimate
 3. Stakeholder approval
 4. Start Phase 1
 See DESIGN_REVIEW_UPDATED.md for detailed questions.
@@ -0,0 +1,226 @@
 # Poimen Declarative Workflow Implementation Plan
 ## Executive Summary
 **Current State**: Hardcoded workflows (OrchestratorWorkflow, TaskUnitWorkflow)
 - Fixed flow: Clone → Plan → Implement → Judge → Commit
 - New flows require recompilation
 - No customer-specified sequencing
 **Target State**: API-driven dynamic workflows
 - Customer sends JSON specifying activity sequence
 - Routing engine executes dynamically
 - Support 3 patterns: Sequential, Await-Task-Complete, Retry
 - Backward compatible with existing cmd/starter
 **Timeline**: 4 weeks | **Effort**: ~80 story points | **Team**: 2-3 engineers
 ---
 ## Phase 1: Core Routing Engine (Week 1) — 16 hours
 **Goal**: Build engine that parses WorkflowSpec and executes activity sequences dynamically
 **Files to Create**:
 - `internal/routing/types.go` — Data structures for WorkflowSpec, ActivityStep, RetryConfig
 - `internal/routing/jsonpath_evaluator.go` — Parameter interpolation using JSONPath
 - `internal/routing/execution_context.go` — Execution state management
 - `internal/routing/activity_registry.go` — Activity name → implementation mapper
 - `statemachine/routing_workflow.go` — **Core RoutingWorkflow that ties everything together**
 - Tests: `*_test.go` files for each module
 **Key Implementation Details**:
 - RoutingWorkflow is the main Temporal workflow
 - It parses activity steps and executes them sequentially
 - Uses JSONPath to interpolate parameters: `${step_1.output.path}`
 - Supports retry logic, error catching, and conditional jumps
 - All activities are pre-registered in worker pool
 **Backward Compatibility**: ✅ Keep OrchestratorWorkflow, TaskUnitWorkflow unchanged
 ---
 ## Phase 2: API Server (Week 2) — 10 hours
 **Goal**: HTTP API that accepts WorkflowSpec JSON and manages workflow execution
 **Files to Create**:
 - `cmd/api-server/main.go` — Server bootstrap, Temporal client connection
 - `cmd/api-server/handlers.go` — POST /api/v1/workflows, GET /api/v1/workflows/{id}/status
 - `cmd/api-server/validation.go` — WorkflowSpec validation (references, timeouts, etc.)
 - `cmd/api-server/server.go` — Server config and lifecycle
 - Tests: handlers, validation, integration tests
 **Endpoints**:
 ```
 POST /api/v1/workflows
  → Submit WorkflowSpec
  → Returns: {workflow_id, status, polling_url}
 GET /api/v1/workflows/{workflow_id}/status
  → Poll execution status
  → Returns: {status, steps_completed, results, final_output}
 ```
 **Key Details**:
 - Validates WorkflowSpec before submission
 - Submits to RoutingWorkflow (not OrchestratorWorkflow)
 - Caches workflow metadata locally
 - Polls Temporal for status
 ---
 ## Phase 3: Homelab Frontend Integration (Week 3) — 5 hours
 **Goal**: Expose API through ingress, connect to gateway
 **Files to Create**:
 - `k8s/api-server-deployment.yaml` — K8s Deployment, 2 replicas
 - `k8s/api-server-service.yaml` — K8s Service for api-server
 - `k8s/api-server-networkpolicy.yaml` — Allow ingress traffic, deny by default
 - `homelab-frontend/k8s/gateway-routes.yaml` — Route /api/v1/workflows to api-server
 **Key Details**:
 - api-server connects to Temporal: `temporal-frontend.temporal:7233`
 - Ingress routes external traffic to api-server:8080
 - NetworkPolicy allows only ingress-nginx → api-server traffic
 ---
 ## Phase 4: Migration & Docs (Week 4) — 7 hours
 **Files to Create**:
 - `docs/API.md` — API reference (endpoints, request/response examples)
 - `docs/MIGRATION.md` — How to migrate from cmd/starter to API
 - `docs/examples/sequential-workflow.json` — Example: Clone → Analyze → Implement → Judge
 - `docs/examples/retry-workflow.json` — Example: with retry + error handling
 - `docs/examples/conditional-workflow.json` — Example: with catch blocks
 **Key Details**:
 - Comprehensive examples for all 3 patterns
 - cmd/starter continues to work (backward compatible)
 - Documentation emphasizes JSONPath parameter passing
 ---
 ## Three Composite Patterns
 ### Pattern A: Sequential (Request-Response)
 ```json
 {
  "activities": [
    {"name": "Step1", "resource": "ActivityA", "parameters": {...}, "next": "Step2"},
    {"name": "Step2", "resource": "ActivityB", "parameters": {"input": "${Step1.output}"}, "next": "Step3"},
    {"name": "Step3", "resource": "ActivityC", "parameters": {...}, "end": true}
  ]
 }
 ```
 **Flow**: Step1 → Step2 (receives Step1's output) → Step3
 ---
 ### Pattern B: Await-Task-Complete (Callback)
 ```json
 {
  "activities": [
    {"name": "LaunchJob", "type": "await-callback", "resource": "LaunchActivity", "heartbeat_timeout": "5m", "next": "ProcessResult"},
    {"name": "ProcessResult", "resource": "ProcessActivity", "parameters": {"result": "${LaunchJob.callback}"}, "end": true}
  ]
 }
 ```
 **Flow**: LaunchJob (waits for callback) → ProcessResult (receives callback data)
 ---
 ### Pattern C: Retry with Error Handling
 ```json
 {
  "activities": [
    {"name": "Generate", "resource": "ImplementerActivity", "retry": {"maxAttempts": 3, "backoffRate": 2.0}, "catch": [{"errorEquals": ["Timeout"], "next": "HandleTimeout"}], "next": "Verify"},
    {"name": "Verify", "resource": "JudgeActivity", "end": true},
    {"name": "HandleTimeout", "type": "pass", "result": {"status": "timeout"}, "end": true}
  ]
 }
 ```
 **Flow**: Generate (retry 3x) → if success: Verify, else: HandleTimeout
 ---
 ## Success Criteria
 - ✅ RoutingWorkflow executes dynamic sequences correctly
 - ✅ JSONPath parameter passing works (nested references, arrays)
 - ✅ Error handling and catch blocks work
 - ✅ Retry logic respects backoff intervals
 - ✅ API validates WorkflowSpec (no invalid refs)
 - ✅ Status polling returns step-by-step results
 - ✅ All tests pass (>90% coverage)
 - ✅ Backward compatible (cmd/starter still works)
 - ✅ API exposed via ingress at `/api/v1/workflows`
 - ✅ Performance: <200ms submit, <100ms poll
 ---
 ## Architecture Diagram
 ```
 ┌─ Customer/Frontend ─────────────────────────┐
 │  POST /api/v1/workflows (JSON WorkflowSpec) │
 └────────────────────┬────────────────────────┘
                     ↓
          ┌──────────────────────┐
          │   API Server         │
          │ • Validate spec      │
          │ • Submit to Temporal │
          │ • Cache metadata     │
          └────────────┬─────────┘
                       ↓
      ┌────────────────────────────────┐
      │    Temporal Server             │
      │  RoutingWorkflow               │
      │  ├─ Parse activities           │
      │  ├─ Evaluate JSONPath          │
      │  ├─ Execute sequence           │
      │  └─ Handle errors/retries      │
      └────────────┬───────────────────┘
                   ↓
      ┌────────────────────────────────┐
      │  Temporal Worker Pool          │
      │  • Poll queue                  │
      │  • Execute activities          │
      │  • Return results              │
      └────────────────────────────────┘
 ```
 ---
 ## Risk Mitigation
 | Risk | Mitigation |
 |------|-----------|
 | JSONPath bugs | Unit tests + property-based testing |
 | Activity registry mismatch | Validate registered activities match spec |
 | Temporal network issues | Retry logic + timeouts in API server |
 | Cache staleness | Event-driven cache invalidation |
 ---
 ## Rollback Plan
 | Phase | Rollback Strategy |
 |-------|------------------|
 | Phase 1 | Keep old workflows, don't use RoutingWorkflow |
 | Phase 2 | Disable api-server, fall back to cmd/starter |
 | Phase 3 | Remove ingress, access api-server directly |
 | Phase 4 | Continue supporting both APIs (cmd/starter + API) |
 ---
 ## Next Steps
 1. **Week 1**: Start Phase 1 implementation (RoutingWorkflow)
 2. **Week 2**: Start Phase 2 (API Server)
 3. **Week 3**: Deploy to K8s (Phase 3)
 4. **Week 4**: Document, test, release (Phase 4)
 5. **Post-Release**: Monitor, gather feedback, plan Phase 2 enhancements (parallel, conditional branching)
@@ -0,0 +1,452 @@
 # Poimen Service Integration & Declarative Workflow Design
 ## Overview
 **Current (Bad)**: Hardcoded workflows
 ```
 CLI flags → OrchestratorWorkflow (fixed flow)
 ```
 **Proposed (Better)**: API-driven, customer-specified workflows
 ```
 API Request (JSON) → Routing Agent → Dynamic Activity Sequence → Results
 ```
 ---
 ## Part 1: Service Integration Patterns (AWS Step Functions Inspired)
 ### Pattern 1: Sequential (Request-Response)
 ```yaml
 # Customer specifies: "Call A, then call B with A's output"
 activities:
  - name: CloneRepo
    type: request-response
    resource: CloneRepoActivity
    parameters:
      repo: "https://github.com/..."
      path: "/tmp/clone"
    timeout: 5m
    retry:
      maxAttempts: 3
      backoff: exponential
  - name: Analyze
    type: request-response
    resource: AnalyzeCodeActivity
    parameters:
      path: $.CloneRepo.path  # ← Reference previous output
    inputPath: "$.CloneRepo"   # ← What to pass
    outputPath: "$"            # ← What to return
    timeout: 10m
 ```
 **Execution**: CloneRepo waits → outputs result → Analyze receives it → outputs
 ---
 ### Pattern 2: Await-Task-Complete (Callback)
 ```yaml
 # Customer specifies: "Call A, wait for callback, then call B"
 activities:
  - name: LaunchJob
    type: await-callback
    resource: LaunchLongRunningJobActivity
    parameters:
      job_id: "job-123"
    heartbeat_timeout: 5m
    next: OnJobComplete
  - name: OnJobComplete
    type: request-response
    resource: ProcessResultActivity
    parameters:
      result: $.LaunchJob.result
 ```
 **Execution**: LaunchJob sent → waits for callback (heartbeat) → ProcessResult receives callback data → outputs
 ---
 ### Pattern 3: Retry with Backoff
 ```yaml
 # Customer specifies: "Call A, if fails retry 3x with exponential backoff, else B"
 activities:
  - name: GenerateCode
    type: request-response
    resource: ImplementerActivity
    parameters:
      task: $.input.taskID
      model: $.input.model
    retry:
      maxAttempts: 3
      backoffRate: 2.0        # 1s → 2s → 4s
      initialInterval: 1s
      maxInterval: 10m
    catch:
      - errorEquals: ["Timeout", "ThrottlingException"]
        next: HandleError
      - errorEquals: ["States.ALL"]
        next: HandleError
    next: JudgeCode
  - name: JudgeCode
    type: request-response
    resource: JudgeActivity
    parameters:
      implementation: $.GenerateCode.code
  - name: HandleError
    type: pass
    result: { status: "failed", reason: $.error }
    resultPath: "$.error_output"
    end: true
 ```
 **Execution**: GenerateCode fails → retry 1 (wait 1s) → fails → retry 2 (wait 2s) → fails → HandleError
 ---
 ## Part 2: Routing Agent Architecture
 ### 2.1 Routing Agent Activity (New)
 ```go
 // internal/routing/routing_agent.go
 type WorkflowSpec struct {
    WorkflowID    string           `json:"workflow_id"`
    Name          string           `json:"name"`
    Input         map[string]interface{} `json:"input"`
    Activities    []ActivityStep   `json:"activities"`
    RetryPolicy   RetryConfig      `json:"retry_policy,omitempty"`
 }
 type ActivityStep struct {
    Name          string                 `json:"name"`
    Type          string                 `json:"type"` // request-response, await-callback, parallel, choice
    Resource      string                 `json:"resource"` // Activity name
    Parameters    map[string]interface{} `json:"parameters"`
    InputPath     string                 `json:"input_path"` // JSONPath: "$" or "$.previous.output"
    OutputPath    string                 `json:"output_path"` // What to extract
    Timeout       time.Duration          `json:"timeout"`
    Retry         *RetryConfig           `json:"retry,omitempty"`
    Catch         []CatchBlock           `json:"catch,omitempty"`
    Next          string                 `json:"next,omitempty"`
    End           bool                   `json:"end,omitempty"`
 }
 type RetryConfig struct {
    MaxAttempts      int           `json:"max_attempts"`
    BackoffRate      float64       `json:"backoff_rate"`
    InitialInterval  time.Duration `json:"initial_interval"`
    MaxInterval      time.Duration `json:"max_interval"`
 }
 type CatchBlock struct {
    ErrorEquals []string `json:"error_equals"`
    Next        string   `json:"next"`
    ResultPath  string   `json:"result_path,omitempty"`
 }
 // RoutingAgentActivity: Receives WorkflowSpec, executes activities sequentially
 func RoutingAgentActivity(ctx context.Context, spec WorkflowSpec) (map[string]interface{}, error) {
    // 1. Parse WorkflowSpec
    // 2. Build execution plan (DAG of activities)
    // 3. For each step:
    //    a. Evaluate JSONPath inputs
    //    b. Execute activity with retries
    //    c. Handle errors (catch blocks)
    //    d. Store output in context
    //    e. Move to next step
    // 4. Return final result
 }
 ```
 ### 2.2 Activity Mapper Registry (New)
 ```go
 // internal/routing/activity_registry.go
 type ActivityMapper struct {
    activities map[string]interface{}
 }
 func NewActivityMapper() *ActivityMapper {
    return &ActivityMapper{
        activities: map[string]interface{}{
            "CloneRepoActivity":      action.CloneRepoActivity,
            "AnalyzeCodeActivity":    action.AnalyzeCodeActivity,
            "ImplementerActivity":    action.ImplementerActivity,
            "JudgeActivity":          action.JudgeActivity,
            "GitCommitActivity":      action.GitCommitActivity,
            // ... all activities registered here
        },
    }
 }
 func (am *ActivityMapper) Get(name string) (interface{}, error) {
    if activity, ok := am.activities[name]; ok {
        return activity, nil
    }
    return nil, fmt.Errorf("activity %s not found", name)
 }
 // cmd/worker/main.go
 w.RegisterActivity(routing.RoutingAgentActivity)  // ← Only register this
 w.RegisterActivity(action.CloneRepoActivity)      // ← Still register all individual activities
 // ... rest of activities
 ```
 ---
 ## Part 3: API Message Format (General Purpose)
 ### 3.1 Workflow Request Format
 ```json
 POST /api/workflows
 {
  "workflow": {
    "id": "workflow-123",
    "name": "code-review-pipeline",
    "version": "v1",
    "input": {
      "repo": "https://github.com/rockliang/poimen",
      "branch": "feature/new-feature",
      "milestone": "T0"
    },
    "activities": [
      {
        "id": "step_1",
        "name": "CloneRepo",
        "type": "request-response",
        "resource": "CloneRepoActivity",
        "parameters": {
          "repo": "${input.repo}",
          "path": "/tmp/work"
        },
        "timeout": "5m",
        "next": "step_2"
      },
      {
        "id": "step_2",
        "name": "AnalyzeCode",
        "type": "request-response",
        "resource": "AnalyzeCodeActivity",
        "parameters": {
          "path": "${step_1.output.path}"
        },
        "inputPath": "$.step_1",
        "outputPath": "$",
        "timeout": "10m",
        "retry": {
          "maxAttempts": 3,
          "backoffRate": 2.0,
          "initialInterval": "1s"
        },
        "catch": [
          {
            "errorEquals": ["Timeout"],
            "next": "handle_timeout"
          }
        ],
        "next": "step_3"
      },
      {
        "id": "step_3",
        "name": "GenerateImplementation",
        "type": "request-response",
        "resource": "ImplementerActivity",
        "parameters": {
          "analysis": "${step_2.output.analysis}",
          "model": "claude-sonnet-5"
        },
        "timeout": "30m",
        "retry": {
          "maxAttempts": 3,
          "backoffRate": 2.0
        },
        "next": "step_4"
      },
      {
        "id": "step_4",
        "name": "Review",
        "type": "request-response",
        "resource": "JudgeActivity",
        "parameters": {
          "code": "${step_3.output.code}",
          "analysis": "${step_2.output.analysis}"
        },
        "timeout": "10m",
        "end": true
      },
      {
        "id": "handle_timeout",
        "name": "TimeoutHandler",
        "type": "pass",
        "result": {
          "status": "failed",
          "reason": "Analysis timed out"
        },
        "end": true
      }
    ]
  }
 }
 ```
 ### 3.2 Workflow Response Format
 ```json
 {
  "workflow_id": "workflow-123",
  "status": "RUNNING",
  "temporal_workflow_id": "orch-workflow-123",
  "task_queue": "poimen-taskqueue",
  "execution_started_at": "2025-01-29T10:00:00Z",
  "monitoring": {
    "web_ui": "http://temporal.local:8080/namespaces/poimen/workflows/orch-workflow-123",
    "poll_url": "/api/workflows/workflow-123/status"
  }
 }
 ```
 ### 3.3 Status Poll Format
 ```json
 GET /api/workflows/workflow-123/status
 {
  "workflow_id": "workflow-123",
  "status": "COMPLETED",
  "steps_completed": 4,
  "total_steps": 4,
  "results": {
    "step_1": {
      "name": "CloneRepo",
      "status": "COMPLETED",
      "duration": "45s",
      "output": {
        "path": "/tmp/work/poimen",
        "commit": "abc123"
      }
    },
    "step_2": {
      "name": "AnalyzeCode",
      "status": "COMPLETED",
      "duration": "120s",
      "output": {
        "analysis": "..." 
      }
    },
    "step_3": {
      "name": "GenerateImplementation",
      "status": "COMPLETED",
      "duration": "300s",
      "output": {
        "code": "..."
      }
    },
    "step_4": {
      "name": "Review",
      "status": "COMPLETED",
      "duration": "60s",
      "output": {
        "verdict": "approved",
        "score": 0.95
      }
    }
  },
  "final_output": {
    "verdict": "approved",
    "score": 0.95
  },
  "completed_at": "2025-01-29T10:10:00Z"
 }
 ```
 ---
 ## Part 4: Implementation Plan
 ### Phase 1: Core Routing Engine (Week 1)
 - [ ] Create `internal/routing/types.go` (WorkflowSpec, ActivityStep structs)
 - [ ] Create `internal/routing/jsonpath_evaluator.go` (parameter interpolation)
 - [ ] Create `internal/routing/execution_context.go` (execution state management)
 - [ ] Create `internal/routing/activity_registry.go` (activity mapper)
 - [ ] Create `statemachine/routing_workflow.go` (RoutingWorkflow implementation)
 - [ ] Update `cmd/worker/main.go` (register RoutingWorkflow)
 - [ ] Write tests for routing engine
 ### Phase 2: API Server (Week 2)
 - [ ] Create `cmd/api-server/main.go` (HTTP server bootstrap)
 - [ ] Create `cmd/api-server/handlers.go` (POST /workflows, GET /status)
 - [ ] Create `cmd/api-server/validation.go` (WorkflowSpec validation)
 - [ ] Create `internal/routing/client.go` (Temporal client wrapper)
 - [ ] Write integration tests
 ### Phase 3: Homelab Frontend Integration (Week 3)
 - [ ] Create `k8s/gateway-routes.yaml` (route /api/v1/workflows to api-server)
 - [ ] Deploy api-server as K8s Service
 - [ ] Update NetworkPolicy to allow traffic to api-server
 - [ ] Test end-to-end API flow
 ### Phase 4: Migration & Cleanup (Week 4)
 - [ ] Keep cmd/starter for backward compatibility
 - [ ] Document migration path
 - [ ] Update examples to use new API
 - [ ] Performance tuning
 ---
 ## Files to Create
 ```
 NEW:
  internal/routing/
    ├─ types.go                    (WorkflowSpec, ActivityStep, RetryConfig)
    ├─ jsonpath_evaluator.go       (Parameter interpolation)
    ├─ execution_context.go        (State management)
    ├─ activity_registry.go        (Activity mapper)
    └─ routing_workflow.go         (RoutingWorkflow implementation)
  statemachine/
    └─ routing_workflow.go         (See Part 5 below)
  cmd/api-server/
    ├─ main.go                     (Server bootstrap)
    ├─ handlers.go                 (HTTP handlers)
    └─ validation.go               (Spec validation)
 MODIFY:
  cmd/worker/main.go              (Register RoutingWorkflow)
 NEW (Homelab Frontend):
  k8s/
    └─ gateway-routes.yaml         (Route definitions)
 ```
 ---
 ## Backward Compatibility
 ✅ **Existing cmd/starter** still works (calls OrchestratorWorkflow)
 ✅ **New API** calls RoutingWorkflow
 ✅ **Worker** registers both workflows
 ✅ **No breaking changes** to current deployments
 ---
 ## Benefits
 ✅ Customer-driven workflows (JSON API)
 ✅ Service Integration patterns (Sequential, Await-Task-Complete, Retry)
 ✅ No recompilation needed
 ✅ Full Temporal observability
 ✅ JSONPath-based parameter passing
 ✅ Composable activities
 ✅ Error handling & retries built-in
@@ -9,6 +9,6 @@ metadata:
    app.kubernetes.io/name: poimen
    app.kubernetes.io/component: orchestrator
 data:
-  GIT_COMMIT: "74a8f2e8"  # Updated automatically by CI/CD
+  GIT_COMMIT: "04792563"  # Updated automatically by CI/CD
  GIT_BRANCH: "main"
  DEPLOYMENT_DATE: "2026-08-31"
@@ -13,7 +13,7 @@ spec:
      labels:
        app: poimen-worker
      annotations:
-        git-commit: "74a8f2e8"  # ✅ Updated on each push, triggers rolling restart
+        git-commit: "04792563"  # ✅ Updated on each push, triggers rolling restart
        deployment-date: "2026-08-31"
    spec:
      containers: