Test
c32f3e7077
ci / test (push) Successful in 1m50s
- Better describes the intent: awaiting task completion via queue - More descriptive than 'await-kmsvc' - Applied to all workflow design documents - Reflects the actual operation: wait for external task to complete Updated files: - DESIGN_MASTER_REVIEW.md - SERVICE_INTEGRATION.md - IMPLEMENTATION_PLAN.md - CLI_DESIGN.md - FINAL_COMPREHENSIVE_DESIGN.md
120 lines
3.1 KiB
Markdown
120 lines
3.1 KiB
Markdown
# Poimen Service Integration & Dynamic Workflows — FINAL DESIGN
|
|
|
|
**Status**: Ready for Implementation Review
|
|
**Updated**: 2025-01-31
|
|
**Scope**: Replace hardcoded workflows with API-driven, JSON-based, customer-specified sequences
|
|
|
|
---
|
|
|
|
## 1-Minute Summary
|
|
|
|
### Problem
|
|
Hardcoded workflows require code changes → recompile → deploy (5-10 min) to change activity sequences.
|
|
|
|
### Solution
|
|
- New **RoutingWorkflow** reads JSON WorkflowSpec and executes dynamically
|
|
- Three interfaces: **CLI** + **HTTP API** + **legacy CLI** (backward compat)
|
|
- Three patterns: Sequential, Await-Task-Complete (KMSvc), Retry with error handling
|
|
- JSONPath parameters: `${step1.output.path}`
|
|
|
|
### Result
|
|
Workflow changes in **seconds** (API call) instead of 5-10 min (recompile)
|
|
|
|
---
|
|
|
|
## The Three Patterns
|
|
|
|
### 1. Sequential (A → B → C)
|
|
Activities execute in sequence, each step passes output to next via JSONPath.
|
|
|
|
### 2. Await-Task-Complete (Launch → Poll Queue → Process)
|
|
Launch long-running job, poll KMSvc queue for result with correlation ID matching, then proceed.
|
|
|
|
### 3. Retry with Error Handling
|
|
Activity retry N times with exponential backoff, jump to catch block on failure.
|
|
|
|
---
|
|
|
|
## The Three Entry Points
|
|
|
|
### CLI (New)
|
|
```bash
|
|
poimen-cli submit workflow.json --wait
|
|
poimen-cli execute template code-review-v1 --input-file input.json
|
|
poimen-cli status wf-abc123
|
|
```
|
|
|
|
### HTTP API (New)
|
|
```bash
|
|
curl -X POST /api/v1/workflows -d @workflow.json
|
|
curl -X POST /api/v1/execute -d '{"template": "...", "input": {...}}'
|
|
curl /api/v1/workflows/wf-abc123/status
|
|
```
|
|
|
|
### Legacy CLI (Unchanged)
|
|
```bash
|
|
go run ./cmd/starter --repo ... --milestone T0 (still works)
|
|
```
|
|
|
|
---
|
|
|
|
## Implementation: 5 Phases
|
|
|
|
| Phase | Files | Hours | Week |
|
|
|-------|-------|-------|------|
|
|
| 1. Routing Engine | `internal/routing/`, `statemachine/routing_workflow.go` | 20-22 | 1 |
|
|
| 2. API Server | `cmd/api-server/` | 12-14 | 2 |
|
|
| 3. CLI | `cmd/cli/` | 8-10 | 2 |
|
|
| 4. K8s Deploy | `k8s/`, `homelab-frontend/k8s/` | 5 | 3 |
|
|
| 5. Docs | `docs/`, examples | 7 | 4 |
|
|
| **TOTAL** | **~15 files** | **52-58** | **4 weeks** |
|
|
|
|
---
|
|
|
|
## Critical KMSvc Questions
|
|
|
|
1. **What is KMSvc?** (Kafka/Redis/SQS/custom?)
|
|
2. **Message format?** (JSON with correlation_id field?)
|
|
3. **Who generates correlation_id?** (Activity or Workflow?)
|
|
4. **Polling strategy?** (Active loop or Temporal Signal?)
|
|
5. **Timeout behavior?** (Catch block, fail, or infinite wait?)
|
|
6. **Consumer group?** (Shared or per-workflow?)
|
|
|
|
---
|
|
|
|
## Success Criteria
|
|
|
|
✅ RoutingWorkflow executes sequences dynamically
|
|
✅ JSONPath parameters resolve correctly
|
|
✅ Error catch blocks work
|
|
✅ Retry with backoff works
|
|
✅ CLI reads JSON (no flags)
|
|
✅ API validates specs
|
|
✅ Backward compatible
|
|
✅ Performance: <200ms submit
|
|
✅ Test coverage: >90%
|
|
|
|
---
|
|
|
|
## Key Files
|
|
|
|
See full design doc (FINAL_COMPREHENSIVE_DESIGN.md in repo) for:
|
|
- Detailed phase breakdown
|
|
- Code examples
|
|
- Risk assessment
|
|
- Design decisions
|
|
- Complete file list
|
|
- Timeline details
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
1. Answer KMSvc Q1-Q6
|
|
2. Review 52-58 hour estimate
|
|
3. Stakeholder approval
|
|
4. Start Phase 1
|
|
|
|
See DESIGN_REVIEW_UPDATED.md for detailed questions.
|
|
|