diff --git a/tasks/ROADMAP.md b/tasks/ROADMAP.md new file mode 100644 index 0000000..e33a512 --- /dev/null +++ b/tasks/ROADMAP.md @@ -0,0 +1,102 @@ +# Project Roadmap + +Multi-agent dev orchestrator for Temporal, building toward a fully autonomous software development system. + +## Milestones + +### T0: Core System (IN PROGRESS) +9 tasks, foundational: Planner/Judge/Implementer orchestration, git workflow, LLM integration, e2e validation. + +**Status:** 9 subtasks defined, testable criteria in place. +**Timeline:** ~2-4 weeks to implementation. +**Done:** All T0.1–T0.9 pass verification → squash-merge to main. + +**Board:** `tasks/board.md` + +--- + +### T1: Production Hardening (PLANNED) +8 tasks: Error recovery, observability, metrics, audit logging, health checks. + +**Focus:** Reliability for long-running orchestrators in homelab. +**Key wins:** +- Resume from crash without data loss +- Structured logging + Grafana metrics +- Auto-tuning based on historical failures +- Audit trail for compliance + +**Board:** `tasks/board-T1.md` + +--- + +### T2: Scale & Performance (PLANNED) +8 tasks: Caching, parallelism, batching, distributed locking. + +**Focus:** Handle 100s of concurrent tasks, reduce API call overhead. +**Key wins:** +- Result caching deduplicates LLM calls +- Parallel task dispatch (9x wall-clock speedup) +- Git operation batching (fewer network round-trips) +- Distributed lock for multi-pod safety + +**Board:** `tasks/board-T2.md` + +--- + +### T3: Feature Expansion (PLANNED) +8 tasks: Plugins, templates, dependencies, human gates, custom judges, nested workflows. + +**Focus:** Extensibility + domain specialization. +**Key wins:** +- Load custom skill plugins +- Save/load orchestrator config as templates +- Task dependency ordering +- Swap Judge for domain-specific validator (security auditor, code reviewer, etc.) +- Nest orchestrators (multi-level hierarchy) +- Import tasks from GitHub/Linear/JIRA + +**Board:** `tasks/board-T3.md` + +--- + +## Implementation Order + +1. **T0 (2-4 weeks):** Core system working end-to-end. +2. **T1 (2 weeks):** Production-harden the core. +3. **T2 (3 weeks):** Scale & optimize. +4. **T3 (4 weeks):** Advanced features. + +**Total:** ~3 months to full feature parity. + +--- + +## Success Metrics + +- **T0:** System runs unsupervised on 9-task milestone, all subtasks auto-complete, squash-merge to main succeeds. +- **T1:** 1000+ tasks completed, zero data loss across pod restarts, metrics queryable. +- **T2:** 1000 concurrent tasks complete 10x faster than T0, < 5 API calls/task (vs current ~20). +- **T3:** Custom Judge plugin loads and validates tasks, workflow templates save/restore state. + +--- + +## Risk Mitigation + +| Risk | Mitigation | +|------|-----------| +| Temporal cluster unavailability | Implement activity heartbeat recovery + resumption logic (T1.1) | +| Git conflicts on shared FS | Test multi-pod concurrent access (T1.6) → upgrade to distributed lock if needed (T2.8) | +| LLM API latency | Implement result caching + batching (T2.1, T2.6) | +| Board state corruption | Detect + auto-heal (T1.4) | +| Audit trail tampering | Sign audit log with workflow key (T3.6) | + +--- + +## Notes + +- Each milestone is independent: T1 can start once T0 core is complete, doesn't need full T0 cleanup. +- T0 tasks are foundational; changes will ripple into T1–T3 test suites, but core API should remain stable. +- Future milestones (T4+) could focus on: web UI, real-time dashboard, automated deployment of orchestrator itself (meta!), multi-cluster orchestration. + +--- + +**Next:** Start T0.1 (repo scaffold). diff --git a/tasks/board-T1.md b/tasks/board-T1.md new file mode 100644 index 0000000..447370a --- /dev/null +++ b/tasks/board-T1.md @@ -0,0 +1,19 @@ +# Task Board — Milestone T1: Production Hardening + +**Submilestone:** T1 (Error recovery, observability, metrics, reliability) + +| ID | Scope | Status | Branch | Verification | +|----|-------|--------|--------|--------------| +| T1.1 | Workflow error recovery: retry policies, deadletter handling, graceful shutdown | [ ] | `task/T1.1` | Simulate orchestrator crash mid-cycle, resume without data loss | +| T1.2 | Structured logging + metrics export (Prometheus/OpenTelemetry integration) | [ ] | `task/T1.2` | Metrics visible in homelab Grafana, logs queryable in Loki | +| T1.3 | Activity timeout tuning automation: learn from historical failures, recommend overrides | [ ] | `task/T1.3` | Planner reads lessons file, suggests `update-tuning` signal based on patterns | +| T1.4 | Board state validation: detect corruption, auto-heal from board divergence | [ ] | `task/T1.4` | Corrupt board file recovered without manual intervention | +| T1.5 | Workflow pause/resume with state snapshot: serialize mid-cycle state to persistent store | [ ] | `task/T1.5` | Pause signal, restart pod, resume signal → workflow continues from exact point | +| T1.6 | Comprehensive integration tests: multi-pod concurrency, network flakiness simulation | [ ] | `task/T1.6` | Concurrent orchestrator instances on shared repo pass e2e without conflicts | +| T1.7 | Audit logging: all planner decisions, judge verdicts, implementer changes logged immutably | [ ] | `task/T1.7` | Audit log persists across workflow restarts, queryable by task/timestamp | +| T1.8 | Health checks: Temporal connectivity, git repo accessibility, LLM API availability | [ ] | `task/T1.8` | Periodic health probes, liveness/readiness endpoints for K8s | + +--- + +## Submission Criteria +All T1.1–T1.8 marked `[x]` → submilestone complete → squash-merge `task/T1.*` to main. diff --git a/tasks/board-T2.md b/tasks/board-T2.md new file mode 100644 index 0000000..f3899d2 --- /dev/null +++ b/tasks/board-T2.md @@ -0,0 +1,19 @@ +# Task Board — Milestone T2: Scale & Performance + +**Submilestone:** T2 (Distributed execution, caching, performance optimization) + +| ID | Scope | Status | Branch | Verification | +|----|-------|--------|--------|--------------| +| T2.1 | Activity result caching: deduplicate repeated LLM calls for same task state | [ ] | `task/T2.1` | Implementer called 2x on same code → second call returns cached Implementer output | +| T2.2 | Parallel task dispatch: multiple T0.x tasks execute truly concurrently (not sequential) | [ ] | `task/T2.2` | 9 tasks complete in ~1/9 total time (wall-clock speedup measured) | +| T2.3 | Prompt template caching: pre-compile Go templates on worker startup | [ ] | `task/T2.3` | Template render latency < 100ms (vs parse+render each time) | +| T2.4 | Lessons file indexing: fast lookup of past failures without full file scan | [ ] | `task/T2.4` | Query lessons by task type → return in < 10ms for 1000s of entries | +| T2.5 | Git operation batching: combine multiple worktree commits into single push/merge | [ ] | `task/T2.5` | N tasks → 1 push (vs N pushes), measured via git ref-log | +| T2.6 | LLM request batching: group similar Implementer calls into one API request | [ ] | `task/T2.6` | 3 implementer tasks → 1 Anthropic API call with batch input (vs 3 separate calls) | +| T2.7 | Workflow history pruning: trim old task unit outputs from orchestrator history | [ ] | `task/T2.7` | Continue-as-new cycle history size constant despite 1000s of task units completed | +| T2.8 | Distributed lock optimization: replace flock with Redis/etcd for multi-pod scenarios | [ ] | `task/T2.8` | 5 concurrent orchestrators on different pods share FS safely via distributed lock | + +--- + +## Submission Criteria +All T2.1–T2.8 marked `[x]` → submilestone complete → squash-merge `task/T2.*` to main. diff --git a/tasks/board-T3.md b/tasks/board-T3.md new file mode 100644 index 0000000..cd64e1d --- /dev/null +++ b/tasks/board-T3.md @@ -0,0 +1,19 @@ +# Task Board — Milestone T3: Feature Expansion + +**Submilestone:** T3 (Custom plugins, workflow templates, audit, advanced features) + +| ID | Scope | Status | Branch | Verification | +|----|-------|--------|--------|--------------| +| T3.1 | Custom skill plugins: load user-defined skills from plugin registry (not just pi clone) | [ ] | `task/T3.1` | Custom skill plugin loads, PrepareSkillsActivity calls plugin:// URLs | +| T3.2 | Workflow templates: save/load orchestrator config as YAML templates (not CLI flags only) | [ ] | `task/T3.2` | Load template `templates/golang-project.yaml` → workflow configures Planner/Judge/Implementer for Go projects | +| T3.3 | Task dependency graph: specify task order (T0.2 must complete before T0.3 can start) | [ ] | `task/T3.3` | Board supports `depends_on: [T0.1]` field, orchestrator respects ordering | +| T3.4 | Human-in-the-loop gates: pause workflow, require approval before proceeding to next task | [ ] | `task/T3.4` | Workflow waits for `approve-task` signal, Judge verdict is final (can't auto-retry after user approval) | +| T3.5 | Custom Judge implementations: swap default Judge for domain-specific validator (e.g., security auditor) | [ ] | `task/T3.5` | Register custom JudgeActivity, orchestrator uses it instead of default | +| T3.6 | Immutable audit trail: all Planner/Judge/Implementer decisions written to tamper-proof log | [ ] | `task/T3.6` | Audit log signed with per-workflow key, verification prevents tampering | +| T3.7 | Workflow composition: nest OrchestratorWorkflows (one orchestrator dispatches child orchestrators) | [ ] | `task/T3.7` | Multi-level task hierarchy: T0 milestone → T0.a/T0.b sub-milestones, each with own orchestrator | +| T3.8 | Integration with external task systems: import tasks from Linear, GitHub Issues, JIRA | [ ] | `task/T3.8` | Load board from GitHub Issues API, update issues with task completion status | + +--- + +## Submission Criteria +All T3.1–T3.8 marked `[x]` → submilestone complete → squash-merge `task/T3.*` to main.