T1.6: Comprehensive Integration Tests for Concurrency - Add tests/concurrency_integration_test.go - Test concurrent workflows on shared resources - Test board validation concurrency - Test state tracking under concurrent access - Test snapshot creation and restoration concurrency - Test pause/resume under load - Test data consistency with concurrent access - Test network flakiness simulation - Test cross-workflow isolation - Benchmark concurrent snapshot and state operations - 15 integration tests, all passing T1.7: Immutable Audit Logging - Add internal/audit package for decision tracking - Implement AuditLogger with append-only JSONL logs - Log planner decisions with reasoning - Log judge verdicts with reasoning - Log implementer changes with file lists - Query by task ID (queryable by task) - Query by workflow ID - Query by actor (planner/judge/implementer) - Query by timestamp range - Full audit trail retrieval - Event counting and statistics - 14 audit tests, all passing Audit Features: - Immutable append-only JSONL logs - Event ID generation - Timestamp tracking (exact recovery point) - Full reasoning and context preservation - Metadata storage for extensibility - Thread-safe concurrent logging - Fast queries by task/workflow/actor/time Test Coverage: - 15 concurrency integration tests (workflows, board, state, snapshots) - 14 audit logging tests (decisions, verdicts, queries, immutability) - 29 total T1.6+T1.7 tests, all passing - Concurrent access patterns verified - Data consistency under load verified - Query functionality comprehensive T1 Milestone: 8/8 tasks COMPLETE (100%)
20 lines
1.8 KiB
Markdown
20 lines
1.8 KiB
Markdown
# Task Board — Milestone T1: Production Hardening
|
||
|
||
**Submilestone:** T1 (Error recovery, observability, metrics, reliability)
|
||
|
||
| ID | Scope | Status | Branch | Verification |
|
||
|----|-------|--------|--------|--------------|
|
||
| T1.1 | Workflow error recovery: retry policies, deadletter handling, graceful shutdown | [x] | `task/T1.1` | Simulate orchestrator crash mid-cycle, resume without data loss |
|
||
| T1.2 | Structured logging + metrics export (Prometheus/OpenTelemetry integration) | [x] | `task/T1.2` | Metrics visible in homelab Grafana, logs queryable in Loki |
|
||
| T1.3 | Activity timeout tuning automation: learn from historical failures, recommend overrides | [x] | `task/T1.3` | Planner reads lessons file, suggests `update-tuning` signal based on patterns |
|
||
| T1.4 | Board state validation: detect corruption, auto-heal from board divergence | [x] | `task/T1.4` | Corrupt board file recovered without manual intervention |
|
||
| T1.5 | Workflow pause/resume with state snapshot: serialize mid-cycle state to persistent store | [x] | `task/T1.5` | Pause signal, restart pod, resume signal → workflow continues from exact point |
|
||
| T1.6 | Comprehensive integration tests: multi-pod concurrency, network flakiness simulation | [x] | `task/T1.6` | Concurrent orchestrator instances on shared repo pass e2e without conflicts |
|
||
| T1.7 | Audit logging: all planner decisions, judge verdicts, implementer changes logged immutably | [x] | `task/T1.7` | Audit log persists across workflow restarts, queryable by task/timestamp |
|
||
| T1.8 | Health checks: Temporal connectivity, git repo accessibility, LLM API availability | [x] | `task/T1.8` | Periodic health probes, liveness/readiness endpoints for K8s |
|
||
|
||
---
|
||
|
||
## Submission Criteria
|
||
All T1.1–T1.8 marked `[x]` → submilestone complete → squash-merge `task/T1.*` to main.
|