Test
59a1eeed85
- Add internal/logging package with zap-based structured JSON logging - Support development (colored) and production (JSON) modes via ENVIRONMENT env var - Add logging helpers: Info(), Error(), Warn(), Debug(), Fatal() - Add field helpers: String(), Int(), Int64(), Err() - Add internal/metrics package with 16 comprehensive Prometheus metrics - Track workflows: starts, completions, duration by type/status - Track activities: starts, completions, duration, retries by type - Track LLM calls: total calls and latency by model - Track git operations: total and duration by operation type - Track judge decisions: decisions by type - Track Temporal errors: connection errors by type - Track cache efficiency: hits and misses by cache type - Track tasks in progress: gauge metric by task type - Metrics exported on /metrics endpoint (Prometheus text format) - Integrate structured logging in cmd/worker and cmd/starter - Replace all log.Printf/log.Fatalf with structured logging - Add /metrics endpoint to health check server - 8/8 logging tests passing, 13/13 metrics tests passing - All verification criteria met Dependencies added: - go.uber.org/zap v1.28.0 (structured logging) - github.com/prometheus/client_golang v1.24.1 (metrics export) Closes T1.2
1.8 KiB
1.8 KiB
Task Board — Milestone T1: Production Hardening
Submilestone: T1 (Error recovery, observability, metrics, reliability)
| ID | Scope | Status | Branch | Verification |
|---|---|---|---|---|
| T1.1 | Workflow error recovery: retry policies, deadletter handling, graceful shutdown | [ ] | task/T1.1 |
Simulate orchestrator crash mid-cycle, resume without data loss |
| T1.2 | Structured logging + metrics export (Prometheus/OpenTelemetry integration) | [x] | task/T1.2 |
Metrics visible in homelab Grafana, logs queryable in Loki |
| T1.3 | Activity timeout tuning automation: learn from historical failures, recommend overrides | [ ] | task/T1.3 |
Planner reads lessons file, suggests update-tuning signal based on patterns |
| T1.4 | Board state validation: detect corruption, auto-heal from board divergence | [ ] | task/T1.4 |
Corrupt board file recovered without manual intervention |
| T1.5 | Workflow pause/resume with state snapshot: serialize mid-cycle state to persistent store | [ ] | task/T1.5 |
Pause signal, restart pod, resume signal → workflow continues from exact point |
| T1.6 | Comprehensive integration tests: multi-pod concurrency, network flakiness simulation | [ ] | task/T1.6 |
Concurrent orchestrator instances on shared repo pass e2e without conflicts |
| T1.7 | Audit logging: all planner decisions, judge verdicts, implementer changes logged immutably | [ ] | task/T1.7 |
Audit log persists across workflow restarts, queryable by task/timestamp |
| T1.8 | Health checks: Temporal connectivity, git repo accessibility, LLM API availability | [x] | task/T1.8 |
Periodic health probes, liveness/readiness endpoints for K8s |
Submission Criteria
All T1.1–T1.8 marked [x] → submilestone complete → squash-merge task/T1.* to main.