feat(T1.4): implement board state validation and auto-healing
- Add internal/board package with validation and state tracking - Implement BoardValidator for comprehensive board file validation - Detect missing headers, malformed tables, invalid task IDs - Validate status fields ([x] or [ ]) - Parse task information from valid boards - Implement StateTracker for actual task state management - Track task progression (pending → in_progress → completed/failed) - Support task metrics attachment and analytics - Implement divergence detection: compare board vs actual states - Implement auto-healing: fix state mismatches between board and reality - RepairBoard() fixes structural corruption issues - HealDivergence() updates board to match actual states - Support both JSON persistence and in-memory operation Validation Features: - Detailed error reporting with line numbers and context - Warning system for suspicious but valid boards - Task ID format validation (T#.# pattern) - Status value normalization ([X] → [x]) - Table structure verification State Management: - Persistent JSON storage of task states - Completion/failure timestamps - Custom metrics per task - Thread-safe RWMutex synchronization - Stats and filtering operations Healing Features: - Non-destructive repairs (report changes) - Board integrity preservation - Divergence detection with timestamps - Batch update capability - Change tracking for audit trail Test Coverage: - 13 validator tests (structure, validation, repair, parsing) - 16 state tracker tests (tracking, persistence, analytics) - 29 total board tests, all passing - Edge cases: empty boards, invalid formats, multiple tasks - Multi-state transitions and metrics Key Design: - Separation of concerns: Validator (format) vs Tracker (state) - JSON persistence (human-readable, debuggable) - Thread-safe concurrent state updates - Detailed error messages with context - Non-breaking repairs (safe by default) Closes T1.4
This commit is contained in:
Test
+443
@@ -0,0 +1,443 @@
|
||||
# T1.4: Board State Validation & Auto-Healing
|
||||
|
||||
**Submilestone:** T1 (Production Hardening)
|
||||
**Status:** ✅ COMPLETE
|
||||
**Branch:** `task/T1.4`
|
||||
|
||||
## Overview
|
||||
|
||||
Implement comprehensive board file validation and automatic corruption recovery to detect and fix inconsistencies between board file state and actual workflow state, preventing manual intervention and ensuring data integrity.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Board Validation
|
||||
|
||||
- Validate markdown structure (headers, table format)
|
||||
- Check task ID format (T1.1, T1.2, etc.)
|
||||
- Validate status fields ([x] or [ ])
|
||||
- Detect malformed rows and missing columns
|
||||
- Generate detailed error and warning reports
|
||||
- Parse task information from valid boards
|
||||
|
||||
### Corruption Detection
|
||||
|
||||
- Detect divergence between board file and actual task states
|
||||
- Track state mismatches (expected vs actual)
|
||||
- Support timestamp-based divergence tracking
|
||||
- Identify missing or invalid task entries
|
||||
|
||||
### Auto-Healing
|
||||
|
||||
- Repair missing markdown headers
|
||||
- Fix malformed status values
|
||||
- Add missing table separators
|
||||
- Correct invalid task IDs
|
||||
- Heal divergences by syncing board with actual states
|
||||
- Preserve task information during repairs
|
||||
|
||||
### State Tracking
|
||||
|
||||
- Persist actual task states to JSON
|
||||
- Track task progression (pending → in_progress → completed/failed)
|
||||
- Store task metrics alongside state
|
||||
- Support multi-task concurrent state updates
|
||||
- Generate statistics and completion reports
|
||||
|
||||
## Implementation
|
||||
|
||||
### Internal Package: `internal/board`
|
||||
|
||||
#### `validator.go`
|
||||
- `BoardValidationError` - Validation error with type, message, line number
|
||||
- `BoardValidator` - Core validation and healing engine
|
||||
- `TaskRow` - Parsed task from board file
|
||||
- Methods:
|
||||
- `ValidateBoard()` - Full board structure validation
|
||||
- `ParseTasks()` - Extract tasks from valid boards
|
||||
- `DetectDivergence()` - Find state mismatches
|
||||
- `HealDivergence()` - Auto-fix state mismatches
|
||||
- `RepairBoard()` - Fix structural issues
|
||||
- Error/warning tracking and reporting
|
||||
- 13/13 unit tests passing ✅
|
||||
|
||||
#### `state.go`
|
||||
- `TaskState` - Actual task state (status, completion time, metrics)
|
||||
- `StateTracker` - Manage actual task states
|
||||
- Methods:
|
||||
- `UpdateTaskState()` - Record task status change
|
||||
- `GetTaskState()` / `GetAllStates()` - Retrieve states
|
||||
- `GetCompletedTasks()` / `GetFailedTasks()` / `GetPendingTasks()` - Filter by status
|
||||
- `AddMetric()` - Attach metrics to tasks
|
||||
- `GetAsCompletionMap()` - Boolean map for comparison
|
||||
- `GetStats()` / `GetLastUpdate()` - Analytics
|
||||
- `Load()` - Persistence from JSON
|
||||
- `Reset()` - Clear all state
|
||||
- 16/16 unit tests passing ✅
|
||||
|
||||
#### Unit Tests: `*_test.go`
|
||||
- 29 tests total, all passing ✅
|
||||
- Validator: parsing, validation, repair, divergence detection/healing
|
||||
- State: tracking, filtering, persistence, metrics
|
||||
- Integration: multi-task scenarios, state transitions
|
||||
|
||||
## Key Features
|
||||
|
||||
### Validation Pipeline
|
||||
|
||||
```
|
||||
Board File Content
|
||||
↓
|
||||
[Check Structure]
|
||||
├─ Has title header
|
||||
├─ Has table separator
|
||||
└─ Has task rows
|
||||
↓
|
||||
[Validate Each Task]
|
||||
├─ Valid task ID format (T#.# or T#)
|
||||
├─ Valid status ([x] or [ ])
|
||||
├─ No missing columns
|
||||
└─ Reasonable description
|
||||
↓
|
||||
[Report Results]
|
||||
├─ Errors (validation failed)
|
||||
└─ Warnings (suspicious but valid)
|
||||
```
|
||||
|
||||
### Corruption Healing
|
||||
|
||||
```go
|
||||
// Board has T1.1, T1.2, T1.3, T1.4
|
||||
// Actual states: T1.1=done, T1.2=done, T1.3=pending, T1.4=done
|
||||
// Board shows: T1.1=done, T1.2=pending, T1.3=pending, T1.4=pending
|
||||
|
||||
actualStates := map[string]bool{
|
||||
"T1.1": true, "T1.2": true,
|
||||
"T1.3": false, "T1.4": true,
|
||||
}
|
||||
|
||||
divergences := validator.DetectDivergence(boardContent, actualStates)
|
||||
// Finds: T1.2 (expected false, actual true), T1.4 (expected false, actual true)
|
||||
|
||||
healed, changes := validator.HealDivergence(boardContent, actualStates)
|
||||
// Fixes: Updates T1.2 and T1.4 status in board file
|
||||
// Changes: ["Fixed T1.2: [ ] → [x]", "Fixed T1.4: [ ] → [x]"]
|
||||
```
|
||||
|
||||
### State Tracking
|
||||
|
||||
```go
|
||||
// Initialize state tracker
|
||||
tracker := NewStateTracker("/var/poimen")
|
||||
|
||||
// Record task progress
|
||||
tracker.UpdateTaskState("T1.1", "in_progress", "task/T1.1", nil)
|
||||
tracker.AddMetric("T1.1", "lines_changed", 1247)
|
||||
tracker.AddMetric("T1.1", "files_modified", 15)
|
||||
|
||||
// Later, task completes
|
||||
tracker.UpdateTaskState("T1.1", "completed", "task/T1.1", nil)
|
||||
|
||||
// Query states
|
||||
completed := tracker.GetCompletedTasks() // ["T1.1", ...]
|
||||
stats := tracker.GetStats()
|
||||
// {"total": 4, "counts": {"completed": 1, "pending": 3}}
|
||||
|
||||
// Persist and recover
|
||||
tracker.Load() // From disk
|
||||
```
|
||||
|
||||
### Board Repair Examples
|
||||
|
||||
```
|
||||
❌ BEFORE: Missing header
|
||||
| T1.1 | Task | [x] | branch | verify |
|
||||
|
||||
✅ AFTER: Header added
|
||||
# Task Board — Milestone T1: Production Hardening
|
||||
| T1.1 | Task | [x] | branch | verify |
|
||||
|
||||
---
|
||||
|
||||
❌ BEFORE: Invalid status
|
||||
| T1.1 | Task | [?] | branch | verify |
|
||||
|
||||
✅ AFTER: Normalized
|
||||
| T1.1 | Task | [ ] | branch | verify |
|
||||
|
||||
---
|
||||
|
||||
❌ BEFORE: Missing separator
|
||||
| ID | Scope | Status | Branch |
|
||||
| T1.1 | Task | [x] | branch |
|
||||
|
||||
✅ AFTER: Separator added
|
||||
| ID | Scope | Status | Branch |
|
||||
|----|-------|--------|--------|
|
||||
| T1.1 | Task | [x] | branch |
|
||||
```
|
||||
|
||||
## Verification Criteria
|
||||
|
||||
✅ **All criteria met:**
|
||||
|
||||
1. **Validation Engine**
|
||||
- Detects missing headers
|
||||
- Detects malformed tables
|
||||
- Validates task IDs
|
||||
- Validates status values
|
||||
- Reports errors and warnings
|
||||
- 13 tests passing
|
||||
|
||||
2. **Corruption Detection**
|
||||
- Identifies task divergences
|
||||
- Tracks expected vs actual states
|
||||
- Timestamps divergences
|
||||
- Handles missing tasks
|
||||
- 4 tests passing
|
||||
|
||||
3. **Auto-Healing**
|
||||
- Adds missing headers
|
||||
- Fixes invalid status values
|
||||
- Adds table separators
|
||||
- Repairs divergent states
|
||||
- Preserves data integrity
|
||||
- 3 tests passing
|
||||
|
||||
4. **State Management**
|
||||
- Tracks task progression
|
||||
- Stores completion timestamps
|
||||
- Records failure information
|
||||
- Supports metrics attachment
|
||||
- Persists state to disk
|
||||
- 16 tests passing
|
||||
|
||||
5. **Integration**
|
||||
- Works with actual board.md format
|
||||
- Compatible with validation/tracking
|
||||
- Supports concurrent updates
|
||||
- Thread-safe operations
|
||||
- 3 tests passing
|
||||
|
||||
6. **Test Coverage**
|
||||
- 29/29 board tests passing ✅
|
||||
- Edge cases covered
|
||||
- Persistence tested
|
||||
- Multi-task scenarios validated
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Unit tests
|
||||
go test -v ./internal/board
|
||||
# Result: PASS (29/29 tests)
|
||||
|
||||
# Full test suite
|
||||
go test -v ./...
|
||||
# Result: All tests pass
|
||||
|
||||
# Integration scenario
|
||||
validator := NewBoardValidator("repo/tasks")
|
||||
|
||||
// Validate board
|
||||
if !validator.ValidateBoard(boardContent) {
|
||||
errors := validator.GetErrors()
|
||||
// Fix: validator.RepairBoard(boardContent)
|
||||
}
|
||||
|
||||
// Parse tasks
|
||||
tasks, _ := validator.ParseTasks(boardContent)
|
||||
for _, task := range tasks {
|
||||
// Track actual state
|
||||
tracker.UpdateTaskState(task.ID, "completed", task.Branch, nil)
|
||||
}
|
||||
|
||||
// Detect divergence
|
||||
tracker.Load()
|
||||
actualStates := tracker.GetAsCompletionMap()
|
||||
divergences := validator.DetectDivergence(boardContent, actualStates)
|
||||
|
||||
// Heal if needed
|
||||
if len(divergences) > 0 {
|
||||
healed, changes := validator.HealDivergence(boardContent, actualStates)
|
||||
// Save healed board
|
||||
ioutil.WriteFile("tasks/board.md", []byte(healed), 0644)
|
||||
}
|
||||
```
|
||||
|
||||
## Kubernetes Integration
|
||||
|
||||
With board healing:
|
||||
|
||||
```yaml
|
||||
# Board state persisted in shared volume
|
||||
volumeMounts:
|
||||
- name: board
|
||||
mountPath: /var/poimen/board
|
||||
|
||||
# State accessible across pod restarts
|
||||
volumes:
|
||||
- name: board
|
||||
persistentVolumeClaim:
|
||||
claimName: poimen-board
|
||||
|
||||
# Liveness check includes board validation
|
||||
livenessProbe:
|
||||
exec:
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- |
|
||||
validator validate /var/poimen/board/board.md || exit 1
|
||||
```
|
||||
|
||||
## Configuration Example
|
||||
|
||||
```go
|
||||
// Initialize validator and tracker
|
||||
validator := NewBoardValidator("/var/poimen/board")
|
||||
tracker := NewStateTracker("/var/poimen")
|
||||
|
||||
// Load existing state from previous run
|
||||
if err := tracker.Load(); err != nil {
|
||||
log.Printf("Warning: could not load previous state: %v", err)
|
||||
}
|
||||
|
||||
// During workflow execution
|
||||
boardContent, _ := ioutil.ReadFile("/var/poimen/board/board.md")
|
||||
|
||||
// Validate board
|
||||
if !validator.ValidateBoard(string(boardContent)) {
|
||||
log.Printf("Board validation errors: %s", validator.ErrorSummary())
|
||||
|
||||
// Attempt repair
|
||||
repaired, _ := validator.RepairBoard(string(boardContent))
|
||||
ioutil.WriteFile("/var/poimen/board/board.md", []byte(repaired), 0644)
|
||||
}
|
||||
|
||||
// Track task progress
|
||||
for _, taskID := range tasksToRun {
|
||||
tracker.UpdateTaskState(taskID, "in_progress", fmt.Sprintf("task/%s", taskID), nil)
|
||||
|
||||
// ... execute task ...
|
||||
|
||||
if taskSuccess {
|
||||
tracker.UpdateTaskState(taskID, "completed", fmt.Sprintf("task/%s", taskID), nil)
|
||||
} else {
|
||||
tracker.UpdateTaskState(taskID, "failed", fmt.Sprintf("task/%s", taskID), taskErr)
|
||||
}
|
||||
}
|
||||
|
||||
// Detect and heal divergence
|
||||
actualStates := tracker.GetAsCompletionMap()
|
||||
divergences := validator.DetectDivergence(string(boardContent), actualStates)
|
||||
|
||||
if len(divergences) > 0 {
|
||||
log.Printf("Detected %d divergences, healing...", len(divergences))
|
||||
healed, changes := validator.HealDivergence(string(boardContent), actualStates)
|
||||
|
||||
for _, change := range changes {
|
||||
log.Printf("Fixed: %s", change)
|
||||
}
|
||||
|
||||
ioutil.WriteFile("/var/poimen/board/board.md", []byte(healed), 0644)
|
||||
}
|
||||
|
||||
// Persist state for next run
|
||||
_ = tracker.Load()
|
||||
```
|
||||
|
||||
## Validation Algorithm
|
||||
|
||||
```
|
||||
Board Validation
|
||||
↓
|
||||
[1] Check Presence
|
||||
├─ Has markdown header ("#")
|
||||
└─ Has table separator ("---")
|
||||
↓
|
||||
[2] Find Task Table
|
||||
├─ Locate header row (| ID | ... |)
|
||||
├─ Skip separator
|
||||
└─ Find first data row
|
||||
↓
|
||||
[3] Validate Each Row
|
||||
├─ Check column count
|
||||
├─ Validate task ID (T#.# format)
|
||||
├─ Validate status ([x] or [ ])
|
||||
└─ Warn on missing/empty fields
|
||||
↓
|
||||
[4] Generate Report
|
||||
├─ Collect all errors
|
||||
├─ Collect all warnings
|
||||
└─ Return validation result (pass/fail)
|
||||
```
|
||||
|
||||
## Healing Algorithm
|
||||
|
||||
```
|
||||
Divergence Healing
|
||||
↓
|
||||
[1] Compare States
|
||||
├─ Board expected: [x] or [ ]
|
||||
└─ Actual state: true or false
|
||||
↓
|
||||
[2] Find Mismatches
|
||||
├─ Board ≠ Actual: need fix
|
||||
└─ Board = Actual: OK
|
||||
↓
|
||||
[3] Update Board
|
||||
├─ Replace [x] with [ ] or vice versa
|
||||
├─ Track changes made
|
||||
└─ Preserve all other fields
|
||||
↓
|
||||
[4] Report Changes
|
||||
├─ List updated tasks
|
||||
├─ Show old → new status
|
||||
└─ Ready to write to disk
|
||||
```
|
||||
|
||||
## Files Changed
|
||||
|
||||
- ✅ `internal/board/validator.go` - Board validation and healing (378 lines)
|
||||
- ✅ `internal/board/validator_test.go` - Validator tests (224 lines)
|
||||
- ✅ `internal/board/state.go` - State tracking (195 lines)
|
||||
- ✅ `internal/board/state_test.go` - State tests (229 lines)
|
||||
- ✅ `tasks/board-T1.md` - Task board update
|
||||
|
||||
## Dependencies
|
||||
|
||||
All internal, no new external dependencies added.
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
1. **Separate Validator & Tracker** - Validation (format) vs State (semantics)
|
||||
2. **JSON Persistence** - Human-readable, easy to inspect/debug
|
||||
3. **Non-destructive Repairs** - Try to fix, report changes, allow rollback
|
||||
4. **Detailed Error Reporting** - Line numbers, context, suggestions
|
||||
5. **Thread-Safe State** - RWMutex for concurrent access
|
||||
6. **Status Normalization** - [X] → [x] for consistency
|
||||
|
||||
## Future Extensions
|
||||
|
||||
- Git integration: auto-commit healed boards
|
||||
- Webhook notifications on divergence
|
||||
- Historical divergence tracking
|
||||
- Predictive healing (forecast issues)
|
||||
- Multi-branch board tracking
|
||||
- Board diffs and change logs
|
||||
|
||||
## Next Steps (T1.5 → T1.6 → T1.7)
|
||||
|
||||
1. **T1.5:** Workflow pause/resume with state snapshots
|
||||
2. **T1.6:** Comprehensive integration tests for concurrency
|
||||
3. **T1.7:** Audit logging (immutable decision log)
|
||||
|
||||
## Notes
|
||||
|
||||
- Board must have at least header and one task row
|
||||
- Task IDs must match format: T# or T#.#
|
||||
- Status values are case-insensitive during repair ([X] becomes [x])
|
||||
- Validation reports are detailed and actionable
|
||||
- State tracking is optional (validator works standalone)
|
||||
- Both validator and tracker are thread-safe
|
||||
- Perfect for container/K8s environments with restart policies
|
||||
Reference in New Issue
Block a user