diff --git a/tasks/M3.8.3-compression-benchmarks.md b/tasks/M3.8.3-compression-benchmarks.md new file mode 100644 index 0000000..73f655d --- /dev/null +++ b/tasks/M3.8.3-compression-benchmarks.md @@ -0,0 +1,72 @@ +# M3.8.3 — Compression Benchmarks & Tuning + +| Field | Value | +|---|---| +| Phase | M3.8 — Context optimization | +| Size | M — 1 day | +| Status | ⬜ Not started | +| Depends | M3.8.1, M3.8.2 | +| Blocks | M3.8.4 | + +## Goal + +Measure compression performance across content types and validate that ratios +meet targets without sacrificing quality. + +## Deliverables + +### 1. Benchmark Suite + +New module: `crates/mem-core/src/optimizer/bench.rs` (100 LOC) + +```rust +pub fn benchmark_all_compressors() -> BenchmarkReport { + // Real-world test fixtures: + // - logs/npm-error.txt (5KB) + // - logs/cargo-fail.txt (8KB) + // - json/array-100.json (15KB) + // - diff/patch-large.diff (10KB) + // - text/prose-1000words.txt (6KB) + + // Measure per-compressor: + // - compression ratio (%) + // - time taken (µs) + // - tokens before/after +} +``` + +Tests (4): +- `test_log_compression_meets_target` (85-95%) +- `test_json_compression_meets_target` (70-90%) +- `test_diff_compression_meets_target` (60-80%) +- `test_text_compression_meets_target` (30-50%) + +### 2. Performance Profile + +Command: +```bash +cargo test --release --lib optimizer::bench 2>&1 | grep "time:" +``` + +Expected output: +``` +log compression: 89% ratio, 1.2ms +json compression: 78% ratio, 2.1ms +diff compression: 64% ratio, 1.5ms +text compression: 38% ratio, 1.8ms +``` + +### 3. Tuning Knobs + +Document per-compressor parameters: +- LogCompressor: error line threshold (currently: any line with "error", "failed", etc.) +- JsonCrusher: importance budget (currently: 55%) +- DiffCompressor: context lines kept (currently: 0) +- TextCompressor: token retention ratio (currently: 40%) + +## Acceptance + +- All 4 compression targets met (measured >= target) +- All benchmark tests passing +- Performance < 3ms per chunk +- Documentation of tuning parameters diff --git a/tasks/M3.8.4-m3.8-gate.md b/tasks/M3.8.4-m3.8-gate.md new file mode 100644 index 0000000..e55c843 --- /dev/null +++ b/tasks/M3.8.4-m3.8-gate.md @@ -0,0 +1,75 @@ +# M3.8.4 — M3.8 Composition Gate + +| Field | Value | +|---|---| +| Phase | M3.8 — Context optimization | +| Size | M — 1 day | +| Status | ⬜ Not started | +| Depends | M3.8.3 (benchmarks) | +| Blocks | M3.9 | + +## Goal + +Verify M3.8 implementation meets safety and performance constraints across +realistic failure scenarios. + +## Gate Assertions + +### Safety (6 assertions) +1. **No data loss** — CCR store retrieves 100% of compressed content +2. **Cache correctness** — Drift metric accurate (< 5% error vs. actual) +3. **Compression lossless** — Decompressed == original (where applicable) +4. **Format stability** — JSON/diff/log structures preserved +5. **Error graceful** — Optimizer failure doesn't crash pipeline +6. **Thread-safe** — Concurrent optimizations don't corrupt state + +### Performance (4 assertions) +1. **Latency** — Per-chunk optimization < 3ms (p99) +2. **Cache hit rate** — Stable prefix unchanged >= 70% across queries +3. **Throughput** — 1000 chunks/sec sustained +4. **Memory** — Cache size stays < 100MB (max 1000 entries @ 100KB each) + +### Quality (3 assertions) +1. **Compression met** — All 4 content types meet targets +2. **No false positives** — Cache eligible when drift < 0.3 +3. **Coverage** — Benchmarks > 95% compression code paths + +## Test Implementation + +File: `tests/it_m3_8_gate.rs` (200 LOC) + +```rust +#[test] +fn m3_8_gate_no_data_loss() { ... } + +#[test] +fn m3_8_gate_cache_correctness() { ... } + +#[test] +fn m3_8_gate_compression_targets() { ... } + +#[test] +fn m3_8_gate_latency_p99() { ... } + +#[test] +fn m3_8_gate_concurrent_safety() { ... } + +#[test] +fn m3_8_gate_cache_hit_rate() { ... } +``` + +Tests: 6 (safety) + 4 (performance) + 3 (quality) = 13 total + +## Acceptance + +✅ All 13 gate assertions passing +✅ 117+ optimizer unit tests passing +✅ Benchmarks meeting compression targets +✅ No regressions in other modules (prompt, query, etc.) +✅ Documentation complete (CONTEXT_OPTIMIZER.md, headers in code) + +## Success Criteria + +- M3.8.4 gate 100% pass rate +- Zero blocking issues from integration tests +- Ready for M3.8 → production hand-off