plan: M3.8.3 benchmarks + M3.8.4 gate

tasks/M3.8.3-compression-benchmarks.md

@@ -0,0 +1,72 @@
+# M3.8.3 — Compression Benchmarks & Tuning
+
+| Field | Value |
+|---|---|
+| Phase | M3.8 — Context optimization |
+| Size | M — 1 day |
+| Status | ⬜ Not started |
+| Depends | M3.8.1, M3.8.2 |
+| Blocks | M3.8.4 |
+
+## Goal
+
+Measure compression performance across content types and validate that ratios
+meet targets without sacrificing quality.
+
+## Deliverables
+
+### 1. Benchmark Suite
+
+New module: `crates/mem-core/src/optimizer/bench.rs` (100 LOC)
+
+```rust
+pub fn benchmark_all_compressors() -> BenchmarkReport {
+    // Real-world test fixtures:
+    // - logs/npm-error.txt (5KB)
+    // - logs/cargo-fail.txt (8KB)
+    // - json/array-100.json (15KB)
+    // - diff/patch-large.diff (10KB)
+    // - text/prose-1000words.txt (6KB)
+    
+    // Measure per-compressor:
+    // - compression ratio (%)
+    // - time taken (µs)
+    // - tokens before/after
+}
+```
+
+Tests (4):
+- `test_log_compression_meets_target` (85-95%)
+- `test_json_compression_meets_target` (70-90%)
+- `test_diff_compression_meets_target` (60-80%)
+- `test_text_compression_meets_target` (30-50%)
+
+### 2. Performance Profile
+
+Command:
+```bash
+cargo test --release --lib optimizer::bench 2>&1 | grep "time:"
+```
+
+Expected output:
+```
+log compression: 89% ratio, 1.2ms
+json compression: 78% ratio, 2.1ms
+diff compression: 64% ratio, 1.5ms
+text compression: 38% ratio, 1.8ms
+```
+
+### 3. Tuning Knobs
+
+Document per-compressor parameters:
+- LogCompressor: error line threshold (currently: any line with "error", "failed", etc.)
+- JsonCrusher: importance budget (currently: 55%)
+- DiffCompressor: context lines kept (currently: 0)
+- TextCompressor: token retention ratio (currently: 40%)
+
+## Acceptance
+
+- All 4 compression targets met (measured >= target)
+- All benchmark tests passing
+- Performance < 3ms per chunk
+- Documentation of tuning parameters

tasks/M3.8.4-m3.8-gate.md

@@ -0,0 +1,75 @@
+# M3.8.4 — M3.8 Composition Gate
+
+| Field | Value |
+|---|---|
+| Phase | M3.8 — Context optimization |
+| Size | M — 1 day |
+| Status | ⬜ Not started |
+| Depends | M3.8.3 (benchmarks) |
+| Blocks | M3.9 |
+
+## Goal
+
+Verify M3.8 implementation meets safety and performance constraints across
+realistic failure scenarios.
+
+## Gate Assertions
+
+### Safety (6 assertions)
+1. **No data loss** — CCR store retrieves 100% of compressed content
+2. **Cache correctness** — Drift metric accurate (< 5% error vs. actual)
+3. **Compression lossless** — Decompressed == original (where applicable)
+4. **Format stability** — JSON/diff/log structures preserved
+5. **Error graceful** — Optimizer failure doesn't crash pipeline
+6. **Thread-safe** — Concurrent optimizations don't corrupt state
+
+### Performance (4 assertions)
+1. **Latency** — Per-chunk optimization < 3ms (p99)
+2. **Cache hit rate** — Stable prefix unchanged >= 70% across queries
+3. **Throughput** — 1000 chunks/sec sustained
+4. **Memory** — Cache size stays < 100MB (max 1000 entries @ 100KB each)
+
+### Quality (3 assertions)
+1. **Compression met** — All 4 content types meet targets
+2. **No false positives** — Cache eligible when drift < 0.3
+3. **Coverage** — Benchmarks > 95% compression code paths
+
+## Test Implementation
+
+File: `tests/it_m3_8_gate.rs` (200 LOC)
+
+```rust
+#[test]
+fn m3_8_gate_no_data_loss() { ... }
+
+#[test]
+fn m3_8_gate_cache_correctness() { ... }
+
+#[test]
+fn m3_8_gate_compression_targets() { ... }
+
+#[test]
+fn m3_8_gate_latency_p99() { ... }
+
+#[test]
+fn m3_8_gate_concurrent_safety() { ... }
+
+#[test]
+fn m3_8_gate_cache_hit_rate() { ... }
+```
+
+Tests: 6 (safety) + 4 (performance) + 3 (quality) = 13 total
+
+## Acceptance
+
+✅ All 13 gate assertions passing
+✅ 117+ optimizer unit tests passing
+✅ Benchmarks meeting compression targets
+✅ No regressions in other modules (prompt, query, etc.)
+✅ Documentation complete (CONTEXT_OPTIMIZER.md, headers in code)
+
+## Success Criteria
+
+- M3.8.4 gate 100% pass rate
+- Zero blocking issues from integration tests
+- Ready for M3.8 → production hand-off