73 lines
1.8 KiB
Markdown
73 lines
1.8 KiB
Markdown
# M3.8.3 — Compression Benchmarks & Tuning
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Phase | M3.8 — Context optimization |
|
|
| Size | M — 1 day |
|
|
| Status | ⬜ Not started |
|
|
| Depends | M3.8.1, M3.8.2 |
|
|
| Blocks | M3.8.4 |
|
|
|
|
## Goal
|
|
|
|
Measure compression performance across content types and validate that ratios
|
|
meet targets without sacrificing quality.
|
|
|
|
## Deliverables
|
|
|
|
### 1. Benchmark Suite
|
|
|
|
New module: `crates/mem-core/src/optimizer/bench.rs` (100 LOC)
|
|
|
|
```rust
|
|
pub fn benchmark_all_compressors() -> BenchmarkReport {
|
|
// Real-world test fixtures:
|
|
// - logs/npm-error.txt (5KB)
|
|
// - logs/cargo-fail.txt (8KB)
|
|
// - json/array-100.json (15KB)
|
|
// - diff/patch-large.diff (10KB)
|
|
// - text/prose-1000words.txt (6KB)
|
|
|
|
// Measure per-compressor:
|
|
// - compression ratio (%)
|
|
// - time taken (µs)
|
|
// - tokens before/after
|
|
}
|
|
```
|
|
|
|
Tests (4):
|
|
- `test_log_compression_meets_target` (85-95%)
|
|
- `test_json_compression_meets_target` (70-90%)
|
|
- `test_diff_compression_meets_target` (60-80%)
|
|
- `test_text_compression_meets_target` (30-50%)
|
|
|
|
### 2. Performance Profile
|
|
|
|
Command:
|
|
```bash
|
|
cargo test --release --lib optimizer::bench 2>&1 | grep "time:"
|
|
```
|
|
|
|
Expected output:
|
|
```
|
|
log compression: 89% ratio, 1.2ms
|
|
json compression: 78% ratio, 2.1ms
|
|
diff compression: 64% ratio, 1.5ms
|
|
text compression: 38% ratio, 1.8ms
|
|
```
|
|
|
|
### 3. Tuning Knobs
|
|
|
|
Document per-compressor parameters:
|
|
- LogCompressor: error line threshold (currently: any line with "error", "failed", etc.)
|
|
- JsonCrusher: importance budget (currently: 55%)
|
|
- DiffCompressor: context lines kept (currently: 0)
|
|
- TextCompressor: token retention ratio (currently: 40%)
|
|
|
|
## Acceptance
|
|
|
|
- All 4 compression targets met (measured >= target)
|
|
- All benchmark tests passing
|
|
- Performance < 3ms per chunk
|
|
- Documentation of tuning parameters
|