docs: update M3.8 task specs (M3.8.3-6 detailed)
M3.8.3 ✅ COMPLETE (7 tests) - MetricsCollector: per-project aggregation - Structured logging (tracing) - Prometheus export format M3.8.4 ✅ IMPLICIT (no work needed) - Query path already clean (no compression) - Only cache_metrics() uses optimizer (for observability) M3.8.5 ⏳ ACTIVE (16 tests spec'd) - Compression ratio benchmarks (5 tests: log/json/text/diff/mixed) - Search quality validation (8 tests: pgvector/opensearch/fusion) - Performance baseline (3 tests: latency/throughput/memory) M3.8.6 ⏳ PENDING (13 gate assertions) - Safety (6): no data loss, deterministic, structure preservation - Performance (4): latency p99 <3ms, throughput 1000+/sec, memory <100MB - Quality (3): compression targets, search improvement, cache accuracy Project progress: 64/78 complete (82%), 8/13 gates green Total M3.8 tests: 103 (62+5+7+0+16+13)
This commit is contained in:
@@ -0,0 +1,171 @@
|
||||
# M3.8.6 — M3.8 Composition Gate
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | M3.8 — Context optimization |
|
||||
| Size | M — 1 day |
|
||||
| Status | ⬜ Not started |
|
||||
| Depends | M3.8.5 (benchmarks) |
|
||||
| Blocks | Production deployment |
|
||||
|
||||
## Goal
|
||||
|
||||
Verify M3.8 implementation meets all safety, performance, and quality constraints
|
||||
before production rollout.
|
||||
|
||||
## Gate Assertions
|
||||
|
||||
### Safety (6 assertions)
|
||||
|
||||
1. **No data loss** — Optimized chunks preserve all semantic content
|
||||
```rust
|
||||
assert!(semantic_similarity(original, optimized) > 0.95);
|
||||
```
|
||||
|
||||
2. **Deterministic output** — Same input always produces same output
|
||||
```rust
|
||||
assert_eq!(optimize(text), optimize(text));
|
||||
```
|
||||
|
||||
3. **Structure preservation** — JSON/logs remain parseable
|
||||
```rust
|
||||
assert!(parse_json(&optimized).is_ok());
|
||||
assert!(grep_logs(&optimized).count() > 0);
|
||||
```
|
||||
|
||||
4. **Metadata preserved** — Breadcrumb, role, provenance untouched
|
||||
```rust
|
||||
assert_eq!(original.provenance, optimized.provenance);
|
||||
assert_eq!(original.breadcrumb, optimized.breadcrumb);
|
||||
```
|
||||
|
||||
5. **Error handling** — Graceful fallback on optimization failure
|
||||
```rust
|
||||
assert!(optimize_with_fallback(bad_input).is_ok());
|
||||
```
|
||||
|
||||
6. **Thread safety** — Concurrent optimization doesn't corrupt state
|
||||
```rust
|
||||
assert!(concurrent_optimize(1000).all_ok());
|
||||
```
|
||||
|
||||
### Performance (4 assertions)
|
||||
|
||||
1. **Latency** — Per-record optimization <3ms p99
|
||||
```rust
|
||||
assert!(latency_p99() < Duration::from_millis(3));
|
||||
```
|
||||
|
||||
2. **Throughput** — Sustained 1000+ records/sec
|
||||
```rust
|
||||
assert!(throughput_records_per_sec() >= 1000);
|
||||
```
|
||||
|
||||
3. **Memory** — Cache stays <100MB (max 1000 entries)
|
||||
```rust
|
||||
assert!(cache_size_mb() < 100);
|
||||
```
|
||||
|
||||
4. **No regressions** — Existing tests still pass
|
||||
```rust
|
||||
assert!(all_prompt_tests_pass());
|
||||
assert!(all_ingest_tests_pass());
|
||||
```
|
||||
|
||||
### Quality (3 assertions)
|
||||
|
||||
1. **Compression targets met** — All content types
|
||||
```rust
|
||||
assert!(log_ratio >= 85.0 && log_ratio <= 95.0);
|
||||
assert!(json_ratio >= 70.0 && json_ratio <= 90.0);
|
||||
assert!(text_ratio >= 30.0 && text_ratio <= 50.0);
|
||||
```
|
||||
|
||||
2. **Search quality improves** — pgvector + OpenSearch
|
||||
```rust
|
||||
assert!(embedding_similarity > 0.95);
|
||||
assert!(opensearch_mrr_improvement > 10);
|
||||
```
|
||||
|
||||
3. **No false positives** — Cache eligibility accurate
|
||||
```rust
|
||||
assert!(drift_metric_accurate < 0.05); // <5% error
|
||||
```
|
||||
|
||||
## Test Implementation
|
||||
|
||||
File: `tests/it_m3_8_gate.rs` (400 LOC)
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn m3_8_gate_no_data_loss() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_deterministic() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_structure_preservation() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_metadata_preservation() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_error_handling() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_thread_safety() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_latency_p99() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_throughput_sustained() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_memory_bounded() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_no_regressions() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_compression_targets() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_search_quality() { ... }
|
||||
|
||||
#[test]
|
||||
fn m3_8_gate_cache_eligibility() { ... }
|
||||
```
|
||||
|
||||
Total: **13 gate assertions**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
✅ All 13 assertions passing
|
||||
✅ All 62 M3.8.1 optimizer tests passing
|
||||
✅ All 5 M3.8.2 ingest tests passing
|
||||
✅ All 7 M3.8.3 metrics tests passing
|
||||
✅ All 16 M3.8.5 benchmark tests passing
|
||||
✅ All 11 existing prompt tests passing
|
||||
✅ No regressions in other modules
|
||||
✅ Documentation complete
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- **Safety**: 6/6 assertions ✅
|
||||
- **Performance**: 4/4 assertions ✅
|
||||
- **Quality**: 3/3 assertions ✅
|
||||
- **Coverage**: 100% of compressors tested
|
||||
- **Documentation**: BENCHMARKS.md + GATE.md
|
||||
|
||||
## Timeline
|
||||
|
||||
- M3.8.1: ✅ Done (62 tests)
|
||||
- M3.8.2: ✅ Done (5 tests)
|
||||
- M3.8.3: ✅ Done (7 tests)
|
||||
- M3.8.4: ✅ Done (implicit, 0 tests)
|
||||
- M3.8.5: ⏳ In progress (16 tests)
|
||||
- M3.8.6: ⏳ Next (13 tests)
|
||||
|
||||
**Total M3.8**: 103 tests
|
||||
**Expected gate pass rate**: 100%
|
||||
Reference in New Issue
Block a user