Story Crater Bot
ecd8f510f3
M3.8.3 ✅ COMPLETE (7 tests) - MetricsCollector: per-project aggregation - Structured logging (tracing) - Prometheus export format M3.8.4 ✅ IMPLICIT (no work needed) - Query path already clean (no compression) - Only cache_metrics() uses optimizer (for observability) M3.8.5 ⏳ ACTIVE (16 tests spec'd) - Compression ratio benchmarks (5 tests: log/json/text/diff/mixed) - Search quality validation (8 tests: pgvector/opensearch/fusion) - Performance baseline (3 tests: latency/throughput/memory) M3.8.6 ⏳ PENDING (13 gate assertions) - Safety (6): no data loss, deterministic, structure preservation - Performance (4): latency p99 <3ms, throughput 1000+/sec, memory <100MB - Quality (3): compression targets, search improvement, cache accuracy Project progress: 64/78 complete (82%), 8/13 gates green Total M3.8 tests: 103 (62+5+7+0+16+13)
2.2 KiB
2.2 KiB
M3.8.3 — Metrics & Monitoring (UPDATED)
| Field | Value |
|---|---|
| Phase | M3.8 — Context optimization |
| Size | M — 1 day |
| Status | ✅ COMPLETE |
| Depends | M3.8.1, M3.8.2 |
| Blocks | M3.8.4 |
Deliverables Completed
✅ MetricsCollector (7 tests)
- Per-project aggregation of OptimizationMetrics
- Merge metrics from multiple optimization runs
- Structured logging via tracing
- Prometheus-compatible export format
✅ Test Coverage (7 tests)
test_collector_merge_single_project— Store and retrieve per-project metricstest_collector_merge_multiple_projects— Aggregate across projectstest_collector_merge_aggregates— Multiple runs per projecttest_collector_nonexistent_project— Handle missing project gracefullytest_collector_per_compressor_stats— Track per-compressor breakdownstest_prometheus_export_format— Valid Prometheus text formattest_prometheus_compression_ratio— Correct ratio calculations
✅ Prometheus Export
- Counter:
m3_8_optimization_records_total - Gauge:
m3_8_optimization_input_bytes_total - Gauge:
m3_8_optimization_output_bytes_total - Gauge:
m3_8_optimization_compression_ratio - Per-compressor stats with labels
Integration Pattern
use mem_ingest::{MetricsCollector, optimize_record_with_metrics, OptimizationMetrics};
use std::sync::{Arc, Mutex};
let optimizer = ContextOptimizer::from_env()?;
let collector = MetricsCollector::new();
for project_id in projects {
let metrics = Arc::new(Mutex::new(OptimizationMetrics::default()));
for record in source.records() {
let optimized = optimize_record_with_metrics(
record,
&optimizer,
&metrics,
)?;
embed_and_index(&optimized)?;
}
let final_metrics = metrics.lock().unwrap().clone();
collector.merge_project(project_id, final_metrics);
}
// Log summary
collector.log_all_projects();
// Export for Prometheus
let prometheus_text = collector.prometheus_export();
http_server.register_metrics_endpoint("/metrics", prometheus_text);
Status
✅ Complete and Production-Ready
- All 7 tests passing
- Prometheus format validated
- Ready to integrate into rebuild pipeline