CacheMetrics struct (40 LOC): - stable_prefix_bytes, dynamic_tail_bytes - drift_metric (0.0-1.0 ratio) - cache_eligible flag (drift < 0.3) - compression_ratio() and header_* methods PromptBuilder::cache_metrics() (40 LOC): - Calculates cache alignment metrics for query+chunk pairs - Integrates CacheAligner output - Gets compression ratio from ContextOptimizer - Used for HTTP headers and observability HTTP headers ready for client integration: - X-Cache-Stable-Bytes - X-Cache-Drift - X-Cache-Eligible - X-Compression-Ratio 3 new tests: - test_cache_metrics_stable_query - test_cache_metrics_compression_ratio - test_cache_metrics_header_drift Total: 117 mem-core tests (114 before + 3 new)
1.9 KiB
1.9 KiB
M3.8.2 — CacheAligner: HTTP Headers + Metrics
| Field | Value |
|---|---|
| Phase | M3.8 — Context optimization |
| Size | M — 1–2 days |
| Status | ⬜ Not started |
| Depends | M3.8.1 (CacheAligner complete) |
| Blocks | M3.8.3 |
Goal
Integrate CacheAligner output into HTTP response headers and observability metrics so that:
- LLM provider can use cache hints
- Monitoring can track cache effectiveness
- Debugging can identify cache misses
Deliverables
1. PromptBuilder::cache_metrics()
New method returning cache metadata:
pub struct CacheMetrics {
pub stable_prefix_bytes: usize,
pub dynamic_tail_bytes: usize,
pub drift_metric: f32, // 0.0-1.0 ratio
pub cache_eligible: bool, // true if drift < 0.3
}
impl PromptBuilder {
pub fn cache_metrics(query: &Query, chunk: &Chunk) -> Result<CacheMetrics>
}
Tests (3):
test_cache_metrics_stable_querytest_cache_metrics_high_drifttest_cache_metrics_zero_drift
2. HTTP Response Headers
Add to PromptBuilder output:
X-Cache-Stable-Bytes: size of cacheable prefixX-Cache-Drift: 0.0-1.0 ratioX-Cache-Eligible: "true"/"false"X-Compression-Ratio: original vs. compressed
Tests (4):
test_headers_present_in_responsetest_headers_accurate_valuestest_headers_skipped_when_disabledtest_headers_format_valid
3. Observability Hooks
Integrate with logging:
pub fn log_cache_metrics(metrics: &CacheMetrics) {
tracing::info!(
stable_bytes = metrics.stable_prefix_bytes,
drift = metrics.drift_metric,
eligible = metrics.cache_eligible,
"cache_alignment"
);
}
Tests (2):
test_metrics_logged_on_alignmenttest_drift_high_triggers_warning
Acceptance
- All 9 new tests passing
- Existing 114 mem-core tests still pass
- Cache metrics accurately reflect alignment
- HTTP headers present and valid
- Zero performance overhead (< 1ms additional)