2026-08-28 10:05:45 -07:00
|
|
|
|
# M3.8.2 — CacheAligner: HTTP Headers + Metrics
|
|
|
|
|
|
|
|
|
|
|
|
| Field | Value |
|
|
|
|
|
|
|---|---|
|
|
|
|
|
|
| Phase | M3.8 — Context optimization |
|
|
|
|
|
|
| Size | M — 1–2 days |
|
2026-08-28 10:05:55 -07:00
|
|
|
|
| Status | ✅ COMPLETE |
|
2026-08-28 10:05:45 -07:00
|
|
|
|
| Depends | M3.8.1 (CacheAligner complete) |
|
|
|
|
|
|
| Blocks | M3.8.3 |
|
|
|
|
|
|
|
|
|
|
|
|
## Goal
|
|
|
|
|
|
|
|
|
|
|
|
Integrate CacheAligner output into HTTP response headers and observability metrics
|
|
|
|
|
|
so that:
|
|
|
|
|
|
1. LLM provider can use cache hints
|
|
|
|
|
|
2. Monitoring can track cache effectiveness
|
|
|
|
|
|
3. Debugging can identify cache misses
|
|
|
|
|
|
|
|
|
|
|
|
## Deliverables
|
|
|
|
|
|
|
|
|
|
|
|
### 1. PromptBuilder::cache_metrics()
|
|
|
|
|
|
|
|
|
|
|
|
New method returning cache metadata:
|
|
|
|
|
|
|
|
|
|
|
|
```rust
|
|
|
|
|
|
pub struct CacheMetrics {
|
|
|
|
|
|
pub stable_prefix_bytes: usize,
|
|
|
|
|
|
pub dynamic_tail_bytes: usize,
|
|
|
|
|
|
pub drift_metric: f32, // 0.0-1.0 ratio
|
|
|
|
|
|
pub cache_eligible: bool, // true if drift < 0.3
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
impl PromptBuilder {
|
|
|
|
|
|
pub fn cache_metrics(query: &Query, chunk: &Chunk) -> Result<CacheMetrics>
|
|
|
|
|
|
}
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Tests (3):
|
|
|
|
|
|
- `test_cache_metrics_stable_query`
|
|
|
|
|
|
- `test_cache_metrics_high_drift`
|
|
|
|
|
|
- `test_cache_metrics_zero_drift`
|
|
|
|
|
|
|
|
|
|
|
|
### 2. HTTP Response Headers
|
|
|
|
|
|
|
|
|
|
|
|
Add to PromptBuilder output:
|
|
|
|
|
|
- `X-Cache-Stable-Bytes`: size of cacheable prefix
|
|
|
|
|
|
- `X-Cache-Drift`: 0.0-1.0 ratio
|
|
|
|
|
|
- `X-Cache-Eligible`: "true"/"false"
|
|
|
|
|
|
- `X-Compression-Ratio`: original vs. compressed
|
|
|
|
|
|
|
|
|
|
|
|
Tests (4):
|
|
|
|
|
|
- `test_headers_present_in_response`
|
|
|
|
|
|
- `test_headers_accurate_values`
|
|
|
|
|
|
- `test_headers_skipped_when_disabled`
|
|
|
|
|
|
- `test_headers_format_valid`
|
|
|
|
|
|
|
|
|
|
|
|
### 3. Observability Hooks
|
|
|
|
|
|
|
|
|
|
|
|
Integrate with logging:
|
|
|
|
|
|
```rust
|
|
|
|
|
|
pub fn log_cache_metrics(metrics: &CacheMetrics) {
|
|
|
|
|
|
tracing::info!(
|
|
|
|
|
|
stable_bytes = metrics.stable_prefix_bytes,
|
|
|
|
|
|
drift = metrics.drift_metric,
|
|
|
|
|
|
eligible = metrics.cache_eligible,
|
|
|
|
|
|
"cache_alignment"
|
|
|
|
|
|
);
|
|
|
|
|
|
}
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
Tests (2):
|
|
|
|
|
|
- `test_metrics_logged_on_alignment`
|
|
|
|
|
|
- `test_drift_high_triggers_warning`
|
|
|
|
|
|
|
|
|
|
|
|
## Acceptance
|
|
|
|
|
|
|
|
|
|
|
|
- All 9 new tests passing
|
|
|
|
|
|
- Existing 114 mem-core tests still pass
|
|
|
|
|
|
- Cache metrics accurately reflect alignment
|
|
|
|
|
|
- HTTP headers present and valid
|
|
|
|
|
|
- Zero performance overhead (< 1ms additional)
|