plan: M3.8 context optimizer (4 tasks, Headroom-inspired)

tasks/INDEX.md

@@ -66,15 +66,17 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked
 | 5 | Skills | M4.x | 3 | 2 | 0 | 1 | ⬜ M4.3 |
 | 5.5 | Reference corpora | M3.6.x | 5 | 1 | 0 | 4 | ⬜ M3.6.6 |
 | 5.6 | Tool context | M3.7.x | 4 | 2 | 0 | 2 | ⬜ M3.7.6 |
+| 5.7 | Context optimization | M3.8.x | 4 | 0 | 0 | 4 | ⬜ M3.8.4 |
 | 6 | Post-training | M5.x | 6 | 0 | 0 | 6 | ⬜ M5.6 |
 | 7 | agent-manager migration | M6.x | 6 | 0 | 0 | 6 | ⬜ M6.6 |
 | 8 | Source connectors | M7.x | 10 | 0 | 0 | 10 | ⬜ M7.10 |
 | 9 | Hybrid search | M8.x | 9 | 1 | 1 | 7 | ⬜ M8.9 |
-| | **Total** | | **70** | **60** | **0** | **10** | 6/11 green |
+| | **Total** | | **74** | **60** | **0** | **14** | 6/12 green |
 
 **Current status — 2025-01-28.** Completed phases M0.x, M1.x fully archived (16/16 tasks). **M2.1-6 ✅** (embeddings, CNPG, schema, pgvector, obsidian projector, rebuild). **M3.x ✅** (4/4). **M3.5.x ✅** (10/10 complete + archived). **M3.7.7-8 ✅** (failure diagnosis). **M4.1-2 ✅** (skill drafting + derived filter). **M3.6.1 ✅** (DocCorpusSource). **M3.6.3 ❌ retired** (Obsidian UI replaces CLI). **M8.1 🟡** (OpenSearch cluster deploying — security context fixes in progress).
 
 **Current work:**
+- M3.8.1: Context optimizer (Headroom-inspired pre-LLM compression)
 - M8.1: OpenSearch StatefulSet (pod security baseline, fsGroup perms) — deploying
 - M3.7.4: Context endpoint glue (uses tier logic + hybrid search)
 - Obsidian service: ✅ Deployed (ppatlabs/obsidian:latest, REST API on 27124)
@@ -194,6 +196,19 @@ Ids are `M3.7.x` and frozen. `M3.7.1`, `M3.7.2`, `M3.7.3`, `M3.7.5` retired.
 | M3.7.7 | Failure signature extraction + normalisation | M | — | ✅ ARCHIVED |
 | M3.7.8 | Symptom projection at ingest | M | — | ✅ ARCHIVED |
 
+## 5.7 — Context optimization · M3.8.x
+
+Headroom-inspired pre-LLM compression. Sits between hybrid search retrieval
+and the LLM gateway. Search indexes (pgvector + OpenSearch) stay at full
+fidelity; only evidence chunks entering the prompt get optimized.
+
+| Task | Title | Size | Flags | Status |
+|---|---|---|---|---|
+| [M3.8.1](M3.8.1-context-optimizer.md) | Context optimizer pipeline (router + compressors + CCR) | L | — | ⬜ |
+| M3.8.2 | CacheAligner integration with prompt builder | M | — | ⬜ |
+| M3.8.3 | Compression benchmarks + tuning | M | — | ⬜ |
+| M3.8.4 | **M3.8 composition gate** | M | gate | ⬜ |
+
 ## 6 — Post-training · M5.x
 
 Python, separate from the Rust workspace. The boundary is the JSONL log.