docs: clarify optimizer sits in query path only, full lifecycle diagram

This commit is contained in:
Story Crater Bot
2026-08-28 09:19:35 -07:00
parent 0b2932bb77
commit 2c37d7b6f2
+59 -11
View File
@@ -925,20 +925,51 @@ temp paths, ANSI codes, verbose JSON arrays, and passing test output. Feeding
this noise to the GRU-Mem gate wastes tokens, risks hallucination on irrelevant this noise to the GRU-Mem gate wastes tokens, risks hallucination on irrelevant
details, and breaks LLM provider KV cache (dynamic content in prefix). details, and breaks LLM provider KV cache (dynamic content in prefix).
### Where It Sits ### Full Lifecycle: Where Optimization Happens
Two paths through the system. Optimization touches **only the query path**,
never the ingest path.
``` ```
Query → Hybrid Search (pgvector 60% + OpenSearch 40%) ═══ INGEST PATH (full fidelity, NO optimization) ═══════════════════
Agent transcript / tool output / log
│ full-fidelity chunks (untouched)
┌───────────────────────┐ ┌───────────────────────┐
│ CONTEXT OPTIMIZER │ │ Chunker (M3.6.1) │ heading-boundary split
└───────────┬───────────┘
┌───────────────────────┐
│ Embed (nomic 768d) │ full text → vector
└───────────┬───────────┘
┌─────┴─────┐
▼ ▼
pgvector OpenSearch ← FULL TEXT stored here
(semantic) (BM25 lexical) never compressed
═══ QUERY PATH (optimized before LLM) ══════════════════════════
User query
┌───────────────────────┐
│ Hybrid Search │ pgvector 60% + OpenSearch 40%
│ (full-text match) │ searches FULL text, not compressed
└───────────┬───────────┘
│ retrieved chunks (full fidelity)
┌───────────────────────┐
│ CONTEXT OPTIMIZER │ ← COMPRESSION HAPPENS HERE
│ │ │ │
│ 1. Magika Detect │ ML content type classification (<1ms) │ 1. Magika Detect │ classify content type (<1ms)
│ 2. CacheAligner │ Move timestamps/UUIDs to tail │ 2. CacheAligner │ stabilize prefix for KV cache
│ 3. Compressor Per-type compression │ 3. Compressor │ shrink per content type
│ 4. CCR Store Cache originals for retrieval │ 4. CCR Store │ cache originals (reversible)
│ │ │ │
└───────────┬───────────┘ └───────────┬───────────┘
@@ -949,11 +980,28 @@ Query → Hybrid Search (pgvector 60% + OpenSearch 40%)
└───────────┬───────────┘ └───────────┬───────────┘
LLM Gateway ┌───────────────────────┐
│ LLM Gate (GRU-Mem) │ evaluate evidence, update memory
└───────────┬───────────┘
│ <check>yes/no</check>
│ <update>memory</update>
┌───────────────────────┐
│ JSONL Event Log │ append memory update event
└───────────┬───────────┘
│ event stored (full text, not compressed)
┌─────┴─────┐
▼ ▼
pgvector OpenSearch ← memory update indexed
(re-embed) (re-index) at full fidelity
``` ```
**Critical invariant:** Search indexes (pgvector + OpenSearch) NEVER see **Key rule:** Compression is ephemeral. It exists only in the prompt for one
compressed content. Compression only happens in the prompt assembly path. LLM call. The event log, search indexes, and stored memories never see
compressed content. If you rebuild from log, you get full-fidelity text.
### Stage 1: Content Detection (Magika ML) ### Stage 1: Content Detection (Magika ML)