docs: clarify optimizer sits in query path only, full lifecycle diagram
This commit is contained in:
+59
-11
@@ -925,20 +925,51 @@ temp paths, ANSI codes, verbose JSON arrays, and passing test output. Feeding
|
|||||||
this noise to the GRU-Mem gate wastes tokens, risks hallucination on irrelevant
|
this noise to the GRU-Mem gate wastes tokens, risks hallucination on irrelevant
|
||||||
details, and breaks LLM provider KV cache (dynamic content in prefix).
|
details, and breaks LLM provider KV cache (dynamic content in prefix).
|
||||||
|
|
||||||
### Where It Sits
|
### Full Lifecycle: Where Optimization Happens
|
||||||
|
|
||||||
|
Two paths through the system. Optimization touches **only the query path**,
|
||||||
|
never the ingest path.
|
||||||
|
|
||||||
```
|
```
|
||||||
Query → Hybrid Search (pgvector 60% + OpenSearch 40%)
|
═══ INGEST PATH (full fidelity, NO optimization) ═══════════════════
|
||||||
|
|
||||||
|
Agent transcript / tool output / log
|
||||||
│
|
│
|
||||||
│ full-fidelity chunks (untouched)
|
|
||||||
▼
|
▼
|
||||||
┌───────────────────────┐
|
┌───────────────────────┐
|
||||||
│ CONTEXT OPTIMIZER │
|
│ Chunker (M3.6.1) │ heading-boundary split
|
||||||
|
└───────────┬───────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌───────────────────────┐
|
||||||
|
│ Embed (nomic 768d) │ full text → vector
|
||||||
|
└───────────┬───────────┘
|
||||||
|
│
|
||||||
|
┌─────┴─────┐
|
||||||
|
▼ ▼
|
||||||
|
pgvector OpenSearch ← FULL TEXT stored here
|
||||||
|
(semantic) (BM25 lexical) never compressed
|
||||||
|
|
||||||
|
|
||||||
|
═══ QUERY PATH (optimized before LLM) ══════════════════════════
|
||||||
|
|
||||||
|
User query
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌───────────────────────┐
|
||||||
|
│ Hybrid Search │ pgvector 60% + OpenSearch 40%
|
||||||
|
│ (full-text match) │ searches FULL text, not compressed
|
||||||
|
└───────────┬───────────┘
|
||||||
|
│
|
||||||
|
│ retrieved chunks (full fidelity)
|
||||||
|
▼
|
||||||
|
┌───────────────────────┐
|
||||||
|
│ CONTEXT OPTIMIZER │ ← COMPRESSION HAPPENS HERE
|
||||||
│ │
|
│ │
|
||||||
│ 1. Magika Detect │ ML content type classification (<1ms)
|
│ 1. Magika Detect │ classify content type (<1ms)
|
||||||
│ 2. CacheAligner │ Move timestamps/UUIDs to tail
|
│ 2. CacheAligner │ stabilize prefix for KV cache
|
||||||
│ 3. Compressor │ Per-type compression
|
│ 3. Compressor │ shrink per content type
|
||||||
│ 4. CCR Store │ Cache originals for retrieval
|
│ 4. CCR Store │ cache originals (reversible)
|
||||||
│ │
|
│ │
|
||||||
└───────────┬───────────┘
|
└───────────┬───────────┘
|
||||||
│
|
│
|
||||||
@@ -949,11 +980,28 @@ Query → Hybrid Search (pgvector 60% + OpenSearch 40%)
|
|||||||
└───────────┬───────────┘
|
└───────────┬───────────┘
|
||||||
│
|
│
|
||||||
▼
|
▼
|
||||||
LLM Gateway
|
┌───────────────────────┐
|
||||||
|
│ LLM Gate (GRU-Mem) │ evaluate evidence, update memory
|
||||||
|
└───────────┬───────────┘
|
||||||
|
│
|
||||||
|
│ <check>yes/no</check>
|
||||||
|
│ <update>memory</update>
|
||||||
|
▼
|
||||||
|
┌───────────────────────┐
|
||||||
|
│ JSONL Event Log │ append memory update event
|
||||||
|
└───────────┬───────────┘
|
||||||
|
│
|
||||||
|
│ event stored (full text, not compressed)
|
||||||
|
▼
|
||||||
|
┌─────┴─────┐
|
||||||
|
▼ ▼
|
||||||
|
pgvector OpenSearch ← memory update indexed
|
||||||
|
(re-embed) (re-index) at full fidelity
|
||||||
```
|
```
|
||||||
|
|
||||||
**Critical invariant:** Search indexes (pgvector + OpenSearch) NEVER see
|
**Key rule:** Compression is ephemeral. It exists only in the prompt for one
|
||||||
compressed content. Compression only happens in the prompt assembly path.
|
LLM call. The event log, search indexes, and stored memories never see
|
||||||
|
compressed content. If you rebuild from log, you get full-fidelity text.
|
||||||
|
|
||||||
### Stage 1: Content Detection (Magika ML)
|
### Stage 1: Content Detection (Magika ML)
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user