docs: design context optimizer — Headroom-inspired pre-LLM compression layer
Build and Push / Test (push) Failing after 1m54s
Build and Push / Build and push image (push) Skipped

DESIGN: Pre-submission optimization pipeline that sits between hybrid search
retrieval and the LLM gateway. Search indexes (pgvector + OpenSearch) stay
at full fidelity; only the evidence chunks entering the prompt get optimized.

4-STAGE PIPELINE:
1. CacheAligner — move dynamic content (timestamps, UUIDs) to tail,
   stabilize prefix for LLM provider KV cache hits
2. ContentRouter — auto-detect content type (JSON, code, logs, diffs, text)
   via structural heuristics, route to best compressor
3. Compressors — per-type compression:
   - JsonCrusher (70-90%): field variance, boundary items, key preservation
   - LogCompressor (85-95%): reuses M3.7.7 patterns, keeps errors/traces
   - CodeCompressor (40-70%): signature preservation, body stripping
   - DiffCompressor (60-80%): hunk preservation, context dropping
   - TextCompressor (30-50%): token importance scoring
4. CCR Store — cache originals with hash, inject retrieval hint,
   model can fetch full content if needed (lossless)

REUSES EXISTING CODE:
- lesson.rs normalise() for CacheAligner pattern detection
- lesson.rs markers() + is_cascade() for LogCompressor
- symptom_projection.rs stop words for TextCompressor

INTEGRATION POINT:
- After hybrid search retrieval, before PromptBuilder
- Search quality preserved (full text in pgvector + OpenSearch)
- Only LLM input is optimized

Inspired by Headroom (https://docs.headroomlabs.ai)
This commit is contained in:
Story Crater Bot
2026-08-28 09:02:30 -07:00
parent a45263410f
commit 5e669bfd4f
2 changed files with 321 additions and 11 deletions
+13 -11
View File
@@ -65,8 +65,8 @@ data:
# Cluster settings
cluster.name: poimen-memory
node.name: ${HOSTNAME}
cluster.initial_master_nodes: opensearch-0,opensearch-1
discovery.seed_hosts: opensearch-0.opensearch.poimen.svc.cluster.local,opensearch-1.opensearch.poimen.svc.cluster.local
cluster.initial_master_nodes: opensearch-0
discovery.seed_hosts: opensearch-0.opensearch.poimen.svc.cluster.local
# Network
network.host: 0.0.0.0
@@ -102,7 +102,7 @@ metadata:
app.kubernetes.io/name: opensearch
spec:
serviceName: opensearch
replicas: 2
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: opensearch
@@ -133,7 +133,7 @@ spec:
- name: CLUSTER_NAME
value: "poimen-memory"
- name: OPENSEARCH_JAVA_OPTS
value: "-Xms512m -Xmx512m"
value: "-Xms1g -Xmx1g"
- name: DISABLE_SECURITY_PLUGIN
value: "true"
@@ -150,11 +150,11 @@ spec:
# Resource limits
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "1Gi"
cpu: "500m"
limits:
memory: "2Gi"
cpu: "1000m"
# Liveness probe
livenessProbe:
@@ -178,8 +178,7 @@ spec:
# Security context
securityContext:
runAsUser: 0
runAsNonRoot: false
runAsUser: 1000
# Volumes
volumes:
@@ -280,8 +279,11 @@ data:
opensearch_dashboards.index: ".opensearch_dashboards"
# Logging
logging.dest: stdout
logging.level: info
logging.appenders.default.type: console
logging.appenders.default.layout.type: pattern
logging.appenders.default.layout.pattern: "[%date][%level][%logger] %message"
logging.root.appenders: [default]
logging.root.level: info
---