poimen-memory

Gated recurrent memory over agent context. Reads session history chunk-by-chunk, keeps only what answers standing questions, projects result into an Obsidian vault and a pgvector index.

Status: 78/78 tasks complete, all 13 phases done. Production-deployed on Kubernetes via ArgoCD. See CLAUDE.md for full API reference.

Problem

Agent sessions grow faster than anyone reads them, and most of the volume is noise. One real pi session in this project:

assistant   1445
toolResult  1261     43% — ls output, file reads, mostly evidence-free
user         196
+ 8 compaction events

Compaction fires 8 times per session. Context gets discarded, not retained — root causes, decisions and gotchas evaporate when window rolls.

Mechanism

GRU-Mem (arXiv 2602.10560). Two text-controlled gates on a recurrent memory loop:

• update gate — memory only mutates when chunk contains evidence. Blocks the memory explosion that ungated recurrent memory hits.
• exit gate — stop scanning once evidence sufficient.

Paper reports up to 400% speedup and better accuracy than ungated, because unbounded memory growth degrades later updates.

sessions ─> chunk (5000 tok) ─> controller ─> gates ─> memory ─> projections

Controller emits structured output; loop acts on it:

<think>   reason about chunk vs question
<check>   yes|no        -> U_t, update or discard
<update>  candidate memory M̂_t
<next>    continue|end  -> E_t, exit or continue

Memory tiers

Level	What	From	Bounded
L0	evidence chunk, verbatim	update gate opening	no, but sparse (~17 of 412)
L1	per-query memory, M_t	gated loop over chunks	1024 tok
L2	project synthesis	gated loop over L1 memories	1024 tok

L2 is not new machinery — same loop, same prompt, L1 memories as input stream. Level is a parameter.

Tiers form a provenance graph. Each L1 records its L0 parents, each L2 its L1 parents. Same relation becomes both memory_edge rows and Obsidian wikilinks.

Standing queries

Update gate needs a referent. Paper's agent is φθ(Q, C_t, M_{t-1}) — gate is defined as "does this chunk contain useful information about the problem". No Q, no gate, and r_update becomes undefinable, which kills post-training.

So each project declares durable questions. One query = one L1 memory = one note.

# queries/poimen.yaml
project: poimen
roots: [/Users/rockliang/workplace/Poimen/agent-rust]
queries:
  - id: infra-root-causes
    question: What infrastructure bugs were found, what was the root cause, how was it isolated?
  - id: architecture-decisions
    question: What architectural decisions were made, with reasoning and rejected alternatives?
synthesis:
  question: What is the current state of this project, and what should someone know before working on it?
  exit_gate: true

Exit gate off at L1, on at L2. Paper §3.3 makes this call: for "what are all the X" questions you cannot know evidence is sufficient without reading everything. L1 extraction is that shape. At L2 input is a handful of memories and sufficiency is decidable. Gate still recorded at L1 — signal needed for post-training.

Authority model

JSONL log authoritative. Vault and vector index are projections.

Anything not rebuildable byte-identically from the log has hidden inputs, and that is a bug. Gate M2.8 enforces it destructively:

rm -rf vault/poimen
psql -c "delete from memory_node where project='poimen'"
mem rebuild --from-log --project poimen
git -C vault diff --exit-code      # empty diff is the only pass

Buys three things: re-embedding after model change is a rebuild not a migration, Obsidian edits cannot corrupt the record, post-training corpus is the log itself.

Skills

A skill is a projection, not a level. L0/L1/L2 are descriptive — what happened. A skill is procedural — what to do next time. Gated loop does not produce it.

Format free: SKILL.md is YAML frontmatter + markdown, which is an Obsidian note. So vault/skills/<name>/SKILL.md is both, no conversion:

pi --skill vault/skills/
ln -s .../vault/skills/<name> ~/.claude/skills/<name>

Drafts land in _drafts/, promotion is a human git mv. This is the one cycle in the design:

emitted skill auto-loads -> appears in future transcripts
-> ingested as evidence -> reinforces the memory that emitted it

No external verifier breaks it. Two guards: _drafts/ is a directory (cannot be globbed into --skill), and every artifact carries generated_from so ingest tags matching chunks derived: true and refuses them as evidence.

Separate weights

Memory policy is a LoRA adapter on Qwen2.5-3B-Instruct, not a fine-tuned model. Reason is VRAM: one GPU, OLLAMA_MAX_LOADED_MODELS=2, already holding ornith:35b + qwen2.5:3b. Separate full model evicts something, and eviction is a weights reload measured in tens of seconds. Adapter rides the resident base.

Also: post-training emits ~50 MB, not 6 GB. Swap without redeploy. Regression reverts by pointing at previous adapter.

Ollama cannot hot-swap LoRA. Serving one needs vLLM with --enable-lora (pattern already exists — reasoning predictor is vLLM v0.11.0). Phases M0–M4 run prompted-only, so decision is deferred, not dodged.

Layout

DESIGN.md            full design, 460 lines
memory-tasks/        37 task files + INDEX.md — tracked
crates/
  mem-core/          domain types; Level; gate parser; the gated loop
  mem-chunk/         RecordSource trait; ChunkPolicy; FlushTrigger
  mem-llm/           gateway client — chat, embeddings, rerank
  mem-ingest/        source adapters: pi sessions, claude transcripts
  mem-store/         JSONL log; pgvector repo; Obsidian projector
  mem-cli/           binary `mem`
queries/             standing query YAML per project
log/                 JSONL event log — authoritative, tracked
vault/               Obsidian output

mem-chunk is separate and stream-shaped from day one. Sources today are files with an EOF; telemetry or a live tail will not have one. RecordSource returns impl Stream<Item = Record>; batch sources become streams via futures::stream::iter, so it costs nothing now and removes a rewrite later.

Commands

# Ingest knowledge via gated loop (LLM evaluates + compacts automatically)
mem learn knowledge/rust.md                          # single file
mem learn knowledge/ --project myproject              # directory
mem learn knowledge/ --dry-run                        # preview chunks
mem learn knowledge/ --memory-budget 8192             # larger memory window
mem learn knowledge/ --model ornith:35b               # use stronger model

# Traditional ingest (from session transcripts)
mem ingest --project poimen --dry-run
mem ingest --project poimen --query infra-root-causes

# Failure capture + lesson derivation
mem capture --cmd "cargo build" --exit 1 --output-file error.log
mem sig --tool cargo --file error.log                 # extract failure signature
mem resolve --json                                    # pair failure with fix
mem lookup --tool cargo --file error.log              # search known fixes

# Skills + projections
mem skill draft --from poimen/infra-root-causes
mem materialize                                       # generate SKILL.md files
mem verify --project poimen                           # provenance graph closure

# Server
mem serve --port 8080

HTTP API

# Health
curl http://localhost:8080/health

# Learn — gated loop ingest (LLM evaluates + compacts)
curl -X POST http://localhost:8080/memory/learn \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"project": "knowledge", "text": "## Rust\n- ownership...", "query": "key patterns?"}'
# Returns: {chunks_seen, chunks_used, memory: "compacted...", stored: true}

# Ingest — queue-based async ingest
curl -X POST http://localhost:8080/memory/ingest ...

# Query — hybrid semantic + lexical search
curl http://localhost:8080/memory/query?project=poimen&q=port+conflict

# Context — three-tier retrieval (signature > vector > reference)
curl -X POST http://localhost:8080/memory/context \
  -d '{"project": "poimen", "tool": "cargo", "task": "build", "budget": 4096}'

Learning Flow

Agent/CLI → POST /memory/learn → chunk markdown → gated loop:
  For each chunk:
    LLM evaluates: does this add new knowledge? (update gate)
    If yes → LLM rewrites memory incorporating new fact (compaction)
    If no → chunk rejected, memory unchanged
  → Store compacted memory in pgvector (embedded, searchable)
  → Return {chunks_seen, chunks_used, memory, stored}

Memory never grows unbounded — every update is a rewrite, not an append. The LLM acts as both evaluator and compactor in one pass.

M3.8 Pluggable Query Optimization

Purpose: Compress and optimize search results before passing them to the LLM context window, improving token efficiency and response quality.

Architecture

M3.8 provides dual-path optimization:

Ingest-Time Optimization (M3.8.2)

When documents are ingested, they're automatically optimized before embedding:

Records → optimize_record_with_metrics() → Clean chunks (85-95% of original)
       → Embed (pgvector) → Index (OpenSearch)

Benefits:

• Better pgvector embeddings (clean text = higher semantic quality)
• Better OpenSearch BM25 ranking (signal-rich text = stronger matches)
• One-time cost per document
• All queries benefit from cleaner search index

Query-Time Optimization (QueryOptimizer)

When search results are retrieved, they're optimized before LLM processing:

Hybrid search results → QueryOptimizer.optimize_chunks() → Clean chunks
                     → LLM context window

Benefits:

• Smaller context window (fewer tokens to LLM)
• Faster response generation
• Focus on signal (removes noise like timestamps, debug lines, repetitive keys)

Using Query Optimization

1. Basic Query with Auto-Optimization

use mem_core::optimizer::QueryOptimizer;

// Create optimizer (loads config from env vars)
let query_optimizer = QueryOptimizer::from_env();

// Get search results
let chunks = hybrid_search(&question).await?;

// Auto-optimize before LLM
let optimized = query_optimizer.optimize_chunks(&chunks).await?;

// Build context from clean chunks
let context = optimized.join("\n---\n");
let response = llm.prompt(&context, &question).await?;

2. Prompt Construction with Optimization

use mem_core::optimizer::QueryOptimizer;
use mem_core::prompt::PromptBuilder;

let query_optimizer = QueryOptimizer::from_env();

// Retrieve and optimize
let chunks = hybrid_search(query).await?;
let optimized_chunks = query_optimizer.optimize_chunks(&chunks).await?;

// Build cache-aligned prompt with optimized chunks
let (system, user_message) = PromptBuilder::build_cache_aligned(
    query,
    previous_memory.as_deref(),
    /* use optimized chunks */
)?;

let response = llm.prompt(system, user_message).await?;

3. Custom Query Optimizer Implementation

For domain-specific optimization (e.g., medical, legal, technical content):

use mem_core::optimizer::{OptimizerPlugin, OptimizationResult, PluginMetrics};
use async_trait::async_trait;

struct MedicalOptimizer;

#[async_trait]
impl OptimizerPlugin for MedicalOptimizer {
    fn name(&self) -> &str { "medical-optimizer" }
    
    fn supported_types(&self) -> Vec<&str> {
        vec!["text/medical", "text/clinical", "application/json"]
    }
    
    async fn optimize(&self, content: &str) -> Result<OptimizationResult, String> {
        // Remove patient IDs, reduce duplicate diagnosis entries
        let cleaned = clean_medical_data(content);
        let ratio = cleaned.len() as f32 / content.len() as f32;
        
        Ok(OptimizationResult {
            original: content.to_string(),
            optimized: cleaned,
            ratio,
            plugin: self.name().to_string(),
            metadata: Default::default(),
        })
    }
    
    fn metrics(&self) -> PluginMetrics { Default::default() }
}

// Register and use
let service = OptimizerServiceBuilder::new()
    .with_optimizer(Arc::new(MedicalOptimizer))
    .with_format(Arc::new(JsonFormatter))
    .build()?;

let optimized = service.optimize(
    clinical_note,
    "text/clinical",
    None
).await?;

4. Optimized Query with Metrics Tracking

use mem_core::optimizer::{QueryOptimizer, QueryOptimizationMetrics};
use mem_core::prompt::CacheMetrics;

let query_optimizer = QueryOptimizer::from_env();

let chunks = hybrid_search(query).await?;
let optimized = query_optimizer.optimize_chunks(&chunks).await?;

// Track optimization effectiveness
let metrics: Vec<QueryOptimizationMetrics> = chunks
    .iter()
    .zip(&optimized)
    .map(|(orig, opt)| {
        QueryOptimizationMetrics {
            original_bytes: orig.text.len(),
            cache_stable_bytes: /* from CacheMetrics */,
            cache_drift: /* from CacheMetrics */,
            is_cache_eligible: /* from CacheMetrics */,
            has_optimizer: true,
        }
    })
    .collect();

tracing::info!(
    chunks = chunks.len(),
    compression_ratio = format!(
        "{:.1}%",
        (optimized.iter().map(|o| o.len()).sum::<usize>() as f32 
         / chunks.iter().map(|c| c.text.len()).sum::<usize>() as f32) * 100.0
    ),
    "query optimization complete"
);

// Query with optimized context
let response = llm.prompt(&optimized.join("\n---\n"), &question).await?;

5. Batch Optimization for Multiple Queries

use mem_core::optimizer::QueryOptimizer;

let query_optimizer = QueryOptimizer::from_env();

// Process multiple queries with shared optimizer
let results = futures::stream::iter(queries)
    .then(|query| async move {
        let chunks = hybrid_search(&query).await?;
        let optimized = query_optimizer.optimize_chunks(&chunks).await?;
        let response = llm.prompt(&optimized.join("\n---\n"), &query.question).await?;
        Ok((query, response))
    })
    .collect::<Vec<_>>()
    .await;

6. Conditional Optimization (Graceful Fallback)

use mem_core::optimizer::QueryOptimizer;

let query_optimizer = QueryOptimizer::from_env();

let chunks = hybrid_search(query).await?;

// Try optimization, fall back to original if it fails
let context = match query_optimizer.optimize_chunks(&chunks).await {
    Ok(optimized) => {
        tracing::info!("query optimization succeeded");
        optimized.join("\n---\n")
    }
    Err(e) => {
        tracing::warn!("query optimization failed: {}, using original", e);
        chunks.iter().map(|c| c.text.clone()).collect::<Vec<_>>().join("\n---\n")
    }
};

let response = llm.prompt(&context, &question).await?;

Environment Configuration

# Enable/disable query optimization
export MEM_QUERY_OPTIMIZER=on              # or "off"

# Custom optimizer service (optional)
export MEM_QUERY_OPTIMIZER_SERVICE=/path/to/config.yml

# Compression targets (if using custom optimizers)
export MEM_COMPRESSION_TARGETS='{
  "logs": {"min": 0.05, "max": 0.95},
  "json": {"min": 0.10, "max": 0.90},
  "text": {"min": 0.30, "max": 0.70}
}'

# Ingest-time optimization
export MEM_CONTEXT_OPTIMIZER=on

Compression Targets by Content Type

Type	Target	Typical	Example
Logs	85-95% removal	10-15% remaining	ERROR + timestamps → ERROR only
JSON	70-90% removal	10-30% remaining	Minified + key filtering
Text/Markdown	30-50% removal	50-70% remaining	Prose kept, formatting removed
Code/Diffs	60-80% removal	20-40% remaining	Context lines removed

Performance Targets

Metric	Target	Status
Ingest latency	<1ms per record	✅ Passing
Query latency	<50ms P95	✅ Passing
Compression ratio	Within targets	✅ Passing
Graceful fallback	Always succeeds	✅ Passing

Monitoring

Track optimization effectiveness via structured logging:

tracing::info!(
    event = "query_optimization",
    chunks_count = chunks.len(),
    original_bytes = total_input,
    optimized_bytes = total_output,
    compression_ratio = format!("{:.1}%", ratio),
    elapsed_ms = elapsed.as_secs_f64() * 1000.0,
    has_optimizer = query_optimizer.enabled,
    "query optimization metrics"
);

Export to Prometheus (ingest-time metrics):

curl http://localhost:9090/metrics | grep m3_8_optimization

Best Practices

1. Always gracefully fall back — Optimization may fail; original chunks should be used
2. Set reasonable compression targets — Too aggressive = information loss; too loose = waste
3. Monitor metrics — Track compression ratios per content type to ensure targets are met
4. Test custom optimizers — Validate that cleaned content preserves semantic meaning
5. Use batch operations — optimize_chunks() is more efficient than single-chunk calls
6. Cache formatter instances — Create format handlers once, reuse across queries

Further Reading

• M3.8 Pluggable Optimizer Guide — Full architecture details
• M3.8 Completion Summary — Implementation status
• Query Optimizer Source — Implementation code

Verified environment facts

Checked against the running cluster, not assumed:

Fact	Value
Embedding dims	768, nomic-ai/nomic-embed-text-v2-moe
Embedding batch limit	32 (batch size 1200 > maximum allowed batch size 32)
pgvector	0.7.0 available in stock CNPG image, no custom build
CNPG operator	1.30.0, declarative Database.spec.extensions
Ollama context cap	32768 (OLLAMA_CONTEXT_LENGTH) — cluster-side, overrides client config
Controller	qwen2.5:3b-instruct — paper's exact 3B backbone
Gateway auth	apikey: header. Authorization: Bearer returns 401
Rerank response	bare array, not {"data":[...]}; sorted by score, map back via index

Budget fits the 32K cap: 5000 chunk + ~3200 prompt/memory + 2048 response.

Phases

Each ends in a composition gate. No phase starts until predecessor gate is green.

	Phase	Tasks	Gate asserts
M0	Read-only spine	8	third source needs no downstream change; runs offline
M1	Gated loop at L1	8	update-rate < 30%, memory flat not climbing
M2	Projections	8	rebuild byte-identical from log alone
M3	L2 + retrieval	4	hit rate ≥ 0.8, provenance precision ≥ 0.9
M4	Skills	3	draft not loadable; promoted skill never becomes evidence
M5	Post-training	6	adapter beats prompted baseline on held-out project

Update-rate is the number to watch. It is what distinguishes a gate from an expensive summarizer. Tool results are 43% of records and mostly evidence-free, so a correct gate rejects the large majority of chunks.

M0 and M2.2 need no model access and can start immediately. M5.4 (vLLM + LoRA) is homelab work independent of the rest of M5.

Reading order

1. This file
2. memory-tasks/INDEX.md — board, ordering rules, verification practice
3. DESIGN.md — full design, schemas, risks
4. Individual task files — self-contained, no DESIGN.md read required

Trigger build run 130

CI trigger