feat: POST /memory/learn endpoint + refactor mem learn CLI
Learning flow now goes through the service, not local JSONL: - POST /memory/learn: accepts markdown, chunks it, runs gated loop (LLM evaluates + compacts), stores in pgvector. OpenAI-style API. - mem learn CLI: reads files, calls POST /memory/learn per file - Removed cmd_compact (gated loop IS the compaction) - Updated README with new commands and API docs Memory never grows unbounded — every update is a rewrite, not append. The gated loop LLM acts as evaluator + compactor in one pass.
This commit is contained in:
@@ -4,8 +4,8 @@ Gated recurrent memory over agent context. Reads session history chunk-by-chunk,
|
||||
keeps only what answers standing questions, projects result into an Obsidian
|
||||
vault and a pgvector index.
|
||||
|
||||
**Status: design complete, no code yet.** 37 tasks in [memory-tasks/](memory-tasks/INDEX.md),
|
||||
0 done. Start at [M0.1](memory-tasks/M0.1-cargo-workspace.md).
|
||||
**Status: 78/78 tasks complete, all 13 phases done.** Production-deployed on Kubernetes
|
||||
via ArgoCD. See [CLAUDE.md](CLAUDE.md) for full API reference.
|
||||
|
||||
## Problem
|
||||
|
||||
@@ -171,16 +171,71 @@ with an EOF; telemetry or a live tail will not have one. `RecordSource` returns
|
||||
## Commands
|
||||
|
||||
```sh
|
||||
mem ingest --project poimen --dry-run # chunk plan, zero model calls
|
||||
# Ingest knowledge via gated loop (LLM evaluates + compacts automatically)
|
||||
mem learn knowledge/rust.md # single file
|
||||
mem learn knowledge/ --project myproject # directory
|
||||
mem learn knowledge/ --dry-run # preview chunks
|
||||
mem learn knowledge/ --memory-budget 8192 # larger memory window
|
||||
mem learn knowledge/ --model ornith:35b # use stronger model
|
||||
|
||||
# Traditional ingest (from session transcripts)
|
||||
mem ingest --project poimen --dry-run
|
||||
mem ingest --project poimen --query infra-root-causes
|
||||
mem synthesize --project poimen # L2 pass, exit gate on
|
||||
mem rebuild --from-log --project poimen # drop and rebuild projections
|
||||
mem verify --project poimen # provenance graph closure
|
||||
mem query "why did requests over 10KB fail?"
|
||||
|
||||
# Failure capture + lesson derivation
|
||||
mem capture --cmd "cargo build" --exit 1 --output-file error.log
|
||||
mem sig --tool cargo --file error.log # extract failure signature
|
||||
mem resolve --json # pair failure with fix
|
||||
mem lookup --tool cargo --file error.log # search known fixes
|
||||
|
||||
# Skills + projections
|
||||
mem skill draft --from poimen/infra-root-causes
|
||||
mem label --project poimen # evidence labels for training
|
||||
mem materialize # generate SKILL.md files
|
||||
mem verify --project poimen # provenance graph closure
|
||||
|
||||
# Server
|
||||
mem serve --port 8080
|
||||
```
|
||||
|
||||
## HTTP API
|
||||
|
||||
```sh
|
||||
# Health
|
||||
curl http://localhost:8080/health
|
||||
|
||||
# Learn — gated loop ingest (LLM evaluates + compacts)
|
||||
curl -X POST http://localhost:8080/memory/learn \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"project": "knowledge", "text": "## Rust\n- ownership...", "query": "key patterns?"}'
|
||||
# Returns: {chunks_seen, chunks_used, memory: "compacted...", stored: true}
|
||||
|
||||
# Ingest — queue-based async ingest
|
||||
curl -X POST http://localhost:8080/memory/ingest ...
|
||||
|
||||
# Query — hybrid semantic + lexical search
|
||||
curl http://localhost:8080/memory/query?project=poimen&q=port+conflict
|
||||
|
||||
# Context — three-tier retrieval (signature > vector > reference)
|
||||
curl -X POST http://localhost:8080/memory/context \
|
||||
-d '{"project": "poimen", "tool": "cargo", "task": "build", "budget": 4096}'
|
||||
```
|
||||
|
||||
### Learning Flow
|
||||
|
||||
```
|
||||
Agent/CLI → POST /memory/learn → chunk markdown → gated loop:
|
||||
For each chunk:
|
||||
LLM evaluates: does this add new knowledge? (update gate)
|
||||
If yes → LLM rewrites memory incorporating new fact (compaction)
|
||||
If no → chunk rejected, memory unchanged
|
||||
→ Store compacted memory in pgvector (embedded, searchable)
|
||||
→ Return {chunks_seen, chunks_used, memory, stored}
|
||||
```
|
||||
|
||||
Memory never grows unbounded — every update is a rewrite, not an append.
|
||||
The LLM acts as both evaluator and compactor in one pass.
|
||||
|
||||
## M3.8 Pluggable Query Optimization
|
||||
|
||||
**Purpose**: Compress and optimize search results before passing them to the LLM context window, improving token efficiency and response quality.
|
||||
|
||||
Reference in New Issue
Block a user