poimenandrock
fb61de6b47
## Changes
### Entity Extraction
- Switch from WikiLinkFallbackExtractor to LlmEntityExtractor when LLM_ENDPOINT set
- `clean_llm_response()`: strips `<think>` tags, markdown fences, extracts JSON
- Handle array responses (Ollama returns `[...]` not `{entities: [...]}`)
- EntityType custom Deserialize: unknown variants → Unknown (no crash)
- Increase timeout 30s→90s, max_tokens 500→1500 for reasoning models
- Graceful reflection fallback: keep entities if verification fails
### Fact Extraction (NEW)
- LlmFactExtractor: LLM-based relationship extraction between entity pairs
- Validates source/target against known entity list (drops hallucinated edges)
- Same robust JSON cleaning for reasoning models + Ollama
- IngestWorker auto-selects LLM vs Simple based on LLM_ENDPOINT env
### K8s Deployment
- Add `command: ["/app/mem"]` (fix args replacing CMD)
- Add LLM_ENDPOINT, LLM_MODEL env vars for in-cluster LLM
## E2E Tested (local Ollama qwen2.5:3b)
- 12 entities extracted (person, tool, concept, organization)
- 5 edges with relationships and facts
- 781 tests pass
## Zep Paper Alignment (§2.2)
- Entity extraction + resolution (§2.2.1)
- Fact extraction between entity pairs (§2.2.2)
- Temporal edge invalidation ready (t_valid/t_invalid schema)
- Reflection verification (§2.2.1, graceful fallback)
---------
Co-authored-by: rock <[email protected]>
Reviewed-on: #48
Co-authored-by: poimen <[email protected]>
137 lines
5.4 KiB
Markdown
137 lines
5.4 KiB
Markdown
# Poimen Memory System
|
|
|
|
## Project Status
|
|
|
|
**Architecture**: Temporal Knowledge Graph for Agent Memory (Zep paper alignment — arXiv:2501.13956)
|
|
|
|
**Current**: Ingest pipeline with LLM entity + fact extraction working E2E. Deployed to K8s.
|
|
|
|
### What Works
|
|
- ✅ HTTP server (actix-web) with 15+ endpoints
|
|
- ✅ LLM entity extraction (LlmEntityExtractor) — extracts person/tool/concept/org entities
|
|
- ✅ LLM fact extraction (LlmFactExtractor) — extracts relationships between entities
|
|
- ✅ Reasoning model support — strips `<think>` tags, markdown fences
|
|
- ✅ Ollama + vLLM + OpenAI-compatible API support
|
|
- ✅ Entity persistence to pgvector (memory_entity table)
|
|
- ✅ Edge persistence (memory_edge table with temporal fields)
|
|
- ✅ Graph query endpoints (entities, edges, BFS traversal)
|
|
- ✅ Visualization (React Flow JSON, force-directed layout, SSE streaming)
|
|
- ✅ JWT auth (Authentik OIDC) with RBAC
|
|
- ✅ K8s deployment (CNPG postgres, ConfigMap, SOPS secrets)
|
|
- ✅ CI: PR builds push :SHA tag, main merges retag :latest
|
|
- ✅ 781 tests passing
|
|
|
|
### Deployment
|
|
- **Namespace**: `poimen`
|
|
- **Image**: `forgejo.riotpiao.com/riotpiao-poimen/poimen-memory:latest`
|
|
- **DB**: CNPG cluster `memory-db` (pgvector)
|
|
- **LLM**: `reasoning-predictor.llm-serving.svc.cluster.local` (ornith:35b / qwen2.5:3b)
|
|
- **Auth**: Authentik OIDC (`MEM_AUTH_MODE=none` for dev)
|
|
- **Registry**: Forgejo container registry (FORGEJO_REGISTRY_USER/TOKEN secrets)
|
|
|
|
### Key Env Vars
|
|
```
|
|
DATABASE_URL postgresql://...
|
|
MEM_AUTH_MODE none|jwt|apikey
|
|
LLM_ENDPOINT http://localhost:11434/v1/chat/completions (Ollama)
|
|
LLM_MODEL qwen2.5:3b | ornith:35b | reasoning
|
|
LLM_API_KEY (for authenticated LLM APIs)
|
|
MEM_API_KEY (server API key, fallback "test-key")
|
|
OPENSEARCH_HOSTS (optional, hybrid search)
|
|
GATEWAY_URL (optional, external queue)
|
|
```
|
|
|
|
## Rules
|
|
|
|
1. **No progress markdown files.** Track via Forgejo issues + PRs only.
|
|
2. **Obsidian vault repo**: `ssh://[email protected]:2222/rock/poimen-obesdient-memory.git`
|
|
3. **Secrets via KSOPS**: Age-based SOPS encryption. Never commit plaintext.
|
|
4. **Tea CLI**: `poimen` login has API token `1f717a00134f17c9d2d656c620b955e03ea41276`
|
|
|
|
## Architecture (Zep Paper §2)
|
|
|
|
### Three-Tier Knowledge Graph
|
|
```
|
|
Episode Subgraph (raw messages)
|
|
→ Entity Subgraph (extracted entities + facts/edges)
|
|
→ Community Subgraph (clusters, planned Phase 4)
|
|
```
|
|
|
|
### Ingest Pipeline (4 stages)
|
|
1. **Entity extraction** — LLM extracts named entities with type + summary
|
|
2. **Deduplication** — HashSet on normalized name
|
|
3. **Fact extraction** — LLM extracts relationships between entity pairs
|
|
4. **Contradiction detection** — pre-filter + review queue
|
|
|
|
### Retrieval (3 methods, §3)
|
|
- Cosine semantic similarity (pgvector HNSW)
|
|
- BM25 full-text (OpenSearch, optional)
|
|
- BFS graph traversal (depth 1-3)
|
|
|
|
### Extractors
|
|
- `LlmEntityExtractor`: calls LLM_ENDPOINT, parses JSON, handles reasoning models
|
|
- `LlmFactExtractor`: takes entity list + text, extracts edges between known entities
|
|
- `WikiLinkFallbackExtractor`: pattern-matches `[[wiki links]]` (no LLM)
|
|
- `SimpleFactExtractor`: verb pattern matching (no LLM)
|
|
- Selection: LLM extractors when `LLM_ENDPOINT` set, else fallbacks
|
|
|
|
### LLM Response Cleaning
|
|
`clean_llm_response()` handles:
|
|
- `<think>...</think>` blocks (reasoning models)
|
|
- Markdown code fences (```json ... ```)
|
|
- Array responses (wrap in `{"entities": [...]}`)
|
|
- Extract first JSON object from mixed text
|
|
|
|
## Crate Structure
|
|
|
|
```
|
|
crates/
|
|
mem-core/ — Entity, Edge, domain types (174 tests)
|
|
mem-store/ — DB repos, schema, vector store
|
|
mem-ingest/ — Entity/fact extraction, contradiction detection (87 tests)
|
|
mem-llm/ — Embeddings, chat, rerank clients
|
|
mem-cli/ — HTTP server, handlers, query, ingest worker (496 tests)
|
|
```
|
|
|
|
## API Endpoints
|
|
|
|
```
|
|
GET /health
|
|
POST /memory/ingest — Queue ingest job
|
|
GET /memory/ingest/{id} — Check job status
|
|
GET /memory/query?project=&question= — Graph query
|
|
POST /memory/query — Unified query
|
|
POST /memory/context — Three-tier retrieval
|
|
POST /memory/learn — Direct learn
|
|
POST /memory/visualize — React Flow JSON
|
|
POST /memory/visualize/stream — SSE streaming
|
|
POST /memory/compact — Trigger compaction
|
|
GET /memory/projects — List projects
|
|
GET /memory/skills — List skills
|
|
GET /memory/vault — Browse vault
|
|
POST /memory/synthesis/* — Entity linking, alias detection
|
|
```
|
|
|
|
## Current PRs / Branches
|
|
|
|
- **PR #48** `feat/memory-ingest-retrieval` — LLM entity + fact extraction, deployment fixes
|
|
- **PR #47** merged — Agent entity types (Phase 3.1)
|
|
- **PR #46** merged — Integration test fixes, CI
|
|
|
|
## Next Steps
|
|
|
|
1. Merge PR #48 → new image with LLM extraction
|
|
2. Query retrieval E2E — verify entities/edges returned in query results
|
|
3. Visualization E2E — test /memory/visualize with extracted graph
|
|
4. Restore 198 deleted tests from PR #46
|
|
5. Community detection (Phase 4, Zep §2.3)
|
|
6. Temporal edge invalidation (Zep §2.2.3)
|
|
7. Reranker (cross-encoder, RRF, episode-mentions — Zep §3.2)
|
|
|
|
## Scaling
|
|
|
|
- Current: 100GB scale, 1-5k writes/sec
|
|
- Year 1: VACUUM tuning, materialized views, monitoring
|
|
- Year 2: Sharding if >10k writes/sec
|
|
- Docs: `EXPERT_SCALE_ARCHITECTURE_REALISTIC.md`
|