Poimen Memory

Poimen (ποιμήν) — Greek for "shepherd". Guiding AI agents to grounded knowledge.

Agent-ready Graph-RAG system with hallucination prevention and enterprise RBAC.

Poimen Memory is a knowledge retrieval system designed for AI agents. It learns from conversations and documents, builds wiki-link knowledge graphs, and serves grounded context that reduces hallucinations. Agents cite sources instead of fabricating answers.

Why Poimen?

Problem	Poimen Solution
LLMs hallucinate facts	Three-tier retrieval grounds responses in verified knowledge
Vector search misses context	Wiki-link graph propagates relevance to connected docs
Agents forget across sessions	Persistent memory with provenance tracking
Multi-tenant data leakage	Hierarchical RBAC with project/visibility scopes
Context window limits	Budget-aware assembly with intelligent compression

Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                              AI Agents                                       │
│                    (Claude, GPT, Local LLMs, etc.)                          │
└─────────────────────────────────┬───────────────────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                         Poimen Memory API                                    │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐ │
│  │   /query    │  │  /context   │  │   /ingest   │  │      /learn         │ │
│  │ Hybrid RAG  │  │ Three-Tier  │  │  Add Facts  │  │ Chunk + Synthesize  │ │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘  └──────────┬──────────┘ │
│         │                │                │                     │           │
│         └────────────────┴────────────────┴─────────────────────┘           │
│                                   │                                          │
│                          ┌────────┴────────┐                                │
│                          │  Access Guard   │  ← JWT roles + RBAC scopes     │
│                          │  (Authentik)    │                                │
│                          └────────┬────────┘                                │
└───────────────────────────────────┼─────────────────────────────────────────┘
                                    │
        ┌───────────────────────────┼───────────────────────────┐
        │                           │                           │
        ▼                           ▼                           ▼
┌───────────────┐         ┌─────────────────┐         ┌─────────────────┐
│   pgvector    │         │   OpenSearch    │         │    Obsidian     │
│  (Semantic)   │         │   (Lexical)     │         │  (Reference)    │
│               │         │                 │         │                 │
│ HNSW cosine   │         │  BM25 ranking   │         │  Markdown docs  │
│ 768-dim vecs  │         │  Full-text      │         │  Wiki-links     │
└───────────────┘         └─────────────────┘         └─────────────────┘
        │                           │                           │
        └───────────────────────────┴───────────────────────────┘
                                    │
                          ┌─────────┴─────────┐
                          │  Wiki-Link Graph  │
                          │  PageRank boost   │
                          │  Provenance trace │
                          └───────────────────┘

Core Features

1. Graph-RAG Retrieval

Traditional RAG retrieves isolated chunks. Poimen builds a wiki-link graph from [[linked-documents]] and propagates relevance scores to connected knowledge.

Document A: "Kubernetes uses [[etcd]] for state storage"
Document B: "[[etcd]] requires TLS certificates"
Document C: "Generate certs with [[cfssl]]"

Query: "Kubernetes certificate issues"
  → Finds A (direct match)
  → Boosts B (linked from A)
  → Surfaces C (2-hop connection)

2. Three-Tier Context Lookup

Agents call /memory/context with tool + task + failure log. Poimen returns grounded knowledge in priority order:

Tier	Source	Latency	Use Case
Tier 1	Exact signature match	<50ms	Known error patterns
Tier 2	Graph-boosted hybrid search	<500ms	Similar problems
Tier 3	Reference corpus fallback	<1s	Documentation

curl -X POST /memory/context \
  -d '{"tool": "kubectl", "task": "debug-pod", "failure_log": "CrashLoopBackOff"}'

# Returns:
{
  "tier": 1,
  "lessons": [{
    "text": "CrashLoopBackOff: check container logs with kubectl logs -p",
    "seen_count": 23,
    "provenance": ["session-123", "session-456"]
  }]
}

3. Hallucination Prevention

Every retrieved chunk includes:

• provenance[] — Which sessions/documents contributed this fact
• source — Original file or conversation URI
• seen_count — How many times this pattern was observed
• score — Retrieval confidence (semantic + lexical + graph boost)

Agents can cite sources: "Based on 23 previous occurrences (source: troubleshooting/k8s.md)..."

4. Hierarchical RBAC

Two-level access control integrated with Authentik OIDC:

Level 1: Capabilities (HTTP endpoint access)

memory:read   → /query, /context, /projects, /skills
memory:write  → /ingest, /learn
*             → all endpoints (admin)

Level 2: Resource Scopes (fine-grained filtering)

Scope	Description	Example
projects	Allowed project names	[homelab, portfolio]
visibility	Public or private docs	public
owner	Resource ownership	self (own only)
groups	Required group membership	[engineering]

Built-in Roles:

# Admin: full access
- role: admin
  rules:
    - resources: ["*"]
      verbs: [read, write, delete, query]

# Portfolio visitor: public docs only, own conversations
- role: portfolio-agent
  rules:
    - resources: [wiki, embedding]
      verbs: [read, query]
      scope:
        projects: [homelab, portfolio]
        visibility: public
    - resources: [conversation]
      verbs: [read, write]
      scope:
        owner: self  # Can only access own conversations

# Authenticated user: all docs, own conversations
- role: authenticated-user
  rules:
    - resources: [wiki, embedding, skill]
      verbs: [read, query]
      # No visibility restriction → sees public + private
    - resources: [conversation]
      verbs: [read, write, delete]
      scope:
        owner: self

JWT Claims → RBAC:

{
  "sub": "alice",
  "roles": ["authenticated-user", "homelab-team"],
  "groups": ["engineering"],
  "permissions": ["memory:read", "memory:write"]
}

Agents only retrieve knowledge they're authorized to access. Results are filtered post-retrieval by AccessGuard.

5. Budget-Aware Context Assembly

LLM context windows are limited. Poimen optimizes what fits:

Budget: 8192 tokens
  │
  ├─ Tier 1 lessons (never dropped)     → 2000 tokens
  ├─ Tier 2 relevant chunks             → 4000 tokens
  ├─ Tier 3 reference excerpts          → 1500 tokens
  └─ Skills/tools                       →  500 tokens
                                        ────────────
                                          8000 tokens ✓

If over budget:
  1. Drop Tier 3 first
  2. Drop lowest-score Tier 2
  3. Compress remaining chunks
  4. Never drop Tier 1

Quick Start

Prerequisites

• Rust 1.75+
• PostgreSQL 15+ with pgvector extension
• OpenSearch 2.x
• (Optional) Authentik for OIDC

Run Locally

# Clone
git clone https://github.com/your-org/poimen-memory.git
cd poimen-memory

# Start dependencies
docker-compose up -d postgres opensearch

# Configure
cp .env.example .env
# Edit .env with your settings

# Build and run
cargo build --release
./target/release/mem serve

# Health check
curl http://localhost:8080/health

Docker

docker run -d \
  -e PGVECTOR_HOST=postgres:5432 \
  -e OPENSEARCH_HOST=opensearch:9200 \
  -p 8080:8080 \
  ghcr.io/your-org/poimen-memory:latest

Kubernetes (ArgoCD)

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: poimen-memory
spec:
  source:
    repoURL: https://github.com/your-org/poimen-memory
    path: k8s/app
  destination:
    namespace: poimen

API Usage

Ingest Knowledge

# From conversation
curl -X POST http://localhost:8080/memory/ingest \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "project": "homelab",
    "source": "conversation://claude/session-123",
    "records": [
      {"text": "To fix port 8080 conflict, use: kubectl delete pod -l app=nginx"},
      {"text": "etcd backup: etcdctl snapshot save /backup/etcd.db"}
    ]
  }'

Query Memory

# Hybrid search (semantic + lexical + graph)
curl "http://localhost:8080/memory/query?project=homelab&query=kubernetes%20port%20conflict" \
  -H "Authorization: Bearer $TOKEN"

# Response
{
  "results": [{
    "text": "To fix port 8080 conflict...",
    "score": 0.92,
    "source": "conversation://claude/session-123",
    "provenance": ["session-123"]
  }]
}

Get Agent Context

# Tool-specific context with failure diagnosis
curl -X POST http://localhost:8080/memory/context \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "project": "homelab",
    "tool": "kubectl",
    "task": "debug-pod",
    "failure_log": "Error: ImagePullBackOff",
    "budget": 4096
  }'

Learn from Documents

# Chunk, embed, and synthesize
curl -X POST http://localhost:8080/memory/learn \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "project": "homelab",
    "text": "# Kubernetes Networking\n\nPods communicate via [[CNI]] plugins...",
    "chunk_size": 2000
  }'

Retrieval Pipeline

Query: "fix kubernetes certificate error"
                    │
                    ▼
        ┌───────────────────────┐
        │   Query Optimizer     │
        │  Classify: bug_fix    │
        │  Route: hybrid        │
        └───────────┬───────────┘
                    │
        ┌───────────┴───────────┐
        │                       │
        ▼                       ▼
┌───────────────┐       ┌───────────────┐
│   Semantic    │       │    Lexical    │
│   pgvector    │       │  OpenSearch   │
│  cosine sim   │       │    BM25       │
└───────┬───────┘       └───────┬───────┘
        │                       │
        └───────────┬───────────┘
                    │
                    ▼
        ┌───────────────────────┐
        │     RRF Fusion        │
        │  60% semantic         │
        │  40% lexical          │
        └───────────┬───────────┘
                    │
                    ▼
        ┌───────────────────────┐
        │  Wiki-Link Graph      │
        │  PageRank boost       │
        │  Link-distance decay  │
        └───────────┬───────────┘
                    │
                    ▼
        ┌───────────────────────┐
        │   RBAC Filter         │
        │  Project scope        │
        │  Visibility check     │
        └───────────┬───────────┘
                    │
                    ▼
        ┌───────────────────────┐
        │  Deduplication        │
        │  Shingle Jaccard      │
        │  >0.5 = duplicate     │
        └───────────┬───────────┘
                    │
                    ▼
        ┌───────────────────────┐
        │  Budget Assembly      │
        │  Rank by score        │
        │  Fit to token limit   │
        └───────────┬───────────┘
                    │
                    ▼
            Final Results
         (with provenance)

Configuration

Environment Variables

Variable	Description	Default
PGVECTOR_HOST	PostgreSQL host	localhost:5432
PGVECTOR_DB	Database name	memory
OPENSEARCH_HOST	OpenSearch host	localhost:9200
OBSIDIAN_URL	Obsidian REST API	(optional)
MEM_AUTH_MODE	jwt or apikey	jwt
AUTHENTIK_ISSUER	OIDC issuer URL	(required for jwt)
RBAC_ROLES_DIR	Custom role definitions	(builtin only)

Custom Roles

# config/roles/my-team.yaml
name: my-team
rules:
  - resources: [wiki, embedding]
    verbs: [read, write, query]
    scope:
      projects: [my-project]
      visibility: private  # Can access private docs

Project Structure

poimen-memory/
├── crates/
│   ├── mem-cli/          # HTTP server, RBAC, handlers
│   │   └── src/
│   │       ├── http_server.rs
│   │       ├── rbac/     # Access control
│   │       ├── hybrid_retrieval.rs
│   │       └── query_optimizer.rs
│   ├── mem-core/         # Domain types, scoring
│   └── mem-ingest/       # Wiki-link parsing, chunking
├── config/
│   └── roles/            # YAML role definitions
├── docs/
│   ├── API.md            # API reference
│   └── RBAC.md           # Access control guide
├── k8s/                  # Kubernetes manifests
└── tests/                # Integration tests (670+)

Performance

Metric	Target	Actual
Tier 1 latency	<50ms	12ms
Hybrid search	<500ms	145ms
NDCG@10	>0.85	0.88
Test coverage	>600	670

Contributing

# Run tests
cargo test --all

# Run specific test
cargo test -p mem-cli http_server::tests

# Check formatting
cargo fmt --check
cargo clippy

License

MIT