Root causes of zero entity extraction:
1. IngestWorker used WikiLinkFallbackExtractor (wiki links only)
Fix: Use LlmEntityExtractor when LLM_ENDPOINT is set
2. ExtractedEntity.entity_type vs LLM returning "type"
Fix: serde alias "type" -> entity_type, default confidence
3. Reasoning models output <think>...</think> before JSON
Fix: strip_thinking_tags() extracts JSON from response
4. Reflection verification crashes pipeline on parse failure
Fix: graceful fallback, keep all entities if reflection fails
Tested with reasoning-predictor (qwen2.5:3b) via port-forward.
Input: "Kubernetes uses etcd as its backing store. ArgoCD deploys applications to Kubernetes via GitOps. Rock manages the homelab cluster with Talos Linux."
Entities Extracted (6)
name | entity_type
-------------+-------------
ArgoCD | tool
GitOps | concept
Kubernetes | tool
Rock | person
Talos Linux | tool
etcd | tool
Edges Extracted (4)
source | target | relation | fact
----------------+---------------+--------------+------------------------------------------------------------
Kubernetes | etcd | USES | Kubernetes uses etcd as its backing store.
ArgoCD | Kubernetes | DEPLOYS_TO | ArgoCD deploys applications to Kubernetes via GitOps.
ArgoCD | GitOps | USES | ArgoCD uses the GitOps methodology to deploy applications.
Rock | Talos Linux | MANAGES | Rock manages a homelab cluster using Talos Linux.
- Remove obsidian.yaml deployment
- Remove OBSIDIAN_URL from configmap
- Remove from kustomization.yaml
- obsidian_ref_source.rs kept as dead code (no callers)
- Reference docs now handled via memory graph entities
- Scaled obsidian-server to 0 in cluster
Triggers on:
- Push to main when crates/mem-store/migrations/*.sql changes
- Manual workflow_dispatch (runs ALL migrations)
On push: detects changed migration files, runs only those.
On dispatch: runs all migrations in order (idempotent).
Requires DB_USER + DB_PASSWORD secrets in Forgejo.
Connects to memory-db-rw.poimen.svc.cluster.local.
All migrations use IF NOT EXISTS / IF EXISTS guards.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Changes
Entity Extraction
clean_llm_response(): strips<think>tags, markdown fences, extracts JSON[...]not{entities: [...]})Fact Extraction (NEW)
K8s Deployment
command: ["/app/mem"](fix args replacing CMD)E2E Tested (local Ollama qwen2.5:3b)
Zep Paper Alignment (§2.2)
Entity extraction fixes: - clean_llm_response() strips <think> tags, markdown fences, extracts JSON - Handle array responses (wrap in {"entities": [...]}) - EntityType custom Deserialize: unknown variants map to Unknown (not crash) - Increase timeout to 90s for reasoning models - Increase max_tokens to 1500 for reasoning model overhead Fact extraction (new): - LlmFactExtractor: LLM-based relationship extraction between entities - Validates source/target against known entity list (no hallucinated edges) - Same clean_llm_response() for reasoning model + Ollama compatibility - Graceful fallback: returns empty on LLM error (no pipeline crash) - IngestWorker uses LlmFactExtractor when LLM_ENDPOINT set K8s deployment: - Add LLM_ENDPOINT, LLM_API_BASE, LLM_MODEL env vars - Points to in-cluster reasoning-predictor service Tested E2E with local Ollama (qwen2.5:3b): - 12 entities extracted (person, tool, concept, organization) - 5 edges with meaningful relationships and facts - 781 tests passE2E Proof — ornith:35b via api.riotpiao.com
Ingest
Input:
"Kubernetes uses etcd as its backing store. ArgoCD deploys applications to Kubernetes via GitOps. Rock manages the homelab cluster with Talos Linux."Entities Extracted (6)
Edges Extracted (4)
Query
Visualize
Observability Logs
Auth Flow
Token scope:
openid roles→ roles includellm:inferencePipeline Summary
All components now emit target="observability" structured logs: chunk_optimizer: event=chunk_optimize: input, after_threshold_filter, after_dedup, dedup_removed, selected, budget_bytes result_compressor: event=result_compress: input_count, estimated_bytes, compressed_bytes, budget_bytes, strategy query_router: event=query_route: route, candidates, prefiltered, selected, latency_ms cache_alignment: event=cache_preload: preloaded, cache_hits, cache_misses, hit_ratio full_pipeline: event=full_pipeline_complete: query, candidates, prefiltered, optimized, dedup_removed, boosts_applied, cache_hit_ratio, budget_bytes, total_ms compaction: event=compaction_complete: mode, duration_ms, duplicate_edges_deleted, stale_facts_deleted, semantic_merged, llm_calls, bytes_freed 781 tests pass.E2E Proof — Homelab CLAUDE.md Ingest + Query + Visualize (local qwen2.5:3b)
Ingest: 3 records from homelab/CLAUDE.md
Entities Extracted (15)
Edges Extracted (5)
Query
Visualize
Observability Logs (all
target="observability")Entity extraction (6 LLM calls, 3 records × 2 calls each):
Fact extraction (3 LLM calls):
Ingest complete:
Entity Upsert Dedup
ON CONFLICT (project_id, name) DO UPDATE— re-ingesting same content merges, incrementssource_count.Known Gaps
column "fact_invalid_at" does not exist— compaction SQL references old schema column