11 Commits
Author SHA1 Message Date
poimenandrock fb61de6b47 feat: LLM entity + fact extraction pipeline (Zep paper alignment) (#48)
CI / CI (push) Successful in 12m9s
Deploy / Tag & Push Latest (push) Failing after 41s
DB Migration / Run Migrations (push) Failing after 18s
## Changes

### Entity Extraction
- Switch from WikiLinkFallbackExtractor to LlmEntityExtractor when LLM_ENDPOINT set
- `clean_llm_response()`: strips `<think>` tags, markdown fences, extracts JSON
- Handle array responses (Ollama returns `[...]` not `{entities: [...]}`)
- EntityType custom Deserialize: unknown variants → Unknown (no crash)
- Increase timeout 30s→90s, max_tokens 500→1500 for reasoning models
- Graceful reflection fallback: keep entities if verification fails

### Fact Extraction (NEW)
- LlmFactExtractor: LLM-based relationship extraction between entity pairs
- Validates source/target against known entity list (drops hallucinated edges)
- Same robust JSON cleaning for reasoning models + Ollama
- IngestWorker auto-selects LLM vs Simple based on LLM_ENDPOINT env

### K8s Deployment
- Add `command: ["/app/mem"]` (fix args replacing CMD)
- Add LLM_ENDPOINT, LLM_MODEL env vars for in-cluster LLM

## E2E Tested (local Ollama qwen2.5:3b)
- 12 entities extracted (person, tool, concept, organization)
- 5 edges with relationships and facts
- 781 tests pass

## Zep Paper Alignment (§2.2)
- Entity extraction + resolution (§2.2.1)
- Fact extraction between entity pairs (§2.2.2)
- Temporal edge invalidation ready (t_valid/t_invalid schema)
- Reflection verification (§2.2.1, graceful fallback)

---------

Co-authored-by: rock <[email protected]>
Reviewed-on: #48
Co-authored-by: poimen <[email protected]>
2026-09-11 01:11:15 +00:00
rock 2501a68528 fix: add PodSecurity contexts to all poimen deployments
- runAsNonRoot, runAsUser 1000, seccompProfile RuntimeDefault
- Drop ALL capabilities, no privilege escalation
- readOnlyRootFilesystem on memory (with /tmp emptyDir)
- git-sync init runs as root with only CHOWN+DAC_OVERRIDE caps
- All pods use their service accounts
2026-08-30 07:20:08 -07:00
rock 10d7a0be77 fix: chown vault to uid 1000 after git-sync (obsidian runs as 1000) 2026-08-28 20:44:38 -07:00
rock c1167563b1 fix: add safe.directory for git-sync init container 2026-08-28 20:43:40 -07:00
rock 82507cf2a3 fix: move obsidian vault PVC to homelab repo (infra-managed) 2026-08-28 20:42:18 -07:00
rock 23019fdb27 fix: obsidian vault PVC ReadWriteMany for shared access 2026-08-28 20:28:37 -07:00
rock a60c74fc78 fix: move obsidian ingress to homelab repo, use obsidian.riotpiao.com
vault.riotpiao.com was already taken by HashiCorp Vault.
Ingress now managed centrally in homelab/k8s/bootstrap/ingress/ingress.yaml
2026-08-28 16:43:36 -07:00
rock 2ddd2d6cdf fix: remove broken auth annotations from obsidian ingress
Bearer auth-url was misconfigured (pointed to token endpoint, not
forward-auth). No Authentik outpost deployed yet. Remove for now,
vault.riotpiao.com accessible directly. TODO: add forward-auth
once outpost is set up.
2026-08-28 16:42:00 -07:00
rock b94898d0d4 feat: obsidian git-sync from poimen-obesdient-memory repo
- Add git-sync init container to clone/pull vault content
- Add SOPS-encrypted SSH deploy key (obsidian-git-ssh-secret.enc.yaml)
- Add .sops.yaml config (age encryption, same key as homelab)
- Repo: ssh://[email protected]:2222/rock/poimen-obesdient-memory.git
- Deploy key added to Forgejo repo (read-only)
2026-08-28 16:31:48 -07:00
Story Crater Bot 69ec8aeec2 fix: Obsidian service port and health checks, use Longhorn storage 2026-08-27 21:37:47 -07:00
Story Crater Bot 0eecca815b refactor: replace Obsidian projector with standalone service (ppatlabs/obsidian) 2026-08-27 21:35:07 -07:00