poimen-memory/tasks/M7.2-obsidian-connector.md

M7.2 — Obsidian vault connector

Field	Value
Phase	M7 — Source connectors
Size	M — 1–3 days
Status	⬜ Not started
Flags	—
Spec	inlined below
Blocks	M7.10
Depends	M7.1, M3.6.1

Goal

Wrap the existing DocCorpusSource (M3.6.1) in the SourceConnector interface so the Obsidian vault is managed through the unified connector framework — with change detection, config-driven setup, and registry integration.

Facts (inlined — no spec read needed)

The Obsidian vault is the primary reference source for the cluster. It is human-maintained, optionally git-backed, and contains runbooks, procedures, and domain knowledge that agents query through the memory service.

DocCorpusSource already handles markdown parsing, heading-boundary chunking, breadcrumb paths, and file filtering. This task wraps it, not rewrites it.

Deployment models:

1. Git-sync sidecar — a sidecar container clones the vault repo into a shared PVC. Memory service reads from the PVC via this connector.
2. Local mount — for development, mount the vault directory directly.
3. PVC direct — vault files managed via kubectl cp or a web uploader.

Configuration:

connectors:
  - kind: obsidian
    name: homelab-vault
    config:
      root: /data/vault
      extensions: [md, markdown, txt]
      exclude_dirs: [.obsidian, .trash, .git]
      max_file_size: 10485760  # 10MB

Steps

1. Implement ObsidianConnector in mem-ingest/src/connectors/obsidian.rs.
2. list_documents() — walk root directory, filter by extension, compute sha256 per file, return SourceDocument per file.
3. fetch_document() — read file content, return as DocumentContent with metadata (file path, last modified, size).
4. health_check() — verify root exists, is readable, count files.
5. Register "obsidian" kind in the connector registry factory.
6. source_type() returns Reference (vault docs bypass the gated loop).
7. Reuse DocCorpusSource internals for heading-based chunking when the sync framework (M7.6) processes this connector's documents.

Acceptance

• ObsidianConnector implements SourceConnector fully.
• list_documents() respects extensions, exclude_dirs, max_file_size.
• fetch_document() returns content matching file on disk.
• health_check() distinguishes readable vs. missing root directory.
• Config-driven: changing root path in YAML changes what gets scanned.

Verify

Harness: fixture directory with markdown files, config YAML.

Integration test — tests/it_obsidian_connector.rs:

1. a1_list_filters_extensions — fixture with .md, .txt, .json; assert only .md and .txt are listed.
2. a2_list_excludes_dirs — fixture with .obsidian/ subdir; assert its files are excluded.
3. a3_fetch_returns_content — fetch a known doc; assert text matches file.
4. a4_fetch_unknown_errors — fetch non-existent doc_id; assert error.
5. a5_health_check_reachable — valid root; assert reachable: true with count.
6. a6_health_check_missing_root — non-existent root; assert reachable: false.
7. a7_content_hash_stable — fetch same file twice; assert same hash.
8. a8_config_from_yaml — parse connector from YAML; assert fields match.

Command: cargo test --test it_obsidian_connector

False pass:

• Testing with an empty directory. Assertions 1–3 need real files.
• Not testing exclude_dirs with nested paths (.obsidian/plugins/x.md).

Traps

• Re-implementing markdown parsing instead of delegating to DocCorpusSource.
• Making doc_id platform-dependent (use relative path from root, unix separators).
• Ignoring symlinks — Obsidian uses them for multi-vault setups.

---

Background: DESIGN.md — source connectors, Obsidian vault as connector