chore: retire M3.6.3 (mem ref CLI), update M3.6.2 to use Obsidian REST API

CHANGES:
- M3.6.3: marked ❌ RETIRED (Obsidian UI replaces CLI corpus management)
- M3.6.2: updated to fetch from Obsidian REST API instead of filesystem
  - ObsidianRefSource: calls /api/vault/listFiles, /api/vault/readFile
  - Users manage corpus in Obsidian UI (not via CLI)
  - Rebuild auto-syncs by re-fetching and comparing file SHAs
  - No separate chunk-level diff CLI needed
- Updated INDEX.md:
  - M3.6.x: 6 tasks → 5 tasks (removed M3.6.3)
  - Progress: 1 ✅, 0 🟡, 5 ⬜ → 1 ✅, 0 🟡, 4 ⬜
  - Total: 71 tasks → 70 tasks
  - Noted M3.6.3 retirement in board description

RATIONALE:
- Obsidian is single source of truth (REST API)
- Users already use Obsidian UI for vault management
- No need for parallel CLI when vault is the interface
- M3.6.2 handles sync via deterministic SHA comparison
- Reduces feature bloat, cleaner architecture

tasks/INDEX.md

@@ -64,15 +64,15 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked
 | 4 | L2 synthesis + retrieval | M3.x | 4 | 4 | 0 | 0 | ✅ M3.4 |
 | 4.5 | Distributed API Layer | M3.5.x | 10 | 10 | 0 | 0 | ✅ M3.5.8 |
 | 5 | Skills | M4.x | 3 | 2 | 0 | 1 | ⬜ M4.3 |
-| 5.5 | Reference corpora | M3.6.x | 6 | 1 | 0 | 5 | ⬜ M3.6.6 |
+| 5.5 | Reference corpora | M3.6.x | 5 | 1 | 0 | 4 | ⬜ M3.6.6 |
 | 5.6 | Tool context | M3.7.x | 4 | 2 | 0 | 2 | ⬜ M3.7.6 |
 | 6 | Post-training | M5.x | 6 | 0 | 0 | 6 | ⬜ M5.6 |
 | 7 | agent-manager migration | M6.x | 6 | 0 | 0 | 6 | ⬜ M6.6 |
 | 8 | Source connectors | M7.x | 10 | 0 | 0 | 10 | ⬜ M7.10 |
 | 9 | Hybrid search | M8.x | 9 | 1 | 1 | 7 | ⬜ M8.9 |
-| | **Total** | | **71** | **60** | **0** | **11** | 6/11 green |
+| | **Total** | | **70** | **60** | **0** | **10** | 6/11 green |
 
-**Current status — 2025-01-28.** Completed phases M0.x, M1.x fully archived (16/16 tasks). **M2.1-6 ✅** (embeddings, CNPG, schema, pgvector, obsidian projector, rebuild). **M3.x ✅** (4/4). **M3.5.x ✅** (10/10 complete + archived). **M4.1-2 ✅** (skill drafting + derived filter). **M3.6.1 ✅** (DocCorpusSource). **M8.1 🟡** (OpenSearch cluster deploying — security context fixes in progress).
+**Current status — 2025-01-28.** Completed phases M0.x, M1.x fully archived (16/16 tasks). **M2.1-6 ✅** (embeddings, CNPG, schema, pgvector, obsidian projector, rebuild). **M3.x ✅** (4/4). **M3.5.x ✅** (10/10 complete + archived). **M3.7.7-8 ✅** (failure diagnosis). **M4.1-2 ✅** (skill drafting + derived filter). **M3.6.1 ✅** (DocCorpusSource). **M3.6.3 ❌ retired** (Obsidian UI replaces CLI). **M8.1 🟡** (OpenSearch cluster deploying — security context fixes in progress).
 
 **Current work:**
 - M8.1: OpenSearch StatefulSet (pod security baseline, fsGroup perms) — deploying
@@ -86,7 +86,7 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked
 **Blocked until M8.2 green:**
 - M8.3-9 (query optimizer, RRF, hybrid endpoint, benchmarks)
 - M3.7.4 (context endpoint needs hybrid search to rank tiers) — now has M3.7.7 + M3.7.8 ✅
-- M3.6.2+ (reference corpus storage depends on query layer)
+- M3.6.2+ (reference corpus indexing depends on dual-write layer for Postgres + OpenSearch)
 
 **Tests: 265+ passing, 2 ignored** (M2.1 +8, M3.5 +16, M4 +20, M3.7.7 +18 unit, M3.7.8 +22 integration). **60/71 tasks complete (85%)**, **6/11 gates green**. M3.7.3 & M3.7.5 retired (hybrid search covers). **M3.7.7 & M3.7.8 ✅ complete** (signature extraction + symptom projection, 40+ tests passing).
 
@@ -137,25 +137,32 @@ Homelab frontend integration: HTTP facade via `api.riotpiao.com`. Runs in parall
 
 ## 5.5 — Reference corpora · M3.6.x
 
-Documentation the local models are weak at — `kubectl`, `tea` — made retrievable
+**Source of truth: Obsidian vault** (REST API, deployed M2.5). Documentation
-as level **R**: embedded and indexed, never evidence. Ids are `M3.6.x` and stay
+the local models are weak at — `kubectl`, `tea` — are stored in Obsidian and
-`M3.6.x`; the phase sits here rather than at 4.6 because [M3.6.4](M3.6.4-reference-cycle-guard.md)
+made retrievable as level **R**: embedded and indexed, never evidence.
-extends M4.2's matcher instead of duplicating it, and because skills are the
-better answer to the same problem and should exist first.
 
-**The load-bearing property is a negative one.** Adding a corpus must not change
+Ids are `M3.6.x` and stay `M3.6.x`; the phase sits here rather than at 4.6 because
-update-rate, must not change default query output, and must not put an R node in
+[M3.6.4](M3.6.4-reference-cycle-guard.md) extends M4.2's matcher instead of
-any provenance chain. R bypasses the recurrence structurally — `run_loop` needs a
+duplicating it, and because skills are the better answer to the same problem and
-`Query` and a corpus has none — not by a flag. [M3.6.6](M3.6.6-m3.6-gate.md)
+should exist first.
-asserts M1.8's numbers are *unchanged*, not merely still-passing, because
+
-documentation fed to the gate would lower update-rate and make M1.8 easier to
+**The load-bearing property is a negative one.** Adding Obsidian vault reference
-clear while the memory got worse.
+documents must not change update-rate, must not change default query output, and
+must not put an R node in any provenance chain. R bypasses the recurrence
+structurally — `run_loop` needs a `Query` and a corpus has none — not by a flag.
+[M3.6.6](M3.6.6-m3.6-gate.md) asserts M1.8's numbers are *unchanged*, not merely
+still-passing, because documentation fed to the gate would lower update-rate and
+make M1.8 easier to clear while the memory got worse.
+
+**M3.6.3 retired:** Obsidian UI replaces CLI corpus management. Users edit files
+in Obsidian; `mem rebuild` auto-fetches from Obsidian REST API and re-indexes only
+changed chunks (SHA comparison, deterministic embedding).
 
 | Task | Title | Size | Flags | Status |
 |---|---|---|---|---|
 | M3.6.1 | `DocCorpusSource` + heading chunking | M | — | ✅ |
-| [M3.6.2](M3.6.2-level-r-storage.md) | Level R: log, index, vault, rebuild parity | M | — | ⬜ |
+| [M3.6.2](M3.6.2-level-r-storage.md) | Level R: Obsidian reference indexing + rebuild parity | M | — | ⬜ |
-| [M3.6.3](M3.6.3-mem-ref-cli.md) | `mem ref` — replace-on-change corpus management | M | — | ⬜ |
+| M3.6.3 | `mem ref` — corpus management CLI | M | — | ❌ RETIRED (Obsidian UI replaces) |
 | [M3.6.4](M3.6.4-reference-cycle-guard.md) | Reference text cannot re-enter as evidence | M | — | ⬜ |
 | [M3.6.5](M3.6.5-query-levels-and-floor.md) | Query: filter-then-recall, R opt-in, floor | M | — | ⬜ |
 | [M3.6.6](M3.6.6-m3.6-gate.md) | **M3.6 composition gate** | M | gate | ⬜ |

tasks/M3.6.2-level-r-storage.md

@@ -1,4 +1,4 @@
-# M3.6.2 — Level R: log record, index rows, vault notes, rebuild parity
+# M3.6.2 — Level R: Obsidian reference indexing and rebuild parity
 
 | Field | Value |
 |---|---|
@@ -8,116 +8,172 @@
 | Flags | — |
 | Spec | inlined below |
 | Blocks | M3.6.6 |
-| Depends | M3.6.1, M1.6, M2.3, M2.4, M2.5, M2.6 |
+| Depends | M3.6.1, M1.6, M2.3, M2.4, M2.5, M2.6, Obsidian service (M2.5) |
 
 ## Goal
 
-Land reference chunks in the log as their own record kind, project them into
+Fetch reference documents from the live Obsidian vault (REST API), chunk them
-Postgres and the vault, and prove the projections are still throwaway.
+via M3.6.1's heading-boundary logic, land them in the log as reference records,
+index them in Postgres, and prove rebuild parity (drop + rebuild from log =
+byte-identical index state).
 
 ## Facts (inlined — no spec read needed)
 
-```jsonl
+**Obsidian is the source of truth for reference documents.** The Obsidian REST API
-{"kind":"reference","level":"R","project":"homelab","source":"file:///.../kubectl.md",
+(deployed in M2.5, accessible at `http://obsidian-server.poimen.svc.cluster.local:8080`)
- "heading_path":"kubectl.md > Common Issues > CrashLoopBackOff","doc_sha":"ab12…",
+provides live access to vault files via:
- "sha256":"cd34…","t":7,"run_id":"ref-2026-08-21T10:02:11Z","text":"…"}
+
+```bash
+GET /api/vault/listFiles                          # List all .md files
+GET /api/vault/readFile?path=kubectl.md           # Read file contents
+GET /api/vault/listFolders                        # Explore structure
 ```
 
-**No migration is needed.** `'R'` ships in M2.3's initial schema, along with the
+**Reference record format (in JSONL log):**
-`CHECK ((level IN ('L2','R')) = (query_id IS NULL))` constraint. Nothing was built
+
-before this phase existed, so the level was never absent from the schema and an
+```jsonl
-`ALTER` here would only undo a deliberate omission that was never made.
+{"kind":"reference","level":"R","source":"obsidian://poimen-vault/kubectl.md",
+ "doc_sha":"ab12…","heading_path":"kubectl.md > Common Issues > CrashLoopBackOff",
+ "sha256":"cd34…","t":7,"run_id":"ref-obsidian-2026-08-21T10:02:11Z","text":"…"}
+```
+
+**No vault projection.** Unlike M3.6.1 (DocCorpusSource which reads from local filesystem),
+M3.6.2 reads directly from Obsidian REST API and indexes into Postgres. The Obsidian
+vault remains the single source of truth; `mem rebuild --from-log` re-fetches from
+Obsidian to regenerate indexes.
 
 `level = 'R'`, `query_id = NULL` (R answers no standing question), `source` holds
-the source URI. `doc_sha` is the whole-document hash; `sha256` is the chunk hash
+the Obsidian URI (`obsidian://vault-name/file.md`). `doc_sha` is whole-document hash;
-and stays the primary identity, same as every other level.
+`sha256` is chunk hash (primary identity).
 
 The embedding goes to `memory_vector(kind='text')`, not to a column on the node.
-R gets **no symptom projection** — M3.7.8 generates those for L1 and L2 only,
+R gets **no symptom projection** — M3.7.8 generates those for L1 and L2 only.
-since documentation headings already read like problems.
 
-**R writes no edges.** Not to parents, not to siblings. A reference chunk has no
+**R writes no edges.** Provenance is the Obsidian URI, not internal edges. Enforced
-provenance inside this system — its provenance is the URI. The rule that makes
+in `mem verify` (M3.6.4).
-this safe is enforced in `mem verify` (M3.6.4), but nothing in this task should
-ever be tempted to create an edge in the first place.
 
-**Vault projection goes somewhere separate.** `vault/reference/<corpus>/<doc>.md`,
+**Rebuild parity is the whole point.** `mem rebuild --from-log` must:
-not into the project notes. The vault is browsed by a human; interleaving
+1. Drop R rows from Postgres
-upstream docs with synthesized project memory makes the vault untrustworthy at a
+2. Drop R vectors from OpenSearch (M8.2)
-glance. One note per source document, sections as headings, each carrying its
+3. Re-fetch documents from Obsidian REST API
-chunk sha as an anchor so `mem query` output can deep-link.
+4. Re-chunk via M3.6.1 heading logic
-
+5. Re-index into Postgres + OpenSearch
-**Rebuild parity is the whole point of the task.** `mem rebuild --from-log` must
+6. Result must be byte-identical (same shas, same vector embeddings)
-drop and reconstruct R rows and R notes byte-identically. If it cannot, R has
-hidden inputs and rule 3 of the design is broken — M2.8 already enforces this
-property for L0/L1/L2 and this task extends the same harness rather than writing
-a second one.
 
 ## Steps
 
-1. Add the `Reference` variant to the log record enum in `mem-core`; serialize
+1. **ObsidianRefSource** in `mem-ingest`: Implement `ChunkedSource` that:
-   with the field set above.
+   - Calls Obsidian REST API (`/api/vault/listFiles`)
-2. `mem-store`: insert R nodes with a single `kind='text'` vector; assert at the
+   - Filters for `.md` files in allowed paths (e.g., `docs/`, `reference/`)
-   repository boundary that no edge insert names an R sha as parent.
+   - Fetches each file via `/api/vault/readFile?path=...`
-4. Obsidian projector: `vault/reference/<corpus>/<doc>.md`, one note per source
+   - Chunks via M3.6.1's heading boundary logic
-   document, chunk shas as heading anchors.
+   - Yields `Record { kind: Reference, ... }`
-5. Extend `mem rebuild --from-log` to replay `Reference` records.
+
-6. Extend the M2.6 rebuild-parity harness to cover a log containing R records.
+2. Add `Reference` variant to log record enum in `mem-core` (already exists from
+   M2.3, just needs the Obsidian source)
+
+3. `mem-store`: Insert R nodes with `kind='text'` vector; assert no edge insert
+   names an R sha as parent (repository boundary check).
+
+4. Dual-write in M8.2: When R records are indexed, write both to Postgres and
+   OpenSearch (L2 already does this; extend for R).
+
+5. **Rebuild: extend M2.6 harness** for `mem rebuild --from-log`:
+   - When replaying R records from log, re-fetch original files from Obsidian
+   - Re-chunk via M3.6.1 logic
+   - Regenerate embeddings (deterministic, so shas match)
+   - Assert byte-identical state vs. original indexing
+
+6. Integration: Update `mem query` to include R results from hybrid search (M8)
 
 ## Acceptance
 
-- A `Reference` record round-trips through the log unchanged.
+- `ObsidianRefSource` fetches all `.md` files from Obsidian REST API.
-- R rows land with `query_id IS NULL` and `source` set to the URI.
+- Reference records (R) with `source=obsidian://...` land in the log.
-- The widened constraint accepts `R` and still rejects `L3`.
+- R rows persist in Postgres with `query_id IS NULL` and `source` set to Obsidian URI.
-- Reference notes land under `vault/reference/`, never in project note dirs.
+- Each R chunk gets a `kind='text'` embedding in OpenSearch.
-- Drop database + vault, `mem rebuild --from-log`, and both come back
+- The M2.3 constraint accepts `R` and rejects `L3`.
-  byte-identical.
+- `mem rebuild --from-log`: drop Postgres R rows + OpenSearch R vectors, re-fetch
+  from Obsidian, re-chunk, re-embed, re-index; result is byte-identical (same shas,
+  same embedding vectors).
+- No hidden inputs: Obsidian REST API is the only external dependency for R.
 
 ## Verify
 
-**Harness:** the M2.6 rebuild harness, extended with a log fixture that contains
+**Harness:** Mock Obsidian REST API (deterministic file list + content). M2.6 rebuild
-L0/L1/L2 *and* R records. Deterministic fake embedder so shas are stable.
+harness extended to cover R records. Deterministic embedder so shas are stable.
 
-**Integration test** — `tests/it_level_r_storage.rs`:
+**Integration test** — `tests/it_obsidian_ref_source.rs` + `tests/it_level_r_storage.rs`:
-1. `a1_record_roundtrip` — serialize then deserialize a `Reference` record;
-   assert field-for-field equality including `doc_sha` and `heading_path`.
-2. `a2_r_inserts` — insert `level='R'` with a `kind='text'` vector; assert both
-   rows persist.
-3. `a3_no_symptom_vector` — assert no R node acquires a `kind='symptom'` vector
-   after a full ingest.
-4. `a4_query_id_null_at_r` — assert every R row has `query_id IS NULL`, and that
-   an R row with one is rejected by M2.3's CHECK.
-5. `a5_no_edges_from_r` — after ingesting the fixture corpus, assert
-   `SELECT count(*) FROM memory_edge WHERE parent_sha IN (SELECT sha256 FROM
-   memory_node WHERE level='R')` is 0.
-6. `a6_vault_path_isolation` — assert every emitted reference note path starts
-   with `vault/reference/` and no project note directory gained a file.
-7. `a7_rebuild_byte_identical` — snapshot database rows and vault files, drop
-   both, `mem rebuild --from-log`, assert byte-identical including R.
-8. `a8_rebuild_is_idempotent` — rebuild twice; assert the second run changes
-   nothing.
 
-**Command:** `cargo test -p mem-store level_r && cargo test -p mem-cli rebuild`
+### ObsidianRefSource tests:
+1. `a1_fetches_md_files` — Mock API returns 3 `.md` files; source emits 3 documents.
+2. `a2_chunks_by_heading` — One source document with 5 headings yields 5 chunks
+   (uses M3.6.1 heading boundary logic).
+3. `a3_generates_doc_sha` — Document SHA (SHA256 of whole file content) is
+   consistent across fetches.
+4. `a4_generates_chunk_shas` — Each chunk sha is deterministic: SHA256(tool +
+   heading_path + text).
+5. `a5_sets_obsidian_uri` — Every R record has `source=obsidian://vault-name/path.md`.
+6. `a6_skips_non_md` — API returns `.json` and `.txt` files; source ignores them.
+
+### Level R storage tests:
+7. `a7_record_roundtrip` — Serialize R record to JSONL, deserialize; assert
+   field-for-field equality.
+8. `a8_r_inserts_to_postgres` — Insert `level='R'` with `kind='text'` vector;
+   assert Postgres row exists with correct `source` URI.
+9. `a9_r_inserts_to_opensearch` — Same chunk indexed in OpenSearch (M8.2
+   dual-write); assert M8 vector store has the chunk.
+10. `a10_no_symptom_projection` — R chunks do not acquire `kind='symptom'`
+    vectors (only L1/L2 get those from M3.7.8).
+11. `a11_query_id_null` — Every R row has `query_id IS NULL`; insertion with
+    non-NULL query_id is rejected by M2.3's CHECK.
+12. `a12_no_edges_from_r` — After ingesting 10 R chunks, `SELECT count(*)
+    FROM memory_edge WHERE parent_sha IN (SELECT sha256 FROM memory_node WHERE
+    level='R')` equals 0.
+
+### Rebuild parity tests:
+13. `a13_rebuild_drops_r_rows` — Ingest R, assert N rows exist. Call `mem
+    rebuild --from-log`, assert same N rows exist (re-fetched, re-embedded,
+    re-indexed).
+14. `a14_rebuild_byte_identical` — Snapshot Postgres R rows + OpenSearch R
+    vectors before rebuild. Drop both. Run `mem rebuild --from-log` against
+    a log containing R records. Assert Postgres rows and OpenSearch vectors
+    are byte-identical (same row order, same field values, same vector embeddings).
+15. `a15_rebuild_idempotent` — Run rebuild twice; second run changes nothing.
+    Snapshot comparison between first and second rebuild result is identical.
+
+**Command:** 
+```bash
+cargo test -p mem-ingest obsidian_ref_source
+cargo test -p mem-store level_r
+cargo test -p mem-cli rebuild  # includes R records in fixture log
+```
 
 **False pass:**
-- Asserting rebuild parity on a log with no R records. It passes trivially and
+- Asserting rebuild parity on a log with no R records. Assertion 14 requires
-  proves nothing about this task; assertion 7 is only meaningful because the
+  mixed-level log (L0/L1/L2 *and* R) or it trivially passes.
-  fixture log is mixed-level.
+- Mocking Obsidian API with hardcoded responses. File hashes must match real
-- Checking edge count is zero *before* ingesting anything. Assertion 5 has to run
+  Obsidian vault file SHA256 (use actual Obsidian instance or hash fixtures).
-  against a populated corpus or it is asserting that an empty table is empty.
+- Checking edge count is zero before ingesting. Assertion 12 must run against
-- Comparing vault files with a normalizing diff. Byte-identical means bytes;
+  populated corpus or it asserts empty table is empty.
-  trailing-newline drift is exactly the class of hidden input this rule exists
+- Byte-identical comparison with normalization (sorting, ignoring order).
-  to catch.
+  Byte-identical means bytes; rebuild parity is broken if row order changes
+  between runs.
 
 ## Traps
 
 - Reusing `run_id` semantics from the gated loop. R has no run in the recurrence
-  sense; use a synthetic `ref-<timestamp>` and do not let it collide with a real
+  sense; use a synthetic `ref-obsidian-<timestamp>` and do not let it collide
-  ingest run in queries that group by `run_id`.
+  with a real ingest run in queries that group by `run_id`.
-- Putting reference notes in the project vault "just for now". The vault is the
+- Caching Obsidian API responses across rebuild runs. Rebuild must re-fetch from
-  human surface and the mixing is not reversible by a later move — links written
+  Obsidian REST API every time (no cache) to ensure file contents and shas are
-  against the old path rot.
+  always in sync with live vault.
-- Dropping the check constraint instead of widening it. Assertion 3 exists
+- Assuming Obsidian file list is sorted. `listFiles` order is arbitrary; source
+  must sort filenames before chunking to ensure deterministic shas across runs.
+- Embedding each R chunk independently. M3.6.1 (DocCorpusSource) uses the same
+  `m` parameter (batching) and deterministic embedder model to ensure chunk shas
+  are stable; M3.6.2 must use identical setup.
+- Dropping the check constraint instead of widening it. Assertion 11 exists
   because `DROP CONSTRAINT` alone passes every other assertion in this file.
 
 ---

tasks/M3.6.3-mem-ref-cli.md

@@ -1,83 +1,107 @@
-# M3.6.3 — `mem ref` — corpus management with replace-on-change
+# M3.6.3 — RETIRED: `mem ref` CLI (replaced by Obsidian UI)
 
 | Field | Value |
 |---|---|
 | Phase | M3.6 — Reference corpora |
 | Size | M — 1–3 days |
-| Status | ⬜ Not started |
+| Status | ❌ RETIRED |
 | Flags | — |
-| Spec | inlined below |
+| Spec | inlined below (historical) |
-| Blocks | M3.6.6 |
+| Blocks | — |
-| Depends | M3.6.2, M2.1 |
+| Depends | — |
 
-## Goal
+## Retirement Rationale
 
-Add, list, refresh and remove reference corpora, so that re-running an ingest
+**Obsidian UI replaces CLI corpus management.**
-against changed upstream docs replaces what is there instead of stacking a second
-copy beside it.
 
-## Facts (inlined — no spec read needed)
+With Obsidian vault deployed (M2.5 complete) as the source of truth for reference
+documents, the CLI reference management commands are now redundant:
 
-```
+- **Document modification:** Users edit files in Obsidian UI (web/desktop)
-mem ref add --project homelab --corpus kubectl ~/workplace/homelab/knowledge/cheatsheets
+- **Corpus management:** Obsidian vault (filesystem) is the authoritative store
-mem ref add --dry-run ...            # chunk plan only, zero model calls (M3.6.1)
+- **Synchronization:** M3.6.2 (ObsidianRefSource) automatically fetches from
-mem ref list --project homelab       # corpus, docs, chunks, last ingest, drift
+  Obsidian REST API on every rebuild
-mem ref sync --corpus kubectl        # re-walk, replace changed docs, report
+- **Change detection:** Rebuild compares file SHA256 hashes and re-chunks/re-embeds
-mem ref rm --corpus kubectl          # tombstone every doc in the corpus
+  only changed documents
+
+**What M3.6.3 did (historical):**
+
+## Historical Functionality (M3.6.3 commands, now retired)
+
+```bash
+# These commands are no longer needed:
+mem ref add --project homelab --corpus kubectl ~/path/to/docs
+mem ref list --project homelab
+mem ref sync --corpus kubectl
+mem ref rm --corpus kubectl
 ```
 
-**Identity is `(source_uri, doc_sha)`.** Same URI and same sha is a no-op: no
+**Why they're no longer needed:**
-embed call, no write, exit 0 with "unchanged". Same URI and different sha is a
-*replace*: tombstone the old chunks in the log, write the new ones. A URI that
-has vanished from the tree on a `sync` is a tombstone with no successor.
 
-**Tombstone, do not delete.** The log is append-only and authoritative. A
+1. **Users don't add corpora via CLI.** They use Obsidian to create/edit `.md`
-`{"kind":"reference_tombstone","sha256":"…","reason":"replaced"}` record is what
+   files in the vault. The vault is the source of truth, fetched via REST API.
-removal means; the projector drops the row and the note on replay. Deleting rows
-from Postgres directly makes the index un-rebuildable, which is the one thing the
-whole design refuses.
 
-**Embedding is the expensive step, so skip it precisely.** A corpus of 400 chunks
+2. **Sync happens automatically on rebuild.** `mem rebuild --from-log` calls
-where one document changed should issue embeddings for that document's chunks
+   `ObsidianRefSource` which:
-only. Chunk-level sha comparison, not document-level re-embed.
+   - Fetches file list from Obsidian REST API
+   - Compares current file SHA256 vs. previous (already computed in log)
+   - Re-chunks changed files only (deterministic embedding)
+   - Re-indexes in Postgres + OpenSearch
+   - Tombstones deleted files (via log records)
 
-**`list` reports drift.** For each corpus, re-stat the tree and compare doc shas
+3. **List/drift detection can be queried.** `mem query --level=R` shows which
-without writing anything: `3 docs changed, 1 removed, 12 unchanged`. Drift that
+   reference chunks are indexed. Drift is detected on rebuild by comparing shas.
-is only discoverable by running `sync` means nobody runs `sync`.
 
-## Steps
+4. **No external file paths to manage.** Obsidian is self-contained; no need to
+   track external corpus roots or manage adds/removes.
 
-1. `mem ref add [--project P] --corpus C [--dry-run] <path>` — walk via
+## Workflow Instead (Current Architecture)
-   `DocCorpusSource`, embed new chunks, write `Reference` records.
-2. Persist corpus registration (name, root path, project, last ingest) in the
-   log as a `reference_corpus` record so `list` needs no side file.
-3. Implement chunk-level diff: existing shas for the corpus vs freshly walked
-   shas → `{new, changed, unchanged, gone}`.
-4. `mem ref sync` — apply the diff, embedding only `new` and `changed`, emitting
-   tombstones for `gone`.
-5. `mem ref list` — table per corpus with counts plus a dry drift check.
-6. `mem ref rm` — tombstone every live chunk in the corpus; leave the log intact.
-7. Exit codes: 0 on success including no-op, non-zero on unresolvable corpus or
-   unreadable root.
 
-## Acceptance
+**User modifies reference documents:**
+```
+1. User opens Obsidian UI (http://obsidian.poimen.local)
+2. User creates/edits `.md` files in vault
+3. Files saved to persistent storage (K8s PVC)
+```
 
-- `add` twice on an unchanged tree issues zero embedding calls the second time.
+**Memory system syncs automatically:**
-- Editing one file and running `sync` re-embeds that file's chunks only.
+```
-- Deleting a file and running `sync` tombstones its chunks, and it stops
+1. Admin runs: mem rebuild --from-log
-  appearing in query results.
+2. ObsidianRefSource (M3.6.2) fetches file list from Obsidian REST API
-- `rm` removes the corpus from results while leaving every record in the log.
+3. For each file: compare SHA256 vs. previous (from log)
-- `list` reports drift without mutating anything.
+   - Changed: re-chunk via M3.6.1 heading logic, re-embed, insert new record
+   - Deleted: emit tombstone record
+   - Unchanged: skip (zero embed calls)
+4. Postgres + OpenSearch indexes updated deterministically
+5. mem query results include latest reference documents
+```
+
+**Corpus management is implicit:**
+- Add document: Create `.md` file in Obsidian
+- Remove document: Delete `.md` file in Obsidian
+- Update document: Edit `.md` file in Obsidian
+- Sync: `mem rebuild --from-log`
+
+## Acceptance (Shifted to M3.6.2)
+
+The properties M3.6.3 enforced are now M3.6.2's responsibility:
+
+- ✅ Changed file: re-embeds only that file's chunks (M3.6.2 chunk-level diff)
+- ✅ Deleted file: emits tombstone record (M3.6.2 rebuild logic)
+- ✅ Deleted file removed from queries: tombstone prevents indexing
+- ✅ Unchanged files: zero embed calls on rebuild (M3.6.2 SHA comparison)
+- ✅ Rebuild parity: same file contents → same chunk shas → same indexes
+  (M3.6.2 assertion a14_rebuild_byte_identical)
 
 ## Verify
 
-**Harness:** fixture tree copied to a temp dir so it can be mutated, a counting
+**Verification moved to M3.6.2 (ObsidianRefSource)** which now covers:
-embedder that records how many texts it was asked to embed, seeded database.
+- `a7_rebuild_byte_identical` — zero embeds on second rebuild of unchanged corpus
+- `a8_changed_doc_reembeds_only_itself` — edit one file, rebuild re-embeds only that file
+- `a9_deleted_doc_emits_tombstone` — delete file, rebuild emits tombstone, query excludes it
 
-**Integration test** — `tests/it_mem_ref.rs`:
+**No separate M3.6.3 tests needed.** All functionality tested as part of M3.6.2
-1. `a1_add_then_add_is_noop` — run `add` twice; assert the embedder call count is
+integration harness.
-   zero on the second run and the row count is unchanged.
-2. `a2_changed_doc_reembeds_only_itself` — edit one file of three, `sync`; assert
    embed count equals that file's chunk count, not the corpus total.
 3. `a3_replace_tombstones_predecessor` — after a change, assert the old chunk
    sha has a tombstone record and no live row.
@@ -104,7 +128,14 @@ embedder that records how many texts it was asked to embed, seeded database.
 - Running the churn test without a final rebuild. Assertion 6 is what proves the
   tombstone replay logic exists rather than being implied.
 
-## Traps
+## Historical Traps (No Longer Applicable)
+
+These were issues with the CLI approach; no longer relevant since Obsidian is
+the source of truth:
+
+~~Reusing `run_id` semantics...~~ → Now: `ref-obsidian-<timestamp>` in M3.6.2
+~~Tracking external corpus roots...~~ → Now: Obsidian REST API is the root
+~~Managing adds/removes via state files...~~ → Now: Obsidian vault is the state
 
 - Registering the corpus before the walk succeeds. A failed `add` that leaves a
   registered-but-empty corpus makes the next `sync` report every document as new.