poimen-memory/tasks/M3.6.3-mem-ref-cli.md

# M3.6.3 — RETIRED: `mem ref` CLI (replaced by Obsidian UI)

| Field | Value |
|---|---|
| Phase | M3.6 — Reference corpora |
| Size | M — 1–3 days |
| Status | ❌ RETIRED |
| Flags | — |
| Spec | inlined below (historical) |
| Blocks | — |
| Depends | — |

## Retirement Rationale

**Obsidian UI replaces CLI corpus management.**

With Obsidian vault deployed (M2.5 complete) as the source of truth for reference
documents, the CLI reference management commands are now redundant:

- **Document modification:** Users edit files in Obsidian UI (web/desktop)
- **Corpus management:** Obsidian vault (filesystem) is the authoritative store
- **Synchronization:** M3.6.2 (ObsidianRefSource) automatically fetches from
  Obsidian REST API on every rebuild
- **Change detection:** Rebuild compares file SHA256 hashes and re-chunks/re-embeds
  only changed documents

**What M3.6.3 did (historical):**

## Historical Functionality (M3.6.3 commands, now retired)

```bash
# These commands are no longer needed:
mem ref add --project homelab --corpus kubectl ~/path/to/docs
mem ref list --project homelab
mem ref sync --corpus kubectl
mem ref rm --corpus kubectl
```

**Why they're no longer needed:**

1. **Users don't add corpora via CLI.** They use Obsidian to create/edit `.md`
   files in the vault. The vault is the source of truth, fetched via REST API.

2. **Sync happens automatically on rebuild.** `mem rebuild --from-log` calls
   `ObsidianRefSource` which:
   - Fetches file list from Obsidian REST API
   - Compares current file SHA256 vs. previous (already computed in log)
   - Re-chunks changed files only (deterministic embedding)
   - Re-indexes in Postgres + OpenSearch
   - Tombstones deleted files (via log records)

3. **List/drift detection can be queried.** `mem query --level=R` shows which
   reference chunks are indexed. Drift is detected on rebuild by comparing shas.

4. **No external file paths to manage.** Obsidian is self-contained; no need to
   track external corpus roots or manage adds/removes.

## Workflow Instead (Current Architecture)

**User modifies reference documents:**
```
1. User opens Obsidian UI (http://obsidian.poimen.local)
2. User creates/edits `.md` files in vault
3. Files saved to persistent storage (K8s PVC)
```

**Memory system syncs automatically:**
```
1. Admin runs: mem rebuild --from-log
2. ObsidianRefSource (M3.6.2) fetches file list from Obsidian REST API
3. For each file: compare SHA256 vs. previous (from log)
   - Changed: re-chunk via M3.6.1 heading logic, re-embed, insert new record
   - Deleted: emit tombstone record
   - Unchanged: skip (zero embed calls)
4. Postgres + OpenSearch indexes updated deterministically
5. mem query results include latest reference documents
```

**Corpus management is implicit:**
- Add document: Create `.md` file in Obsidian
- Remove document: Delete `.md` file in Obsidian
- Update document: Edit `.md` file in Obsidian
- Sync: `mem rebuild --from-log`

## Acceptance (Shifted to M3.6.2)

The properties M3.6.3 enforced are now M3.6.2's responsibility:

- ✅ Changed file: re-embeds only that file's chunks (M3.6.2 chunk-level diff)
- ✅ Deleted file: emits tombstone record (M3.6.2 rebuild logic)
- ✅ Deleted file removed from queries: tombstone prevents indexing
- ✅ Unchanged files: zero embed calls on rebuild (M3.6.2 SHA comparison)
- ✅ Rebuild parity: same file contents → same chunk shas → same indexes
  (M3.6.2 assertion a14_rebuild_byte_identical)

## Verify

**Verification moved to M3.6.2 (ObsidianRefSource)** which now covers:
- `a7_rebuild_byte_identical` — zero embeds on second rebuild of unchanged corpus
- `a8_changed_doc_reembeds_only_itself` — edit one file, rebuild re-embeds only that file
- `a9_deleted_doc_emits_tombstone` — delete file, rebuild emits tombstone, query excludes it

**No separate M3.6.3 tests needed.** All functionality tested as part of M3.6.2
integration harness.
   embed count equals that file's chunk count, not the corpus total.
3. `a3_replace_tombstones_predecessor` — after a change, assert the old chunk
   sha has a tombstone record and no live row.
4. `a4_removed_doc_tombstoned` — delete a file, `sync`; assert its chunks are
   gone from `memory_node` and present in the log.
5. `a5_rm_preserves_log` — count log lines before and after `rm`; assert the
   count only grew.
6. `a6_rebuild_after_churn` — after add/change/sync/rm, `mem rebuild --from-log`;
   assert the reconstructed state matches the live state exactly.
7. `a7_list_is_read_only` — snapshot database and log, run `list`, assert both
   unchanged and that reported drift matches the mutations made.
8. `a8_unreadable_root_exits_nonzero` — point `add` at a missing path; assert
   non-zero exit and no partial corpus registration.

**Command:** `cargo test -p mem-cli mem_ref`

**False pass:**
- Asserting "no duplicate rows" instead of counting embedder calls. A `sync` that
  re-embeds everything and then upserts by sha produces a correct table and a
  bill; assertion 1 and 2 are the only ones that see it.
- Verifying tombstones by querying `memory_node`. The row being absent is the
  projector working; assertion 3 has to read the log to prove the tombstone was
  actually written and the row was not just deleted.
- Running the churn test without a final rebuild. Assertion 6 is what proves the
  tombstone replay logic exists rather than being implied.

## Historical Traps (No Longer Applicable)

These were issues with the CLI approach; no longer relevant since Obsidian is
the source of truth:

~~Reusing `run_id` semantics...~~ → Now: `ref-obsidian-<timestamp>` in M3.6.2
~~Tracking external corpus roots...~~ → Now: Obsidian REST API is the root
~~Managing adds/removes via state files...~~ → Now: Obsidian vault is the state

- Registering the corpus before the walk succeeds. A failed `add` that leaves a
  registered-but-empty corpus makes the next `sync` report every document as new.
- Comparing document mtime instead of sha. Checkouts and rsync rewrite mtimes;
  a corpus that re-embeds on every clone costs real money on the TEI endpoint.
- Making `rm` delete log records "because they are noise". That converts the log
  from authoritative to advisory, and nothing downstream can tell.

---

Background: [DESIGN.md](../DESIGN.md) — reference corpora, JSONL event log