poimen-memory/tasks/M3.6.2-level-r-storage.md

# M3.6.2 — Level R: log record, index rows, vault notes, rebuild parity

| Field | Value |
|---|---|
| Phase | M3.6 — Reference corpora |
| Size | M — 1–3 days |
| Status | ⬜ Not started |
| Flags | — |
| Spec | inlined below |
| Blocks | M3.6.6 |
| Depends | M3.6.1, M1.6, M2.3, M2.4, M2.5, M2.6 |

## Goal

Land reference chunks in the log as their own record kind, project them into
Postgres and the vault, and prove the projections are still throwaway.

## Facts (inlined — no spec read needed)

```jsonl
{"kind":"reference","level":"R","project":"homelab","source":"file:///.../kubectl.md",
 "heading_path":"kubectl.md > Common Issues > CrashLoopBackOff","doc_sha":"ab12…",
 "sha256":"cd34…","t":7,"run_id":"ref-2026-08-21T10:02:11Z","text":"…"}
```

**No migration is needed.** `'R'` ships in M2.3's initial schema, along with the
`CHECK ((level IN ('L2','R')) = (query_id IS NULL))` constraint. Nothing was built
before this phase existed, so the level was never absent from the schema and an
`ALTER` here would only undo a deliberate omission that was never made.

`level = 'R'`, `query_id = NULL` (R answers no standing question), `source` holds
the source URI. `doc_sha` is the whole-document hash; `sha256` is the chunk hash
and stays the primary identity, same as every other level.

The embedding goes to `memory_vector(kind='text')`, not to a column on the node.
R gets **no symptom projection** — M3.7.8 generates those for L1 and L2 only,
since documentation headings already read like problems.

**R writes no edges.** Not to parents, not to siblings. A reference chunk has no
provenance inside this system — its provenance is the URI. The rule that makes
this safe is enforced in `mem verify` (M3.6.4), but nothing in this task should
ever be tempted to create an edge in the first place.

**Vault projection goes somewhere separate.** `vault/reference/<corpus>/<doc>.md`,
not into the project notes. The vault is browsed by a human; interleaving
upstream docs with synthesized project memory makes the vault untrustworthy at a
glance. One note per source document, sections as headings, each carrying its
chunk sha as an anchor so `mem query` output can deep-link.

**Rebuild parity is the whole point of the task.** `mem rebuild --from-log` must
drop and reconstruct R rows and R notes byte-identically. If it cannot, R has
hidden inputs and rule 3 of the design is broken — M2.8 already enforces this
property for L0/L1/L2 and this task extends the same harness rather than writing
a second one.

## Steps

1. Add the `Reference` variant to the log record enum in `mem-core`; serialize
   with the field set above.
2. `mem-store`: insert R nodes with a single `kind='text'` vector; assert at the
   repository boundary that no edge insert names an R sha as parent.
4. Obsidian projector: `vault/reference/<corpus>/<doc>.md`, one note per source
   document, chunk shas as heading anchors.
5. Extend `mem rebuild --from-log` to replay `Reference` records.
6. Extend the M2.6 rebuild-parity harness to cover a log containing R records.

## Acceptance

- A `Reference` record round-trips through the log unchanged.
- R rows land with `query_id IS NULL` and `source` set to the URI.
- The widened constraint accepts `R` and still rejects `L3`.
- Reference notes land under `vault/reference/`, never in project note dirs.
- Drop database + vault, `mem rebuild --from-log`, and both come back
  byte-identical.

## Verify

**Harness:** the M2.6 rebuild harness, extended with a log fixture that contains
L0/L1/L2 *and* R records. Deterministic fake embedder so shas are stable.

**Integration test** — `tests/it_level_r_storage.rs`:
1. `a1_record_roundtrip` — serialize then deserialize a `Reference` record;
   assert field-for-field equality including `doc_sha` and `heading_path`.
2. `a2_r_inserts` — insert `level='R'` with a `kind='text'` vector; assert both
   rows persist.
3. `a3_no_symptom_vector` — assert no R node acquires a `kind='symptom'` vector
   after a full ingest.
4. `a4_query_id_null_at_r` — assert every R row has `query_id IS NULL`, and that
   an R row with one is rejected by M2.3's CHECK.
5. `a5_no_edges_from_r` — after ingesting the fixture corpus, assert
   `SELECT count(*) FROM memory_edge WHERE parent_sha IN (SELECT sha256 FROM
   memory_node WHERE level='R')` is 0.
6. `a6_vault_path_isolation` — assert every emitted reference note path starts
   with `vault/reference/` and no project note directory gained a file.
7. `a7_rebuild_byte_identical` — snapshot database rows and vault files, drop
   both, `mem rebuild --from-log`, assert byte-identical including R.
8. `a8_rebuild_is_idempotent` — rebuild twice; assert the second run changes
   nothing.

**Command:** `cargo test -p mem-store level_r && cargo test -p mem-cli rebuild`

**False pass:**
- Asserting rebuild parity on a log with no R records. It passes trivially and
  proves nothing about this task; assertion 7 is only meaningful because the
  fixture log is mixed-level.
- Checking edge count is zero *before* ingesting anything. Assertion 5 has to run
  against a populated corpus or it is asserting that an empty table is empty.
- Comparing vault files with a normalizing diff. Byte-identical means bytes;
  trailing-newline drift is exactly the class of hidden input this rule exists
  to catch.

## Traps

- Reusing `run_id` semantics from the gated loop. R has no run in the recurrence
  sense; use a synthetic `ref-<timestamp>` and do not let it collide with a real
  ingest run in queries that group by `run_id`.
- Putting reference notes in the project vault "just for now". The vault is the
  human surface and the mixing is not reversible by a later move — links written
  against the old path rot.
- Dropping the check constraint instead of widening it. Assertion 3 exists
  because `DROP CONSTRAINT` alone passes every other assertion in this file.

---

Background: [DESIGN.md](../DESIGN.md) — reference corpora, storage schemas