poimen-memory/tasks/M3.6.2-level-r-storage.md

M3.6.2 — Level R: log record, index rows, vault notes, rebuild parity

Field	Value
Phase	M3.6 — Reference corpora
Size	M — 1–3 days
Status	⬜ Not started
Flags	—
Spec	inlined below
Blocks	M3.6.6
Depends	M3.6.1, M1.6, M2.3, M2.4, M2.5, M2.6

Goal

Land reference chunks in the log as their own record kind, project them into Postgres and the vault, and prove the projections are still throwaway.

Facts (inlined — no spec read needed)

{"kind":"reference","level":"R","project":"homelab","source":"file:///.../kubectl.md",
 "heading_path":"kubectl.md > Common Issues > CrashLoopBackOff","doc_sha":"ab12…",
 "sha256":"cd34…","t":7,"run_id":"ref-2026-08-21T10:02:11Z","text":"…"}

No migration is needed. 'R' ships in M2.3's initial schema, along with the CHECK ((level IN ('L2','R')) = (query_id IS NULL)) constraint. Nothing was built before this phase existed, so the level was never absent from the schema and an ALTER here would only undo a deliberate omission that was never made.

level = 'R', query_id = NULL (R answers no standing question), source holds the source URI. doc_sha is the whole-document hash; sha256 is the chunk hash and stays the primary identity, same as every other level.

The embedding goes to memory_vector(kind='text'), not to a column on the node. R gets no symptom projection — M3.7.8 generates those for L1 and L2 only, since documentation headings already read like problems.

R writes no edges. Not to parents, not to siblings. A reference chunk has no provenance inside this system — its provenance is the URI. The rule that makes this safe is enforced in mem verify (M3.6.4), but nothing in this task should ever be tempted to create an edge in the first place.

Vault projection goes somewhere separate. vault/reference/<corpus>/<doc>.md, not into the project notes. The vault is browsed by a human; interleaving upstream docs with synthesized project memory makes the vault untrustworthy at a glance. One note per source document, sections as headings, each carrying its chunk sha as an anchor so mem query output can deep-link.

Rebuild parity is the whole point of the task. mem rebuild --from-log must drop and reconstruct R rows and R notes byte-identically. If it cannot, R has hidden inputs and rule 3 of the design is broken — M2.8 already enforces this property for L0/L1/L2 and this task extends the same harness rather than writing a second one.

Steps

1. Add the Reference variant to the log record enum in mem-core; serialize with the field set above.
2. mem-store: insert R nodes with a single kind='text' vector; assert at the repository boundary that no edge insert names an R sha as parent.
3. Obsidian projector: vault/reference/<corpus>/<doc>.md, one note per source document, chunk shas as heading anchors.
4. Extend mem rebuild --from-log to replay Reference records.
5. Extend the M2.6 rebuild-parity harness to cover a log containing R records.

Acceptance

• A Reference record round-trips through the log unchanged.
• R rows land with query_id IS NULL and source set to the URI.
• The widened constraint accepts R and still rejects L3.
• Reference notes land under vault/reference/, never in project note dirs.
• Drop database + vault, mem rebuild --from-log, and both come back byte-identical.

Verify

Harness: the M2.6 rebuild harness, extended with a log fixture that contains L0/L1/L2 and R records. Deterministic fake embedder so shas are stable.

Integration test — tests/it_level_r_storage.rs:

1. a1_record_roundtrip — serialize then deserialize a Reference record; assert field-for-field equality including doc_sha and heading_path.
2. a2_r_inserts — insert level='R' with a kind='text' vector; assert both rows persist.
3. a3_no_symptom_vector — assert no R node acquires a kind='symptom' vector after a full ingest.
4. a4_query_id_null_at_r — assert every R row has query_id IS NULL, and that an R row with one is rejected by M2.3's CHECK.
5. a5_no_edges_from_r — after ingesting the fixture corpus, assert SELECT count(*) FROM memory_edge WHERE parent_sha IN (SELECT sha256 FROM memory_node WHERE level='R') is 0.
6. a6_vault_path_isolation — assert every emitted reference note path starts with vault/reference/ and no project note directory gained a file.
7. a7_rebuild_byte_identical — snapshot database rows and vault files, drop both, mem rebuild --from-log, assert byte-identical including R.
8. a8_rebuild_is_idempotent — rebuild twice; assert the second run changes nothing.

Command: cargo test -p mem-store level_r && cargo test -p mem-cli rebuild

False pass:

• Asserting rebuild parity on a log with no R records. It passes trivially and proves nothing about this task; assertion 7 is only meaningful because the fixture log is mixed-level.
• Checking edge count is zero before ingesting anything. Assertion 5 has to run against a populated corpus or it is asserting that an empty table is empty.
• Comparing vault files with a normalizing diff. Byte-identical means bytes; trailing-newline drift is exactly the class of hidden input this rule exists to catch.

Traps

• Reusing run_id semantics from the gated loop. R has no run in the recurrence sense; use a synthetic ref-<timestamp> and do not let it collide with a real ingest run in queries that group by run_id.
• Putting reference notes in the project vault "just for now". The vault is the human surface and the mixing is not reversible by a later move — links written against the old path rot.
• Dropping the check constraint instead of widening it. Assertion 3 exists because DROP CONSTRAINT alone passes every other assertion in this file.

---

Background: DESIGN.md — reference corpora, storage schemas