Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)
ci / markdown (push) Waiting to run
ci / markdown (push) Waiting to run
This commit is contained in:
Story Crater Bot
@@ -0,0 +1,125 @@
|
||||
# M3.6.2 — Level R: log record, index rows, vault notes, rebuild parity
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | M3.6 — Reference corpora |
|
||||
| Size | M — 1–3 days |
|
||||
| Status | ⬜ Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | M3.6.6 |
|
||||
| Depends | M3.6.1, M1.6, M2.3, M2.4, M2.5, M2.6 |
|
||||
|
||||
## Goal
|
||||
|
||||
Land reference chunks in the log as their own record kind, project them into
|
||||
Postgres and the vault, and prove the projections are still throwaway.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
```jsonl
|
||||
{"kind":"reference","level":"R","project":"homelab","source":"file:///.../kubectl.md",
|
||||
"heading_path":"kubectl.md > Common Issues > CrashLoopBackOff","doc_sha":"ab12…",
|
||||
"sha256":"cd34…","t":7,"run_id":"ref-2026-08-21T10:02:11Z","text":"…"}
|
||||
```
|
||||
|
||||
**No migration is needed.** `'R'` ships in M2.3's initial schema, along with the
|
||||
`CHECK ((level IN ('L2','R')) = (query_id IS NULL))` constraint. Nothing was built
|
||||
before this phase existed, so the level was never absent from the schema and an
|
||||
`ALTER` here would only undo a deliberate omission that was never made.
|
||||
|
||||
`level = 'R'`, `query_id = NULL` (R answers no standing question), `source` holds
|
||||
the source URI. `doc_sha` is the whole-document hash; `sha256` is the chunk hash
|
||||
and stays the primary identity, same as every other level.
|
||||
|
||||
The embedding goes to `memory_vector(kind='text')`, not to a column on the node.
|
||||
R gets **no symptom projection** — M3.7.8 generates those for L1 and L2 only,
|
||||
since documentation headings already read like problems.
|
||||
|
||||
**R writes no edges.** Not to parents, not to siblings. A reference chunk has no
|
||||
provenance inside this system — its provenance is the URI. The rule that makes
|
||||
this safe is enforced in `mem verify` (M3.6.4), but nothing in this task should
|
||||
ever be tempted to create an edge in the first place.
|
||||
|
||||
**Vault projection goes somewhere separate.** `vault/reference/<corpus>/<doc>.md`,
|
||||
not into the project notes. The vault is browsed by a human; interleaving
|
||||
upstream docs with synthesized project memory makes the vault untrustworthy at a
|
||||
glance. One note per source document, sections as headings, each carrying its
|
||||
chunk sha as an anchor so `mem query` output can deep-link.
|
||||
|
||||
**Rebuild parity is the whole point of the task.** `mem rebuild --from-log` must
|
||||
drop and reconstruct R rows and R notes byte-identically. If it cannot, R has
|
||||
hidden inputs and rule 3 of the design is broken — M2.8 already enforces this
|
||||
property for L0/L1/L2 and this task extends the same harness rather than writing
|
||||
a second one.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Add the `Reference` variant to the log record enum in `mem-core`; serialize
|
||||
with the field set above.
|
||||
2. `mem-store`: insert R nodes with a single `kind='text'` vector; assert at the
|
||||
repository boundary that no edge insert names an R sha as parent.
|
||||
4. Obsidian projector: `vault/reference/<corpus>/<doc>.md`, one note per source
|
||||
document, chunk shas as heading anchors.
|
||||
5. Extend `mem rebuild --from-log` to replay `Reference` records.
|
||||
6. Extend the M2.6 rebuild-parity harness to cover a log containing R records.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- A `Reference` record round-trips through the log unchanged.
|
||||
- R rows land with `query_id IS NULL` and `source` set to the URI.
|
||||
- The widened constraint accepts `R` and still rejects `L3`.
|
||||
- Reference notes land under `vault/reference/`, never in project note dirs.
|
||||
- Drop database + vault, `mem rebuild --from-log`, and both come back
|
||||
byte-identical.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** the M2.6 rebuild harness, extended with a log fixture that contains
|
||||
L0/L1/L2 *and* R records. Deterministic fake embedder so shas are stable.
|
||||
|
||||
**Integration test** — `tests/it_level_r_storage.rs`:
|
||||
1. `a1_record_roundtrip` — serialize then deserialize a `Reference` record;
|
||||
assert field-for-field equality including `doc_sha` and `heading_path`.
|
||||
2. `a2_r_inserts` — insert `level='R'` with a `kind='text'` vector; assert both
|
||||
rows persist.
|
||||
3. `a3_no_symptom_vector` — assert no R node acquires a `kind='symptom'` vector
|
||||
after a full ingest.
|
||||
4. `a4_query_id_null_at_r` — assert every R row has `query_id IS NULL`, and that
|
||||
an R row with one is rejected by M2.3's CHECK.
|
||||
5. `a5_no_edges_from_r` — after ingesting the fixture corpus, assert
|
||||
`SELECT count(*) FROM memory_edge WHERE parent_sha IN (SELECT sha256 FROM
|
||||
memory_node WHERE level='R')` is 0.
|
||||
6. `a6_vault_path_isolation` — assert every emitted reference note path starts
|
||||
with `vault/reference/` and no project note directory gained a file.
|
||||
7. `a7_rebuild_byte_identical` — snapshot database rows and vault files, drop
|
||||
both, `mem rebuild --from-log`, assert byte-identical including R.
|
||||
8. `a8_rebuild_is_idempotent` — rebuild twice; assert the second run changes
|
||||
nothing.
|
||||
|
||||
**Command:** `cargo test -p mem-store level_r && cargo test -p mem-cli rebuild`
|
||||
|
||||
**False pass:**
|
||||
- Asserting rebuild parity on a log with no R records. It passes trivially and
|
||||
proves nothing about this task; assertion 7 is only meaningful because the
|
||||
fixture log is mixed-level.
|
||||
- Checking edge count is zero *before* ingesting anything. Assertion 5 has to run
|
||||
against a populated corpus or it is asserting that an empty table is empty.
|
||||
- Comparing vault files with a normalizing diff. Byte-identical means bytes;
|
||||
trailing-newline drift is exactly the class of hidden input this rule exists
|
||||
to catch.
|
||||
|
||||
## Traps
|
||||
|
||||
- Reusing `run_id` semantics from the gated loop. R has no run in the recurrence
|
||||
sense; use a synthetic `ref-<timestamp>` and do not let it collide with a real
|
||||
ingest run in queries that group by `run_id`.
|
||||
- Putting reference notes in the project vault "just for now". The vault is the
|
||||
human surface and the mixing is not reversible by a later move — links written
|
||||
against the old path rot.
|
||||
- Dropping the check constraint instead of widening it. Assertion 3 exists
|
||||
because `DROP CONSTRAINT` alone passes every other assertion in this file.
|
||||
|
||||
---
|
||||
|
||||
Background: [DESIGN.md](../DESIGN.md) — reference corpora, storage schemas
|
||||
Reference in New Issue
Block a user