99 lines
3.8 KiB
Markdown
99 lines
3.8 KiB
Markdown
|
# M2.6 — `mem rebuild --from-log`
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
|---|---|
|
|||
|
|
| Phase | M2 — Projections |
|
|||
|
|
| Size | M — 1–3 days |
|
|||
|
|
| Status | ⬜ Not started |
|
|||
|
|
| Flags | — |
|
|||
|
|
| Spec | inlined below |
|
|||
|
|
| Blocks | M2.4, M2.5 |
|
|||
|
|
|
|||
|
|
## Goal
|
|||
|
|
|
|||
|
|
Drop both projections and rebuild them from the log alone — the command that
|
|||
|
|
makes "the log is authoritative" a testable claim instead of a slogan.
|
|||
|
|
|
|||
|
|
## Facts (inlined — no spec read needed)
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
mem rebuild --from-log --project poimen # both projections
|
|||
|
|
mem rebuild --from-log --project poimen --vault-only
|
|||
|
|
mem rebuild --from-log --project poimen --db-only
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The claim: **anything not reconstructible from the log has a hidden input, and
|
|||
|
|
that is a bug.** Rebuild is the executable form of that claim. If it needs the
|
|||
|
|
existing vault or database to produce correct output, something is being carried
|
|||
|
|
across that is not in the record.
|
|||
|
|
|
|||
|
|
Rebuild does **no model calls except embeddings**. Gate decisions, memory text and
|
|||
|
|
provenance are all in the log already; re-running the controller would produce
|
|||
|
|
different text and defeat the purpose.
|
|||
|
|
|
|||
|
|
Order matters: clear → insert all nodes → insert all edges → project vault. Edges
|
|||
|
|
before nodes violates the foreign key (M2.4).
|
|||
|
|
|
|||
|
|
Embeddings are the expensive part. Cache by `sha256` so a rebuild after a vault
|
|||
|
|
template change does not re-embed unchanged nodes.
|
|||
|
|
|
|||
|
|
## Steps
|
|||
|
|
|
|||
|
|
1. `mem rebuild --from-log --project P`.
|
|||
|
|
2. Read every log file for the project, in run-id order.
|
|||
|
|
3. `clear_project`, then two-pass node/edge insert, batching embeddings.
|
|||
|
|
4. Project the vault (M2.5), overwriting.
|
|||
|
|
5. Embedding cache keyed by sha, on disk under `.cache/`, so it survives runs.
|
|||
|
|
6. Report counts: nodes by level, edges, embeddings computed vs cached.
|
|||
|
|
7. Refuse to run if any log file is incomplete (no `run_end`) unless `--allow-partial`
|
|||
|
|
— rebuilding from a half-run silently produces a half-memory.
|
|||
|
|
|
|||
|
|
## Acceptance
|
|||
|
|
|
|||
|
|
- Rebuild from an empty database and empty vault produces the full state.
|
|||
|
|
- Rebuild twice produces identical database rows and identical vault bytes.
|
|||
|
|
- No controller model calls occur.
|
|||
|
|
- An incomplete log is refused by default.
|
|||
|
|
|
|||
|
|
## Verify
|
|||
|
|
|
|||
|
|
**Harness:** log fixture, disposable Postgres, temp vault. A controller client
|
|||
|
|
that panics if called.
|
|||
|
|
|
|||
|
|
**Integration test** — `tests/it_rebuild.rs`:
|
|||
|
|
1. `a1_from_empty` — drop everything, rebuild, assert node counts per level match
|
|||
|
|
the log's records.
|
|||
|
|
2. `a2_idempotent_db` — rebuild twice, assert row count unchanged and no
|
|||
|
|
`created_at` churn on existing rows.
|
|||
|
|
3. `a3_idempotent_vault` — rebuild twice, assert vault bytes identical.
|
|||
|
|
4. `a4_no_controller_calls` — inject a panicking chat client; assert rebuild
|
|||
|
|
succeeds.
|
|||
|
|
5. `a5_embedding_cache` — second rebuild computes zero embeddings.
|
|||
|
|
6. `a6_edge_order` — a log whose first memory references a later-inserted parent
|
|||
|
|
still rebuilds, proving two-pass.
|
|||
|
|
7. `a7_incomplete_refused` — a log with no `run_end` exits non-zero; with
|
|||
|
|
`--allow-partial` it succeeds.
|
|||
|
|
8. `a8_log_is_sufficient` — delete the vault and the database entirely, rebuild,
|
|||
|
|
and assert the result equals a committed golden. This is the authority claim.
|
|||
|
|
|
|||
|
|
**Command:** `cargo test -p mem-cli rebuild`
|
|||
|
|
|
|||
|
|
**False pass:**
|
|||
|
|
- Rebuilding on top of existing state. It masks every hidden input, because the
|
|||
|
|
missing piece is already there from the previous run. Assertions 1 and 8 must
|
|||
|
|
start from nothing.
|
|||
|
|
- Asserting row counts only. A rebuild that inserts the right number of rows with
|
|||
|
|
wrong `parents` passes; assertion 6 and M2.7's edge closure are what check the
|
|||
|
|
graph.
|
|||
|
|
|
|||
|
|
## Traps
|
|||
|
|
|
|||
|
|
- Re-running the controller during rebuild. It produces different memory text
|
|||
|
|
every time, the vault never stabilises, and M2.8 can never pass.
|
|||
|
|
- Caching embeddings by node id rather than content hash. Ids change between
|
|||
|
|
rebuilds; hashes do not, which is the whole point of content identity.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
Background: [DESIGN.md](../DESIGN.md) — Authority model
|