99 lines
3.8 KiB
Markdown
99 lines
3.8 KiB
Markdown
# M2.6 — `mem rebuild --from-log`
|
||
|
||
| Field | Value |
|
||
|---|---|
|
||
| Phase | M2 — Projections |
|
||
| Size | M — 1–3 days |
|
||
| Status | ⬜ Not started |
|
||
| Flags | — |
|
||
| Spec | inlined below |
|
||
| Blocks | M2.4, M2.5 |
|
||
|
||
## Goal
|
||
|
||
Drop both projections and rebuild them from the log alone — the command that
|
||
makes "the log is authoritative" a testable claim instead of a slogan.
|
||
|
||
## Facts (inlined — no spec read needed)
|
||
|
||
```
|
||
mem rebuild --from-log --project poimen # both projections
|
||
mem rebuild --from-log --project poimen --vault-only
|
||
mem rebuild --from-log --project poimen --db-only
|
||
```
|
||
|
||
The claim: **anything not reconstructible from the log has a hidden input, and
|
||
that is a bug.** Rebuild is the executable form of that claim. If it needs the
|
||
existing vault or database to produce correct output, something is being carried
|
||
across that is not in the record.
|
||
|
||
Rebuild does **no model calls except embeddings**. Gate decisions, memory text and
|
||
provenance are all in the log already; re-running the controller would produce
|
||
different text and defeat the purpose.
|
||
|
||
Order matters: clear → insert all nodes → insert all edges → project vault. Edges
|
||
before nodes violates the foreign key (M2.4).
|
||
|
||
Embeddings are the expensive part. Cache by `sha256` so a rebuild after a vault
|
||
template change does not re-embed unchanged nodes.
|
||
|
||
## Steps
|
||
|
||
1. `mem rebuild --from-log --project P`.
|
||
2. Read every log file for the project, in run-id order.
|
||
3. `clear_project`, then two-pass node/edge insert, batching embeddings.
|
||
4. Project the vault (M2.5), overwriting.
|
||
5. Embedding cache keyed by sha, on disk under `.cache/`, so it survives runs.
|
||
6. Report counts: nodes by level, edges, embeddings computed vs cached.
|
||
7. Refuse to run if any log file is incomplete (no `run_end`) unless `--allow-partial`
|
||
— rebuilding from a half-run silently produces a half-memory.
|
||
|
||
## Acceptance
|
||
|
||
- Rebuild from an empty database and empty vault produces the full state.
|
||
- Rebuild twice produces identical database rows and identical vault bytes.
|
||
- No controller model calls occur.
|
||
- An incomplete log is refused by default.
|
||
|
||
## Verify
|
||
|
||
**Harness:** log fixture, disposable Postgres, temp vault. A controller client
|
||
that panics if called.
|
||
|
||
**Integration test** — `tests/it_rebuild.rs`:
|
||
1. `a1_from_empty` — drop everything, rebuild, assert node counts per level match
|
||
the log's records.
|
||
2. `a2_idempotent_db` — rebuild twice, assert row count unchanged and no
|
||
`created_at` churn on existing rows.
|
||
3. `a3_idempotent_vault` — rebuild twice, assert vault bytes identical.
|
||
4. `a4_no_controller_calls` — inject a panicking chat client; assert rebuild
|
||
succeeds.
|
||
5. `a5_embedding_cache` — second rebuild computes zero embeddings.
|
||
6. `a6_edge_order` — a log whose first memory references a later-inserted parent
|
||
still rebuilds, proving two-pass.
|
||
7. `a7_incomplete_refused` — a log with no `run_end` exits non-zero; with
|
||
`--allow-partial` it succeeds.
|
||
8. `a8_log_is_sufficient` — delete the vault and the database entirely, rebuild,
|
||
and assert the result equals a committed golden. This is the authority claim.
|
||
|
||
**Command:** `cargo test -p mem-cli rebuild`
|
||
|
||
**False pass:**
|
||
- Rebuilding on top of existing state. It masks every hidden input, because the
|
||
missing piece is already there from the previous run. Assertions 1 and 8 must
|
||
start from nothing.
|
||
- Asserting row counts only. A rebuild that inserts the right number of rows with
|
||
wrong `parents` passes; assertion 6 and M2.7's edge closure are what check the
|
||
graph.
|
||
|
||
## Traps
|
||
|
||
- Re-running the controller during rebuild. It produces different memory text
|
||
every time, the vault never stabilises, and M2.8 can never pass.
|
||
- Caching embeddings by node id rather than content hash. Ids change between
|
||
rebuilds; hashes do not, which is the whole point of content identity.
|
||
|
||
---
|
||
|
||
Background: [DESIGN.md](../DESIGN.md) — Authority model
|