poimen-memory/tasks/M2.6-rebuild-from-log.md

M2.6 — mem rebuild --from-log

Field	Value
Phase	M2 — Projections
Size	M — 1–3 days
Status	⬜ Not started
Flags	—
Spec	inlined below
Blocks	M2.4, M2.5

Goal

Drop both projections and rebuild them from the log alone — the command that makes "the log is authoritative" a testable claim instead of a slogan.

Facts (inlined — no spec read needed)

mem rebuild --from-log --project poimen            # both projections
mem rebuild --from-log --project poimen --vault-only
mem rebuild --from-log --project poimen --db-only

The claim: anything not reconstructible from the log has a hidden input, and that is a bug. Rebuild is the executable form of that claim. If it needs the existing vault or database to produce correct output, something is being carried across that is not in the record.

Rebuild does no model calls except embeddings. Gate decisions, memory text and provenance are all in the log already; re-running the controller would produce different text and defeat the purpose.

Order matters: clear → insert all nodes → insert all edges → project vault. Edges before nodes violates the foreign key (M2.4).

Embeddings are the expensive part. Cache by sha256 so a rebuild after a vault template change does not re-embed unchanged nodes.

Steps

1. mem rebuild --from-log --project P.
2. Read every log file for the project, in run-id order.
3. clear_project, then two-pass node/edge insert, batching embeddings.
4. Project the vault (M2.5), overwriting.
5. Embedding cache keyed by sha, on disk under .cache/, so it survives runs.
6. Report counts: nodes by level, edges, embeddings computed vs cached.
7. Refuse to run if any log file is incomplete (no run_end) unless --allow-partial — rebuilding from a half-run silently produces a half-memory.

Acceptance

• Rebuild from an empty database and empty vault produces the full state.
• Rebuild twice produces identical database rows and identical vault bytes.
• No controller model calls occur.
• An incomplete log is refused by default.

Verify

Harness: log fixture, disposable Postgres, temp vault. A controller client that panics if called.

Integration test — tests/it_rebuild.rs:

1. a1_from_empty — drop everything, rebuild, assert node counts per level match the log's records.
2. a2_idempotent_db — rebuild twice, assert row count unchanged and no created_at churn on existing rows.
3. a3_idempotent_vault — rebuild twice, assert vault bytes identical.
4. a4_no_controller_calls — inject a panicking chat client; assert rebuild succeeds.
5. a5_embedding_cache — second rebuild computes zero embeddings.
6. a6_edge_order — a log whose first memory references a later-inserted parent still rebuilds, proving two-pass.
7. a7_incomplete_refused — a log with no run_end exits non-zero; with --allow-partial it succeeds.
8. a8_log_is_sufficient — delete the vault and the database entirely, rebuild, and assert the result equals a committed golden. This is the authority claim.

Command: cargo test -p mem-cli rebuild

False pass:

• Rebuilding on top of existing state. It masks every hidden input, because the missing piece is already there from the previous run. Assertions 1 and 8 must start from nothing.
• Asserting row counts only. A rebuild that inserts the right number of rows with wrong parents passes; assertion 6 and M2.7's edge closure are what check the graph.

Traps

• Re-running the controller during rebuild. It produces different memory text every time, the vault never stabilises, and M2.8 can never pass.
• Caching embeddings by node id rather than content hash. Ids change between rebuilds; hashes do not, which is the whole point of content identity.

---

Background: DESIGN.md — Authority model