poimen/tasks/T2.4-schema-evolution-end-to-end.md

T2.4 — Schema evolution end-to-end

Field	Value
Phase	P2 — Durability hard parts
Size	M — 1 to 3 days
Status	Not started
Flags	—
Spec	inlined below
Blocks	—

Goal

Change the event enum for real and prove a log recorded in P1 still folds. This is T0.4's framework exercised against actual history rather than a synthetic fixture.

Facts (inlined — no spec read needed)

• Depends on T0.4 (upcaster registry) and T0.3 (SchemaVersion on every record).
• The rules being tested: variants are never removed or repurposed; deprecated variants stay decodable forever; migrations are upcasters applied on read, never a rewrite of stored records.
• The failure this prevents: two years of records, one WorkEvent variant renamed, and the drop-and-re-fold property is silently gone. Nothing fails at the time of the rename — it fails at the first cold rebuild.
• The v1 fixture must be a real P1 log, not a hand-written one. A synthetic fixture only tests the variants whoever wrote it remembered.

Steps

1. Capture a full P1 run's log as a committed fixture, at v1, with its blobs. Store the expected folded state alongside it.
2. Make a genuine v2 change: add a field to an existing variant, or split one variant into two. Do not rename or delete anything.
3. Bump CURRENT_SCHEMA to v2 and register the v1→v2 upcaster.
4. Run the fold over the v1 fixture under the v2 binary; assert the folded state equals the recorded expectation.
5. Record a fresh v2 fixture from the same workflow and assert the two folded states agree — an upcast v1 log and a native v2 log must be indistinguishable downstream.
6. Extend T0.3's directory-walk fixture test so both version folders are covered permanently, not just in this task's test.

Acceptance

• A v1 log fixture recorded in P1 still folds correctly after the event enum changes.

Verify

Harness: a real P1 run's log committed as a fixture, with its expected folded state committed beside it. Not hand-written.

Integration test — tests/it_schema_evolution.rs:

1. Under the v2 binary, fold the committed v1 fixture → state_a.
2. Assert state_a equals the committed expected state, byte for byte.
3. Record a fresh v2 run of the same workflow → fold → state_b.
4. Assert state_a and state_b are equivalent under the fields that should not have changed. An upcast v1 log and a native v2 log must be indistinguishable downstream.
5. Assert the v1 fixture file's bytes are unchanged by the test run — the fixture is read-only history.
6. Variant-removal guard: a test that enumerates v1's variant set and asserts every one is still decodable under v2.

Command: cargo test -p log schema_evolution

False pass:

• Regenerating the fixture when the test goes red. That is the one move that destroys the guarantee — the fixture is the historical record, the upcaster is what changes. Make the fixture path read-only in the test and assert on that.
• A v2 change that touches no variant present in the P1 fixture, so no upcaster logic runs. Step 6 catches the removal case; choose a v2 change that provably affects a recorded variant.

Traps

• "Fixing" the fixture bytes when the test fails. The fixture is the historical record; the upcaster is what changes.
• A v2 change that removes a variant, which passes this test today because nothing in the P1 fixture used it, and breaks the moment an older log does.

---

Background (not required to do this task): rust-agentic-sys.md §8.7 · rust-agentic-task.md