(tasks) add tasks for harness

2026-08-17 23:05:20 -07:00
parent e2d678b118
commit 5a30d0ffc6
74 changed files with 8204 additions and 1 deletions
@@ -0,0 +1,91 @@
+# T2.5 — Checkpoints
+
+| Field | Value |
+|---|---|
+| Phase | P2 — Durability hard parts |
+| Size | M — 1 to 3 days |
+| Status | Not started |
+| Flags | — |
+| Spec | inlined below |
+| Blocks | — |
+
+## Goal
+
+Materialized state snapshots tagged with their LSN, so restart folds forward from
+the newest one instead of from zero.
+
+## Facts (inlined — no spec read needed)
+
+- A checkpoint is a materialized state snapshot tagged with its LSN. Restart
+  folds forward from the newest one at or below the target LSN.
+- **An optimization only.** Deleting every checkpoint costs startup time and
+  nothing else. That property is the acceptance test, and it is what keeps
+  checkpoints from quietly becoming a second source of truth.
+- Port surface already exists from T0.5:
+  `put_checkpoint(key, upto, state)` and `latest_checkpoint(key, upto) -> Option<Checkpoint>`.
+  `None` means fold from LSN 0.
+- Checkpoints are per `BranchKey`, like everything else in the log.
+
+## Steps
+
+1. Serialize the materialized state (T0.8) deterministically — the same ordered
+   collections, so a checkpoint written twice from the same state is byte-equal.
+2. Write checkpoints on a policy: every N records or every M seconds per branch.
+   Keep the policy in one place and make it configurable rather than scattered.
+3. `rebuild(key, upto)`: call `latest_checkpoint`, deserialize if present, then
+   fold records from `checkpoint.lsn + 1`. On `None`, fold from 0.
+4. Never let the checkpoint be the only holder of a fact. Validate this by making
+   the `None` path the default in tests.
+5. Add a prune path — old checkpoints for a branch are deletable at any time
+   without coordination.
+6. Test both directions: with checkpoints and after deleting them all, comparing
+   final state and reporting both startup times.
+
+## Acceptance
+
+- Deleting all checkpoints changes startup time and **nothing else** — final
+  state identical.
+
+## Verify
+
+**Harness:** a run long enough to trigger the checkpoint policy several times —
+at least 3 checkpoints, so "newest at or below" is a real choice.
+
+**Integration test** — `tests/it_checkpoints.rs`:
+1. Run to completion with checkpointing on. Record final state bytes and startup
+   time for a rebuild.
+2. Delete **every** checkpoint row. Rebuild from LSN 0.
+3. Assert the final state bytes are **identical**; assert startup time differs
+   (log both, do not assert a threshold — that is flaky).
+4. `latest_checkpoint(key, upto)` with `upto` between two checkpoints: assert it
+   returns the **lower** one, not the newest overall.
+5. Determinism: write a checkpoint twice from the same state; assert byte
+   equality of the two serializations.
+6. Crash between checkpoint write and subsequent commits; assert rebuild still
+   lands on the same state.
+7. Prune: delete an older checkpoint while a newer exists; assert rebuild is
+   unaffected.
+
+**Command:** `cargo test -p durability checkpoints`
+
+**False pass:**
+- A run short enough to produce zero or one checkpoint. Then step 2 deletes
+  nothing and the test is vacuous — assert the checkpoint count is ≥ 3 before
+  deleting.
+- Step 4 omitted: an implementation returning the globally newest checkpoint
+  passes everything else and breaks the moment rewind or point-in-time rebuild
+  needs an earlier LSN.
+- Comparing state with `PartialEq` rather than bytes.
+
+## Traps
+
+- Writing a checkpoint inside the same transaction as the commit path and then
+  depending on it for correctness. It is a cache; keep it separable.
+- A checkpoint that serializes a `HashMap`. Byte-identity fails intermittently
+  and reads as flakiness.
+
+---
+
+Background (not required to do this task):
+[rust-agentic-sys.md](../../../rust-agentic-sys.md) §7, §8.6 ·
+[rust-agentic-task.md](../../../rust-agentic-task.md)