105 lines
4.4 KiB
Markdown
105 lines
4.4 KiB
Markdown
# T0.6 — Atomic commit protocol
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Phase | P0 — Foundations |
|
|
| Size | M — 1 to 3 days |
|
|
| Status | Not started |
|
|
| Flags | — |
|
|
| Spec | inlined below |
|
|
| Blocks | T2.1 |
|
|
|
|
## Goal
|
|
|
|
Four table writes in one transaction behind `EventLog::commit`. All four land or
|
|
none do.
|
|
|
|
## Facts (inlined — no spec read needed)
|
|
|
|
```rust
|
|
let txn = db.begin_write()?;
|
|
{
|
|
let mut log = txn.open_table(EVENT_LOG)?;
|
|
let mut state = txn.open_table(RUN_STATE)?;
|
|
let mut position = txn.open_table(CONSUMER_POSITION)?;
|
|
let mut outbox = txn.open_table(OUTBOX)?;
|
|
|
|
log.insert((branch_key, lsn), &record)?; // append: the durable fact
|
|
state.insert((branch_key, attempt_no), &attempt)?; // apply: the derived view
|
|
position.insert(Scoped::new(tenant, stream), pos)?; // advance: where to resume
|
|
outbox.insert((branch_key, lsn), &intent)?; // relay separately
|
|
}
|
|
txn.commit()?; // all four, or none
|
|
```
|
|
|
|
- **Every key carries `BranchKey` or `Scoped<_>`.** The outbox is the one that
|
|
invites the mistake: an outbox keyed on `Lsn` alone reads naturally and is
|
|
wrong, because LSNs are per branch — a bare LSN collides across every branch of
|
|
every run of every tenant. The same key shape also gives the relay its only
|
|
promised ordering: per `BranchKey`, ascending `Lsn`.
|
|
- **Commit per event, not per run.** A projection that accumulates in memory and
|
|
writes at run end loses the whole run on a crash. Episodes are small and
|
|
append-mostly; one fsync per transition is cheap next to model latency.
|
|
- The engine supplies crash-atomicity (shadow paging in `redb`, transactions in
|
|
Postgres). The log supplies history. Because the engine's transaction is
|
|
atomic, append and apply happen together: no torn records, no redo/undo pass,
|
|
no checkpoint-consistency problem.
|
|
|
|
## Steps
|
|
|
|
1. Route every write through `EventLog::commit`. There is no second write path —
|
|
grep for direct `open_table(...).insert` outside the implementation and delete
|
|
what you find.
|
|
2. Confirm each of the four keys: `(BranchKey, Lsn)` for log and outbox,
|
|
`(BranchKey, AttemptNo)` for state, `Scoped<StreamId>` for position.
|
|
3. Build the crash harness: a test-only hook that panics after write *i* of the
|
|
four, for `i in 0..4`. Reopen the database and assert all-or-nothing.
|
|
4. Add the collision test: two runs in two tenants both writing LSN 1, both
|
|
outbox entries present and distinct after commit.
|
|
5. Measure one commit's fsync cost and record it in the test output, so the
|
|
per-event decision stays visible rather than being re-litigated later.
|
|
|
|
## Acceptance
|
|
|
|
- Harness panics between each pair of table writes; on reopen either all four
|
|
landed or none did.
|
|
- Two runs in two tenants writing the same LSN both keep their outbox entries.
|
|
|
|
## Verify
|
|
|
|
**Harness:** a test-only fault hook inside the commit path, `panic_after_write(i)`
|
|
for `i in 0..4`, compiled under a `test-hooks` feature.
|
|
|
|
**Integration test** — `tests/it_commit_atomicity.rs`:
|
|
1. For each `i in 0..4`: run a commit that writes all four tables with the hook
|
|
armed at `i`.
|
|
2. Reopen the database in a **fresh process** (or at minimum a fresh `Database`
|
|
handle — reusing an open handle can mask a durability bug).
|
|
3. Count rows in all four tables. Assert the count vector is either
|
|
"all four present" or "none present". Any mixed vector fails.
|
|
4. Collision case: two runs in two tenants both commit LSN 1 with outbox
|
|
entries; assert both entries exist and are distinct after reopen.
|
|
5. Print the measured fsync duration for one commit, so the per-event cost stays
|
|
visible rather than being re-litigated from memory later.
|
|
|
|
**Command:** `cargo test -p storage --features test-hooks commit_atomicity`
|
|
|
|
**False pass:**
|
|
- The hook checked **before** `begin_write`, so no partial state is ever possible
|
|
and every `i` trivially passes.
|
|
- Asserting on the in-memory result rather than after reopen. The transaction
|
|
object will happily report success.
|
|
- Testing with `Durability::None` in the test config for speed. It passes and
|
|
proves nothing about the shipped path.
|
|
|
|
## Traps
|
|
|
|
- Outbox keyed on `Lsn` alone. It reads naturally and silently loses entries.
|
|
- Buffering state deltas in memory "for efficiency" and flushing at run end.
|
|
|
|
---
|
|
|
|
Background (not required to do this task):
|
|
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §8.1, §8.3 ·
|
|
[rust-agentic-task.md](../../../rust-agentic-task.md)
|