(tasks) add tasks for harness
This commit is contained in:
@@ -0,0 +1,123 @@
|
||||
# T2.1 — Write-ahead intent
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P2 — Durability hard parts |
|
||||
| Size | L — over 3 days |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | T2.2, T6.4 |
|
||||
|
||||
## Goal
|
||||
|
||||
Three intent records around every external effect, so a crash can be classified
|
||||
rather than guessed at.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
```
|
||||
1. append Intent{Pending} ──► commit + fsync
|
||||
◄── window A: crash here, the call was never issued
|
||||
2. append Intent{Dispatched} ──► commit + fsync
|
||||
3. perform the call
|
||||
◄── window B: crash here, the call may have landed
|
||||
4. append outcome, Intent{Committed} ──► commit + fsync
|
||||
```
|
||||
|
||||
- **Three records, not two.** Two cannot separate the windows: a crash before the
|
||||
call and a crash after it both leave a lone `Pending` with no outcome, which
|
||||
makes every interrupted effect maximally suspicious and pushes recoverable work
|
||||
into `Indeterminate`. The second fsync buys the distinction, is paid only on
|
||||
steps with external effects, and is small next to the call it guards.
|
||||
- Appending after the fact records history but does not make a failed step
|
||||
resumable — the dangerous window is *before* the record exists.
|
||||
- Restart classification, from the last committed intent state:
|
||||
|
||||
| Last state | Meaning | Resolution |
|
||||
|---|---|---|
|
||||
| `Pending` | the call was never issued | retry freely, whatever the effect class |
|
||||
| `Dispatched` | the call *may* have been issued | by effect class (T2.2) |
|
||||
| `Committed` | outcome already recorded | nothing to do |
|
||||
|
||||
- Intents always commit with `Durability::Immediate`. **Batching them defeats
|
||||
their only purpose.**
|
||||
- Tool calls need this more than model calls. A model call is a metered read; a
|
||||
tool call writes files, pushes commits, and touches the world.
|
||||
- The intent record is also what decides `Cancelled` vs `Indeterminate` on a
|
||||
cancelled attempt (T0.2, T1.1).
|
||||
|
||||
## Steps
|
||||
|
||||
1. Define `IntentState { Pending, Dispatched, Committed }` and an `IntentId`
|
||||
scoped to `(BranchKey, AttemptNo, step)`. Include the idempotency key or
|
||||
request id the effect class will need at recovery (T2.2).
|
||||
2. Wrap every external effect in the four-step sequence above. Each append is its
|
||||
own `EventLog::commit` with `Immediate` durability — three separate fsyncs, by
|
||||
design.
|
||||
3. Make the wrapper the **only** way to reach a tool. If a call site can issue an
|
||||
effect without an intent, the protocol is decorative.
|
||||
4. Build the crash harness: a test hook that aborts the process at a named point.
|
||||
Two named points minimum — after `Pending` commit, and after `Dispatched`
|
||||
commit but before outcome commit.
|
||||
5. On restart, scan for the newest intent per `(BranchKey, AttemptNo)` and
|
||||
classify per the table. Emit the classification as a log event so the decision
|
||||
itself is recorded.
|
||||
6. Assert the fsync count on the intent path in a test — a batching "optimization"
|
||||
introduced later must fail loudly.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Crash harness kills the process in both windows: between `Pending` and
|
||||
`Dispatched`, and between `Dispatched` and outcome-commit.
|
||||
- Restart classifies the first as "never sent" and the second as "may have been
|
||||
sent". A two-phase implementation cannot pass this — both crashes leave one
|
||||
`Pending` with no outcome and are indistinguishable. That is the point of the
|
||||
test.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** a real child-process kill (`kill -9`), not a panic. Named abort
|
||||
points compiled under a `test-hooks` feature. A recording tool that appends to a
|
||||
file outside the database, so "did the call actually happen" is observable
|
||||
independently of what the log claims.
|
||||
|
||||
**Integration test** — `tests/it_intent_windows.rs`:
|
||||
1. **Window A:** arm the abort after the `Pending` commit, before `Dispatched`.
|
||||
Kill. Reopen.
|
||||
- Assert the last intent state is `Pending`.
|
||||
- Assert restart classifies it **"never sent"**.
|
||||
- Assert the side-channel file is **empty** — the call really did not happen.
|
||||
2. **Window B:** arm the abort after the `Dispatched` commit, before the
|
||||
outcome commit. Kill. Reopen.
|
||||
- Assert the last intent state is `Dispatched`.
|
||||
- Assert restart classifies it **"may have been sent"**.
|
||||
3. Assert the two classifications **differ**. A two-phase implementation makes
|
||||
them identical, which is the whole point of the test.
|
||||
4. Count fsyncs on the intent path for one guarded effect; assert exactly 3
|
||||
commits, so a later batching "optimization" fails loudly.
|
||||
5. Bypass audit: assert no tool can be invoked except through the intent wrapper —
|
||||
a compile-level guard if the tool handle is only constructible inside it.
|
||||
|
||||
**Command:** `cargo test -p durability --features test-hooks intent_windows -- --test-threads=1`
|
||||
|
||||
**False pass:**
|
||||
- Simulating the crash with `panic!` and catching the unwind. Destructors run and
|
||||
buffered writes flush — precisely what a real crash does not do.
|
||||
- Asserting only that both windows "recover". Both recover under a two-phase
|
||||
implementation too; they just recover **identically**. Step 3 is the test.
|
||||
- Trusting the log to tell you whether the call happened. Step 1's side-channel
|
||||
file is the independent witness.
|
||||
|
||||
## Traps
|
||||
|
||||
- Batching the two pre-call fsyncs into one commit. It compiles, it is faster,
|
||||
and it deletes the distinction the task exists to create.
|
||||
- Recording the intent after dispatch "since that is when we know the request
|
||||
id". Derive the request id before dispatch instead.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §8.4 ·
|
||||
[rust-agentic-task.md](../../../rust-agentic-task.md)
|
||||
Reference in New Issue
Block a user