(tasks) add tasks for harness

2026-08-17 23:05:20 -07:00
parent e2d678b118
commit 5a30d0ffc6
74 changed files with 8204 additions and 1 deletions
@@ -0,0 +1,84 @@
+# T1.4 — Attempt lifecycle and retry
+
+| Field | Value |
+|---|---|
+| Phase | P1 — Walking skeleton |
+| Size | M — 1 to 3 days |
+| Status | Not started |
+| Flags | — |
+| Spec | inlined below |
+| Blocks | — |
+
+## Goal
+
+Retry creates attempt N+1 as a new record. Attempt N is never mutated.
+
+## Facts (inlined — no spec read needed)
+
+- Each step execution is one or more kernel attempts. **Retry creates attempt
+  N+1 and never mutates attempt N.** Two payoffs: "did the retry do better, and
+  why" becomes answerable, and log replay is idempotent for free.
+- Attempts within a step are **strictly serial** — a retry needs the prior
+  failure to exist first.
+- The immutability is also the foundation of the cheapest grading signal there
+  is: the attempts of one step are a comparison group on identical context, free
+  and already on disk (T5.10).
+- A failure and its successful retry are **not** a judge comparison. That pair is
+  consumed structurally — what differed between attempt N and N+1 is attributed
+  to the `StepId`. Only same-outcome attempts go to a judge.
+
+## Steps
+
+1. Key attempt state on `(BranchKey, AttemptNo)`. `AttemptNo` starts at 1 and
+   increments; no reuse within a branch.
+2. On retry, allocate `AttemptNo + 1` and write a fresh record. No update path to
+   an existing attempt row exists — do not write one.
+3. Apply the step's `RetryPolicy` (count and backoff) between attempts, awaiting
+   the cancellation token so a cancel during backoff is honoured.
+4. Record on each attempt what differs from its predecessor: context partition
+   (T1.5), tool selection, prompt ref (T1.6). That delta is what T5.10 attributes.
+5. Test: script the stub to fail twice then succeed. Snapshot attempts 1 and 2
+   before attempt 3 runs, snapshot again after, assert byte equality.
+
+## Acceptance
+
+- A step failing twice then succeeding produces **three** attempt records.
+- Attempts 1 and 2 are byte-identical before and after attempt 3.
+
+## Verify
+
+**Harness:** stub model scripted `fail, fail, succeed` for one step.
+
+**Integration test** — `tests/it_retry_immutability.rs`:
+1. Run the step; pause after attempt 2 completes.
+2. Snapshot the **serialized bytes** of attempt records 1 and 2.
+3. Let attempt 3 run to success.
+4. Re-read attempts 1 and 2; assert byte equality with the snapshot.
+5. Assert exactly three attempt records exist, numbered 1, 2, 3.
+6. Assert the attempts ran **serially**: attempt N's terminal timestamp precedes
+   attempt N+1's start.
+7. Assert the per-attempt delta fields (context partition, tool selection, prompt
+   ref) are present, so T5.10 has something to attribute.
+8. Cancel during retry backoff; assert the run stops rather than sleeping out the
+   full delay.
+
+**Command:** `cargo test -p executor retry`
+
+**False pass:**
+- Comparing attempt records through the query surface, which may reconstruct them
+  identically from the log even if the state table was mutated. Compare the
+  stored state rows **and** the log records.
+- Asserting "three records exist" alone — an implementation that appends a new
+  row and *also* updates row 2 passes that. Step 4 is what catches it.
+
+## Traps
+
+- "Updating" the attempt row with the final outcome to keep the table small. It
+  destroys the retry evidence, and the learning loop exists to consume it.
+- Retrying in parallel to save latency. Attempt N+1 needs N's failure.
+
+---
+
+Background (not required to do this task):
+[rust-agentic-sys.md](../../../rust-agentic-sys.md) §5.1, §5.3, §11.5, §11.8 ·
+[rust-agentic-task.md](../../../rust-agentic-task.md)