(tasks) add tasks for harness
This commit is contained in:
@@ -0,0 +1,103 @@
|
||||
# T1.7 — Episode query surface
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P1 — Walking skeleton |
|
||||
| Size | M — 1 to 3 days |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | — |
|
||||
|
||||
## Goal
|
||||
|
||||
Read an episode — run, branches, attempts, refs — with identical results whether
|
||||
served from materialized state or a cold re-fold.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
```rust
|
||||
pub struct AttemptView {
|
||||
pub step: StepId,
|
||||
pub attempt: AttemptNo,
|
||||
pub state: AttemptState,
|
||||
pub workflow_version: WorkflowVersion,
|
||||
pub context: Option<ContextPartition>, // identifiers, small, inline
|
||||
pub prompt: Option<BlobRef>, // large, by reference
|
||||
pub output: Option<BlobRef>,
|
||||
pub tools: Option<ToolInfo>,
|
||||
pub usage: Option<Usage>,
|
||||
}
|
||||
```
|
||||
|
||||
- **Include failed attempts.** "Retried three times because context was missing
|
||||
X" is the learning signal; shipping only the winning attempt discards it.
|
||||
- Queries default to the **live branch**; grading may read all branches. Only the
|
||||
live branch is exported.
|
||||
- Materialized state is a cache of the log. If the query answers differently from
|
||||
a cold re-fold, the state has hidden inputs — that is T0.8's property applied
|
||||
to the read path.
|
||||
- Blob bodies are **not** included. The view carries refs; the caller fetches
|
||||
what it needs (T4.3).
|
||||
|
||||
## Steps
|
||||
|
||||
1. Define `EpisodeView` — run metadata, branch list with fork points, attempts
|
||||
per branch — and `AttemptView` exactly as above.
|
||||
2. Implement the query against materialized state, defaulting to the live branch
|
||||
with an explicit `all_branches` option for grading.
|
||||
3. Implement the same query against a cold re-fold (T0.8's `rebuild`), used as
|
||||
the fallback when state is absent and as the test oracle.
|
||||
4. Include every attempt regardless of outcome. Sort by `(BranchId, AttemptNo)`
|
||||
so ordering is stable.
|
||||
5. Test: build a run with a retry and a rewind; serialize the view from state and
|
||||
from a cold re-fold; assert equal.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Query returns the same structure whether served from materialized state or a
|
||||
cold re-fold.
|
||||
- Failed attempts are present in the returned view.
|
||||
|
||||
## Phase gate
|
||||
|
||||
P1 closes when a full run is recorded and re-derives byte-identically.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** one recorded run containing a retry **and** a rewind — the two cases
|
||||
where the two read paths can diverge.
|
||||
|
||||
**Integration test** — `tests/it_episode_query_equivalence.rs`:
|
||||
1. Build the run: 3 steps, one step failing twice, then a rewind at step 2
|
||||
producing a second branch.
|
||||
2. Query the episode from **materialized state** → `view_a`.
|
||||
3. Drop the state tables; query again, served by cold re-fold → `view_b`.
|
||||
4. Assert `serialize(view_a) == serialize(view_b)`.
|
||||
5. Assert `view_a` contains **all** attempts including the two failures — count
|
||||
them explicitly, do not just check non-empty.
|
||||
6. Assert the default query returns only the **live** branch, and that the
|
||||
explicit all-branches mode returns both.
|
||||
7. Instrument `BlobStore::get`; assert the query itself makes **zero** calls.
|
||||
|
||||
**Command:** `cargo test -p query episode_equivalence`
|
||||
|
||||
**False pass:**
|
||||
- Testing on a linear run with no retry and no rewind. Both paths trivially
|
||||
agree, and the divergence lives exactly in the cases omitted.
|
||||
- Step 5 as `assert!(!attempts.is_empty())`. A view returning only the winner
|
||||
passes that.
|
||||
- Step 4 with a `PartialEq` derived over unordered collections.
|
||||
|
||||
## Traps
|
||||
|
||||
- Filtering to the successful attempt "because that is what the UI shows". The
|
||||
learning loop is the other consumer and it needs the failures.
|
||||
- Eagerly resolving `BlobRef` into text inside the view. That is T4.3's
|
||||
regression, introduced one layer earlier.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §8.5, §10.1 ·
|
||||
[rust-agentic-task.md](../../../rust-agentic-task.md)
|
||||
Reference in New Issue
Block a user