(plan) update the plan to captures more
This commit is contained in:
+31
-1
@@ -211,7 +211,8 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked
|
||||
| 8 | Learning loop | T6.x | 8 | 0 | 0 | 8 | ⬜ T6.8 | — | — |
|
||||
| 9 | Distribution | T7.x | 8 | 0 | 0 | 8 | ⬜ T7.8 | — | — |
|
||||
| 10 | Operability | T8.x | 7 | 0 | 0 | 7 | ⬜ T8.7 | — | — |
|
||||
| | **Total** | | **80** | **3** | **0** | **77** | 0/10 green | **313.0k** | **—** |
|
||||
| 11 | Orchestration | T10.x | 10 | 0 | 0 | 10 | ⬜ T10.10 | — | — |
|
||||
| | **Total** | | **90** | **3** | **0** | **87** | 0/11 green | **313.0k** | **—** |
|
||||
|
||||
**What changed in this revision.** The public surface moved ahead of the
|
||||
durability hard parts, and an embedding phase was added after verification.
|
||||
@@ -361,3 +362,32 @@ whose architecture already expects capabilities to arrive as swappable providers
|
||||
| [T8.5](T8.5-embedded-mode-smoke.md) | Embedded-mode smoke | S | parallel-ok | ⬜ |
|
||||
| [T8.6](T8.6-capacity-admission-control.md) | Capacity admission control | M | — | ⬜ |
|
||||
| [T8.7](T8.7-p8-composition-gate.md) | **Operability composition gate** | L | gate | ⬜ |
|
||||
|
||||
## 11 — Orchestration · T10.x
|
||||
|
||||
Fills §17's deferred line — multi-agent orchestration, revisited now that
|
||||
cross-agent coordination is real. A `Reconciler` port picks up a run at its
|
||||
existing `Verified`/`Graded`/`Ungraded` resting states, decides what runs
|
||||
next toward a `GoalId`-scoped objective, and dispatches through Kafka via
|
||||
the same outbox/inbox discipline T7.4 established — never a direct spawn
|
||||
call. An HTTP surface binds one method table shared with T9.2's eventual
|
||||
stdio transport. Two front doors, one IR: `tasks/*.md` parsed directly
|
||||
(door a), or a free-text problem statement drafted and parked in
|
||||
`Suspended` for human approval before it runs (door b).
|
||||
|
||||
Depends on T1.x, T2.1/T2.3, T3.x, T7.3/T7.4, T9.1 landing first; not blocked
|
||||
on T9.2–T9.6 (the stdio binding develops in parallel against the same
|
||||
method table T10.4 authors).
|
||||
|
||||
| Task | Title | Size | Flags | Status |
|
||||
|---|---|---|---|---|
|
||||
| [T10.1](T10.1-goalid-and-goal-scoped-query.md) | `GoalId` and goal-scoped query | S | — | ⬜ |
|
||||
| [T10.2](T10.2-reconciler-port.md) | `Reconciler` port | M | — | ⬜ |
|
||||
| [T10.3](T10.3-idempotent-run-admission.md) | Idempotent run admission | S | — | ⬜ |
|
||||
| [T10.4](T10.4-http-api-surface.md) | HTTP API surface | L | — | ⬜ |
|
||||
| [T10.5](T10.5-taskboardformat-plugin.md) | `TaskBoardFormat` plugin | M | — | ⬜ |
|
||||
| [T10.6](T10.6-plan-draft-and-suspended-approval.md) | `plan-draft` workflow + `Suspended`-approval flow | M | — | ⬜ |
|
||||
| [T10.7](T10.7-kafka-topics-and-inbox-relay.md) | Kafka topics and inbox relay | L | — | ⬜ |
|
||||
| [T10.8](T10.8-orchestrator-reference-service.md) | Orchestrator reference service | M | — | ⬜ |
|
||||
| [T10.9](T10.9-cli-transcript-and-metrics-ux.md) | CLI: runs list/episode/transcript | S | — | ⬜ |
|
||||
| [T10.10](T10.10-p10-composition-gate.md) | **Orchestration composition gate** | L | gate | ⬜ |
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
# T10.1 — `GoalId` and goal-scoped query
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P10 — Orchestration |
|
||||
| Size | S — under 1 day |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | T10.2, T10.3, T10.4, T10.8 |
|
||||
|
||||
## Goal
|
||||
|
||||
New identity `GoalId(Ulid)` grouping runs that share an objective, optional at
|
||||
spawn. Extend `T1.7`'s query surface to read across `RunId`s sharing one
|
||||
`GoalId`.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
- `GoalId` is tenant-scoped like `RunId`, **not** `TaskId`. `TaskId` is already
|
||||
the grading comparison-group key (§11.5) — reusing it corrupts
|
||||
group-epoch/quorum semantics the learning loop depends on.
|
||||
- Attaching `goal_id` at spawn is **optional**. A run with no `GoalId` behaves
|
||||
exactly as today — this is additive, not a required field.
|
||||
- `GoalView` is a new, separate query. It does not replace per-run
|
||||
`EpisodeView`; it lists member `RunId`s and references their existing
|
||||
metadata, no duplication.
|
||||
- Same parity rule as T1.7: `GoalView` from materialized state must equal
|
||||
`GoalView` from a cold re-fold (T0.8's property applied here too).
|
||||
|
||||
## Steps
|
||||
|
||||
1. Declare `GoalId(Ulid)` in the `ids` crate, alongside `RunId`.
|
||||
2. Add optional `goal_id: Option<GoalId>` to run-spawn input and the run
|
||||
record.
|
||||
3. Index runs by `(TenantId, GoalId)` in materialized state.
|
||||
4. Define `GoalView { goal: GoalId, runs: Vec<RunSummary> }`, `RunSummary`
|
||||
referencing existing run metadata — do not duplicate `EpisodeView` fields.
|
||||
5. Implement the query against materialized state and against cold re-fold.
|
||||
6. Test: two runs sharing a `GoalId`, one plain run with none; assert
|
||||
`GoalView` returns exactly the two, in spawn order.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Two runs sharing a `GoalId` are both returned by `GoalView`; a run with no
|
||||
`GoalId` never appears in any `GoalView`.
|
||||
- `GoalView` from materialized state equals `GoalView` from cold re-fold.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** two runs sharing a `GoalId`, one run with no `GoalId`, one
|
||||
branch/rewind case reused from `T1.7`'s harness.
|
||||
|
||||
**Integration test** — `tests/it_goal_view_equivalence.rs`:
|
||||
1. Spawn run A and run B both with `goal_id=G`; spawn run C with none.
|
||||
2. Query `GoalView(G)` from materialized state → `view_a`.
|
||||
3. Drop state tables; query again via cold re-fold → `view_b`.
|
||||
4. Assert `serialize(view_a) == serialize(view_b)`.
|
||||
5. Assert `view_a` contains exactly `{A, B}`, never `C`.
|
||||
6. Assert order is spawn order, stable.
|
||||
|
||||
**Command:** `cargo test -p query goal_view_equivalence`
|
||||
|
||||
**False pass:**
|
||||
- Testing only one run per goal — a grouping bug is invisible with a single
|
||||
member.
|
||||
- Skipping the "run without `goal_id`" exclusion check. A filter that
|
||||
defaults to "all runs" passes silently without it.
|
||||
|
||||
## Traps
|
||||
|
||||
- Deriving `GoalId` from `TaskId` "since they're both groupings." Different
|
||||
lifecycle, different owner — grading vs orchestration.
|
||||
- Making `goal_id` required at spawn. Breaks every existing plain-workflow
|
||||
spawn call.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[rust-agentic-sys.md](../rust-agentic-sys.md) §11.5 ·
|
||||
[T1.7-episode-query-surface.md](T1.7-episode-query-surface.md) ·
|
||||
[T0.8-fold-and-re-derive.md](T0.8-fold-and-re-derive.md)
|
||||
@@ -0,0 +1,88 @@
|
||||
# T10.10 — Orchestration composition gate
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P10 — Orchestration |
|
||||
| Size | L — over 3 days |
|
||||
| Status | Not started |
|
||||
| Flags | gate |
|
||||
| Spec | inlined below |
|
||||
| Blocks | — |
|
||||
|
||||
## Goal
|
||||
|
||||
Prove the properties no single T10.x task owns: a full goal chain survives a
|
||||
real Kafka outage, dispatch is idempotent under forced redelivery,
|
||||
`Suspended`-approval produces byte-identical IR to hand-authored YAML, and
|
||||
the HTTP method table stands as a published artifact.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
- Mirrors T7.4's fault-injection shape — broker down for the whole test —
|
||||
extended across the full events → reconcile → dispatch → inbox-relay →
|
||||
spawn loop, not just outbox delivery.
|
||||
- Front-door equivalence for door (b) extends T3.10's existing hash-equality
|
||||
gate — same discipline, new door.
|
||||
- Gate script follows board convention: one script, diffed against a
|
||||
committed expected output, empty diff is the only pass.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Compose a fixture: one goal, a 2-run chain (A → reconcile → B → reconcile
|
||||
→ `Stop`), driven through real HTTP calls and real Kafka topics.
|
||||
2. Kill the broker mid-chain — after A completes, before the dispatch
|
||||
message for B is consumed; assert the chain still completes once the
|
||||
broker returns, with B eventually spawned.
|
||||
3. Force redelivery of the `.dispatch` message for B; assert exactly one
|
||||
run B exists.
|
||||
4. Submit a problem statement through door (b), edit the draft, approve;
|
||||
separately hand-author the identical edited content as YAML and load it
|
||||
through door (a); assert both `WorkflowVersion` hashes match.
|
||||
5. Snapshot the HTTP method table (paths + request/response shapes) to a
|
||||
committed fixture; assert no path removed and no field removed relative
|
||||
to the fixture — additions are fine, removals fail the gate.
|
||||
6. Assert a goal-stall metric fires for a deliberately broken `Reconciler`
|
||||
on one goal while a healthy goal in the same tenant proceeds unaffected
|
||||
(T10.8's concurrency claim, at gate scope).
|
||||
|
||||
## Acceptance
|
||||
|
||||
- All six checks above pass with an empty diff against
|
||||
`verify/expected/T10.10.txt`.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** full stack — embedded `redb`, real Kafka (or the board's
|
||||
chosen fault-injection substrate), `axum` API instance, stub `Reconciler`
|
||||
with scripted multi-step decisions.
|
||||
|
||||
**Integration test** — `tests/it_p10_composition_gate.rs`, one numbered
|
||||
assertion per Steps item above (`a1`..`a6`), reported by name per the
|
||||
board's script convention.
|
||||
|
||||
**Command:** `verify/T10.10.sh`, diffed against `verify/expected/T10.10.txt`.
|
||||
|
||||
**False pass:**
|
||||
- Running the chain with the broker never actually taken down — the
|
||||
"survives an outage" claim needs the outage to really happen mid-chain,
|
||||
not just be configured as theoretically tolerable.
|
||||
- Skipping the method-table snapshot diff — a route or field silently
|
||||
removed here breaks every embedder bound to the table (HTTP today, stdio
|
||||
later) without any single T10.x task's own tests catching it, since none
|
||||
of them own the whole table.
|
||||
|
||||
## Traps
|
||||
|
||||
- Treating this gate as "run all T10.x tests together." It must assert
|
||||
properties none of them individually assert, per the board's own gate
|
||||
discipline (INDEX.md, "Verification practice," point 5).
|
||||
- Committing the method-table fixture as freeform prose instead of a
|
||||
diffable, structured snapshot. An unreviewable diff defeats the point of
|
||||
having a fixture at all.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[../INDEX.md](../INDEX.md) (verification practice) ·
|
||||
[T7.4-outbox-relay.md](T7.4-outbox-relay.md) ·
|
||||
[T3.10-front-door-equivalence.md](T3.10-front-door-equivalence.md)
|
||||
@@ -0,0 +1,112 @@
|
||||
# T10.2 — `Reconciler` port
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P10 — Orchestration |
|
||||
| Size | M — 1 to 3 days |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | T10.6, T10.8 |
|
||||
|
||||
## Goal
|
||||
|
||||
New port answering "how far toward the goal, what next" from a run's terminal
|
||||
resting state — distinct from `Verifier` (fail-closed pass/fail) and `Judge`
|
||||
(relative-only comparison).
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
- `Verifier::verify` is fail-closed pass/fail (rust-agentic-sys.md §10).
|
||||
`Judge::compare` is deliberately relative-only, "cannot return an absolute
|
||||
score" (rust-agentic-sys.md §11.1). "How far toward goal, what next" is a
|
||||
third question — forcing it through either port blurs a firewall built on
|
||||
purpose. `Reconciler` is new, sized like `Verifier` (one method, S/M).
|
||||
- Fires **only** off `Verified{pass|fail}`, `Graded`, `Ungraded` — states
|
||||
`T1.3` already designed to rest in, never resolved inline.
|
||||
- Reconciler failure degrades the record, never the run — same principle as
|
||||
§1: "the framework cannot break the agent it runs." A `Reconciler` error or
|
||||
timeout leaves the run at its terminal state; the goal simply does not
|
||||
advance until retried.
|
||||
- The decision does **not** spawn anything itself. Spawning happens only
|
||||
through the admission path (T10.3) via the inbox relay (T10.7) — never a
|
||||
direct call from the `Reconciler` or its caller.
|
||||
- One new `#[non_exhaustive]` `WorkEvent` variant for provenance:
|
||||
`SpawnedFromReconciliation{goal_id, parent_run, decision_ref}`. Added, not
|
||||
replacing any existing variant.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Define `ReconcileCtx` — `GoalId`, `GoalView` (T10.1), the just-terminal
|
||||
run's `EpisodeView`, `WorkflowDef`.
|
||||
2. Define `ReconcileDecision` — `SpawnNext{workflow_ref, input, dispatch_key}`,
|
||||
`Retry{step}`, `Stop{reason}`.
|
||||
3. Define the trait:
|
||||
```rust
|
||||
#[async_trait]
|
||||
pub trait Reconciler: Send + Sync {
|
||||
fn id(&self) -> ReconcilerId;
|
||||
async fn reconcile(&self, cx: &ReconcileCtx) -> Result<ReconcileDecision>;
|
||||
}
|
||||
```
|
||||
4. Wire the trigger: on a run reaching `Verified`/`Graded`/`Ungraded` with a
|
||||
`GoalId` set, call the `Reconciler` bound to that goal.
|
||||
5. On `Reconciler` error or timeout: log, leave the run at its terminal state,
|
||||
do not auto-retry — matches `Verifier`'s explicit-only-retry discipline.
|
||||
6. Add `WorkEvent::SpawnedFromReconciliation`, written on the **next**
|
||||
spawned run's first log record, referencing `parent_run` and the decision.
|
||||
7. Test with a stub `Reconciler` against all three trigger states.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- A run reaching `Verified{pass}` with a `GoalId` triggers exactly one
|
||||
reconcile call.
|
||||
- The decision is durably recorded and traceable from the next spawned run
|
||||
back to the parent run and goal.
|
||||
- A `Reconciler` error leaves the run at its terminal state and does not
|
||||
crash the executor.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** stub `Reconciler` returning each decision variant in turn; one
|
||||
goal with a 2-run chain.
|
||||
|
||||
**Integration test** — `tests/it_reconciler_trigger.rs`:
|
||||
1. Run A (goal G) reaches `Verified{pass}`; assert `reconcile()` called
|
||||
exactly once with the correct `ReconcileCtx`.
|
||||
2. Stub returns `SpawnNext`; assert the decision is persisted, referencing
|
||||
run A.
|
||||
3. Assert the next spawned run's first log record carries
|
||||
`SpawnedFromReconciliation{goal_id: G, parent_run: A, decision_ref}`.
|
||||
4. Repeat for `Graded` and `Ungraded` trigger states.
|
||||
5. Stub `Reconciler` panics or times out; assert run A stays at its terminal
|
||||
state, no crash, no infinite retry.
|
||||
6. Run with no `GoalId` reaches `Verified{pass}`; assert `reconcile()` is
|
||||
never called.
|
||||
|
||||
**Command:** `cargo test -p kernel reconciler_trigger`
|
||||
|
||||
**False pass:**
|
||||
- Asserting reconcile "ran" via a boolean flag instead of asserting the
|
||||
`WorkEvent` provenance record exists — passes even if a crash lost the
|
||||
decision before persistence.
|
||||
- Testing only `Verified{pass}`, skipping `Graded`/`Ungraded` — those are the
|
||||
states most likely miswired, being newer additions to the FSM.
|
||||
|
||||
## Traps
|
||||
|
||||
- `Reconciler` spawning the next run directly. Violates "never bypass the
|
||||
admission path" — see T10.3/T10.7.
|
||||
- Overloading `Judge::compare` to also answer "what next." Collapses a
|
||||
boundary `Judge`'s signature deliberately enforces (rust-agentic-sys.md
|
||||
§11.1, "deliberately cannot return an absolute score").
|
||||
- Auto-retrying a failed `Reconciler` call with no cap — a broken impl then
|
||||
spins forever against every terminal run in the tenant.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[rust-agentic-sys.md](../rust-agentic-sys.md) §1, §10, §11.1, §17 ·
|
||||
[T1.3-run-executor.md](T1.3-run-executor.md) ·
|
||||
[T4.1-verifier-port.md](T4.1-verifier-port.md) ·
|
||||
[T10.1-goalid-and-goal-scoped-query.md](T10.1-goalid-and-goal-scoped-query.md)
|
||||
@@ -0,0 +1,78 @@
|
||||
# T10.3 — Idempotent run admission
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P10 — Orchestration |
|
||||
| Size | S — under 1 day |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | T10.4, T10.7 |
|
||||
|
||||
## Goal
|
||||
|
||||
`dispatch_key` on spawn requests: a redelivered spawn request (from Kafka
|
||||
at-least-once delivery, T10.7) becomes a no-op insert instead of a duplicate
|
||||
run.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
- Same principle as the outbox's `(BranchKey, Lsn)` fold-key idempotency
|
||||
(T7.4) — exactly-once is achieved at the admission boundary, not in
|
||||
transport.
|
||||
- `dispatch_key` is caller-supplied, opaque, unique per intended spawn — e.g.
|
||||
a `Reconciler` decision id.
|
||||
- Admission with no `dispatch_key` behaves exactly as today. This is
|
||||
additive, not a required field.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Add optional `dispatch_key: Option<String>` to spawn-run input.
|
||||
2. Add a unique index on `(TenantId, dispatch_key)` where `dispatch_key` is
|
||||
set.
|
||||
3. On spawn: if `dispatch_key` is set and already seen, return the existing
|
||||
`RunId` instead of creating a new run — no error, no duplicate.
|
||||
4. Test redelivery: same `dispatch_key` submitted twice, concurrently and
|
||||
sequentially.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Two spawn calls with the same `dispatch_key` produce exactly one run; the
|
||||
second call returns the first run's `RunId`.
|
||||
- Spawn calls with no `dispatch_key` are unaffected.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** spawn endpoint (T10.4) or direct kernel call, two concurrent
|
||||
callers.
|
||||
|
||||
**Integration test** — `tests/it_idempotent_admission.rs`:
|
||||
1. Spawn with `dispatch_key=K`; assert one run created, `RunId=R`.
|
||||
2. Spawn again with `dispatch_key=K`; assert no new run created, returned
|
||||
`RunId == R`.
|
||||
3. Fire 20 concurrent spawn calls with the same `dispatch_key`; assert
|
||||
exactly one run exists after all resolve.
|
||||
4. Spawn with no `dispatch_key` twice; assert two distinct runs (unaffected
|
||||
baseline).
|
||||
|
||||
**Command:** `cargo test -p kernel idempotent_admission -- --test-threads=1`
|
||||
|
||||
**False pass:**
|
||||
- Testing only sequential redelivery. Concurrent redelivery is the actual
|
||||
failure mode under Kafka rebalance/retry and needs the unique-index race
|
||||
caught, not a check-then-insert race.
|
||||
|
||||
## Traps
|
||||
|
||||
- Making `dispatch_key` required. Breaks every direct/manual spawn call that
|
||||
isn't part of a goal chain.
|
||||
- Deduping by a hash of `(workflow_ref, input)` instead of the caller-supplied
|
||||
key. Two legitimately identical requests would then silently collapse into
|
||||
one run.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[rust-agentic-sys.md](../rust-agentic-sys.md) §8.3, §9.3 ·
|
||||
[T7.4-outbox-relay.md](T7.4-outbox-relay.md) ·
|
||||
[T3.6-version-pinning-at-spawn.md](T3.6-version-pinning-at-spawn.md)
|
||||
@@ -0,0 +1,101 @@
|
||||
# T10.4 — HTTP API surface
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P10 — Orchestration |
|
||||
| Size | L — over 3 days |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | T10.6, T10.7, T10.9 |
|
||||
|
||||
## Goal
|
||||
|
||||
`axum` HTTP service binding one method table — spawn / status / stream /
|
||||
episode / transcript / rewind / cancel / retry / goal / metrics — to
|
||||
`poimen-sdk` ports. One method set: HTTP today, the same table T9.2's stdio
|
||||
JSON-RPC binds to later.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
- **The protocol is a published artifact** — INDEX.md's own rule for the
|
||||
sidecar applies equally here: methods added, never repurposed; fields
|
||||
added, never removed, from the first commit.
|
||||
- HTTP is a transport binding onto `poimen-sdk` (T9.1), not a bespoke access
|
||||
layer with its own business logic. Handlers translate, they do not decide.
|
||||
- `/v1/problems` and `/v1/plans/{id}/approve|reject` route to T10.6's
|
||||
workflow, not implemented here — this task defines the routes and
|
||||
request/response contracts; T10.6 fills in plan-draft semantics.
|
||||
- `/v1/runs/{id}/stream` tails the same committed log the outbox relay reads
|
||||
— SSE, not a second read path.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Stand up the `axum` service, one route module per resource: runs, plans,
|
||||
goals, metrics.
|
||||
2. `POST /v1/runs` → spawn_run. Request `{tenant, workflow_ref, input,
|
||||
goal_id?, dispatch_key?}`. Response `{run_id, status}`.
|
||||
3. `GET /v1/runs/{id}`, `/episode` (T1.7), `/attempts/{step}/{n}/transcript`
|
||||
(T1.6 `BlobRef` dereference), `/stream` (SSE).
|
||||
4. `POST /v1/runs/{id}/rewind` (T2.3), `/cancel`, `/attempts/{step}/retry`
|
||||
(T1.4).
|
||||
5. `GET /v1/goals/{id}` (T10.1 `GoalView`).
|
||||
6. `GET /v1/metrics`, `/v1/runs/{id}/metrics` (T8.2's existing list, scoped).
|
||||
7. Auth/tenant resolution: reject any request without a resolvable
|
||||
`TenantId` before touching the kernel — no implicit tenant.
|
||||
8. Version the route set from commit one: additive only, document the rule
|
||||
inline in the router module.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Full request/response cycle for spawn → poll → episode → transcript
|
||||
against an embedded `redb` backend.
|
||||
- Rewind, cancel, retry each produce the same log-level effect as calling
|
||||
the underlying kernel function directly — no HTTP-layer divergence.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** embedded `redb`, stub model, `axum` test client
|
||||
(`tower::ServiceExt::oneshot`).
|
||||
|
||||
**Integration test** — `tests/it_http_api_surface.rs`:
|
||||
1. `POST /v1/runs`; assert 200 with `run_id`; assert kernel shows
|
||||
`Scheduled`.
|
||||
2. `GET /v1/runs/{id}` until terminal; assert status matches kernel state
|
||||
exactly at each poll.
|
||||
3. `GET /v1/runs/{id}/episode`; assert shape matches T1.7's `EpisodeView`,
|
||||
includes failed attempts.
|
||||
4. `GET /v1/runs/{id}/attempts/{step}/{n}/transcript`; assert returned bytes
|
||||
match `BlobStore` content directly (T1.6).
|
||||
5. `POST .../rewind`; assert new `BranchId` created, parent branch still
|
||||
readable via all-branches.
|
||||
6. `POST .../retry`; assert new `AttemptNo`, prior attempt record
|
||||
unmodified.
|
||||
7. `POST .../cancel` mid-run; assert `Cancelled` reached (matches T1.3's
|
||||
cancel matrix).
|
||||
8. Request with no resolvable tenant; assert rejected before any kernel call
|
||||
— instrument kernel entrypoints, assert zero calls.
|
||||
|
||||
**Command:** `cargo test -p api http_api_surface`
|
||||
|
||||
**False pass:**
|
||||
- Asserting HTTP 200 without checking underlying kernel state — a route that
|
||||
accepts and silently drops a request passes this.
|
||||
- Testing transcript retrieval only for a successful attempt — failed-attempt
|
||||
transcripts are the debugging case that matters most.
|
||||
|
||||
## Traps
|
||||
|
||||
- Putting business logic (retry policy, plan validation) in the HTTP
|
||||
handler instead of the kernel/port layer. The handler is a transport, not
|
||||
a second brain.
|
||||
- Resolving `BlobRef`s eagerly inside `/episode` "for convenience" — T4.3's
|
||||
regression, one layer up.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §8.5, §10.1 ·
|
||||
[../../INDEX.md](../../INDEX.md) (embedding contract, ~line 60-86) ·
|
||||
[T1.7-episode-query-surface.md](T1.7-episode-query-surface.md) ·
|
||||
[T1.6-prompt-and-output-blob-capture.md](T1.6-prompt-and-output-blob-capture.md)
|
||||
@@ -0,0 +1,96 @@
|
||||
# T10.5 — `TaskBoardFormat` plugin
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P10 — Orchestration |
|
||||
| Size | M — 1 to 3 days |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | — |
|
||||
|
||||
## Goal
|
||||
|
||||
Front door (a): parse `tasks/*.md` — this board's own format — into a
|
||||
`WorkflowDef` via the existing `WorkflowFormat` plugin trait. Zero kernel
|
||||
change.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
- `WorkflowFormat` is already designed as a plugin point — this adds a third
|
||||
format implementation, not a new mechanism. YAML and the Rust builder are
|
||||
the other two; both canonicalize to the same IR (INDEX.md's customization
|
||||
contract).
|
||||
- Step order comes from **phase + row order in INDEX.md**, not
|
||||
filename/id order. Id order is explicitly not execution order (INDEX.md's
|
||||
own "Ordering" section).
|
||||
- Each task file's Acceptance/Verify/Command sections map to a `StepDef`'s
|
||||
verifier binding and completion criteria. Capability ids referenced
|
||||
(verifier/judge/model) must already be registered (T3.9) or load fails
|
||||
naming the id and file — same contract as YAML.
|
||||
- Read-only relative to the board: this parses `tasks/*.md`, it does not
|
||||
write `Status` back. That stays out of scope here.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Implement `TaskBoardFormat: WorkflowFormat`, `load(path: &Path) ->
|
||||
Result<WorkflowDef>` reading a directory of `T*.md` + `INDEX.md`.
|
||||
2. Parse `INDEX.md`'s phase tables for execution order; parse each task
|
||||
file's Goal/Acceptance/Verify sections into one `StepDef` per task.
|
||||
3. Unresolved verifier/judge/model ids referenced in a task file fail at
|
||||
load, naming the id and file — same contract as the other two front
|
||||
doors.
|
||||
4. Canonicalize through the same Blake3 hashing as the other two front doors
|
||||
(T3.1) — same `WorkflowDef` IR.
|
||||
5. Test: load this board's own `tasks/` directory, assert the canonicalized
|
||||
IR is stable across two loads (determinism), and that step order matches
|
||||
`INDEX.md`'s declared phase order, not directory-listing order.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- `agent-rust/tasks/` loads into a `WorkflowDef` with step order matching
|
||||
`INDEX.md`'s declared phase order.
|
||||
- A task file referencing an unregistered verifier id fails at load, naming
|
||||
the id and file — not at spawn.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** this repo's own `agent-rust/tasks/` directory as the test
|
||||
fixture — self-hosting, no synthetic fixture needed.
|
||||
|
||||
**Integration test** — `tests/it_taskboard_format.rs`:
|
||||
1. Load `agent-rust/tasks/`; assert no error.
|
||||
2. Assert step count matches `INDEX.md`'s task count.
|
||||
3. Assert step order matches `INDEX.md`'s phase-table order, not `ls`/
|
||||
filename order — deliberately scramble the directory-read order in the
|
||||
test harness to catch an implementation that trusts `readdir` order.
|
||||
4. Load twice; assert canonicalized `WorkflowVersion` hash identical both
|
||||
times.
|
||||
5. Inject one task file referencing a made-up verifier id; assert load
|
||||
fails, error names the id and file path.
|
||||
|
||||
**Command:** `cargo test -p formats taskboard_format`
|
||||
|
||||
**False pass:**
|
||||
- Step 3 without deliberately scrambling directory-read order — most
|
||||
filesystems return sorted-ish order by accident, hiding a bug that trusts
|
||||
`readdir`.
|
||||
- Skipping the unresolved-id failure test — a format that silently drops
|
||||
unparseable tasks looks like it "works" until one goes missing at spawn.
|
||||
|
||||
## Traps
|
||||
|
||||
- Deriving step order from task-id string sort. `"T10.1" < "T2.1"` as
|
||||
strings — `INDEX.md`'s declared order is the only authority, exactly the
|
||||
bug the board's own "Ordering" section warns about at board scope.
|
||||
- Making `TaskBoardFormat` also write `Status` back into task files. Turns a
|
||||
read-only parser into a mutator with its own concurrency/conflict
|
||||
problems, out of scope.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[../INDEX.md](../INDEX.md) (customization contract ~line 19-58, ordering
|
||||
~line 88-103) ·
|
||||
[T3.1-workflowdef-ir-canonicalization.md](T3.1-workflowdef-ir-canonicalization.md) ·
|
||||
[T3.9-capability-registry.md](T3.9-capability-registry.md)
|
||||
@@ -0,0 +1,101 @@
|
||||
# T10.6 — `plan-draft` workflow + `Suspended`-approval flow
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P10 — Orchestration |
|
||||
| Size | M — 1 to 3 days |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | — |
|
||||
|
||||
## Goal
|
||||
|
||||
Front door (b): a free-text problem statement produces a draft
|
||||
`WorkflowDef`, parked in `Suspended` for human review, resumed through the
|
||||
identical loader front door (a) uses.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
- `Suspended` is an exact fit, verified against source: "a run awaiting
|
||||
human approval or a webhook must not hold an executor slot" (T1.3), lease
|
||||
released outright with no heartbeat while parked (T7.3).
|
||||
- `plan-draft` is itself an ordinary poimen workflow: a `Model` step
|
||||
producing a `WorkflowDef` draft, no new kernel machinery.
|
||||
- The draft is validated against T3.9's capability registry **before** a
|
||||
human ever sees it — an unresolved verifier/judge/model id in the draft
|
||||
fails here, at draft time, not after approval.
|
||||
- Approval feeds the (possibly human-edited) draft through the **same**
|
||||
`WorkflowFormat`/builder + T3.1 canonicalization as door (a) — this is
|
||||
what keeps both doors on one IR without new proof machinery.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Define the built-in `plan-draft` workflow: one `Model` step (problem
|
||||
statement → draft `WorkflowDef`), one validation step (registry check via
|
||||
T3.9).
|
||||
2. On draft ready, transition the `plan-draft` run to `Suspended`, releasing
|
||||
its lease (T7.3).
|
||||
3. `/v1/plans/{id}/approve` (T10.4 route): load the (possibly edited) draft,
|
||||
run it through the exact loader door (a)'s `WorkflowFormat`/builder uses,
|
||||
canonicalize (T3.1), resume — matching T1.3's resume-from-`Suspended`
|
||||
semantics.
|
||||
4. `/v1/plans/{id}/reject`: cancel from `Suspended` (legal transition per
|
||||
T1.3's cancel-from-any-non-terminal rule).
|
||||
5. Test: submit a problem statement, assert `Suspended`, edit the draft,
|
||||
approve, assert the resulting `WorkflowVersion` hash matches
|
||||
hand-authoring the same edited YAML directly.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- A submitted problem statement produces a run parked in `Suspended` with no
|
||||
held executor slot — lease released, per T7.3's own test shape.
|
||||
- Approving a draft (edited or not) produces a `WorkflowVersion`
|
||||
byte-identical to loading the same content by hand through door (a).
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** stub model for the `plan-draft` step, embedded `redb`, T7.3's
|
||||
lease-release assertion pattern reused.
|
||||
|
||||
**Integration test** — `tests/it_plan_draft_approval.rs`:
|
||||
1. `POST /v1/problems` with a free-text statement; assert the run reaches
|
||||
`Suspended`.
|
||||
2. Assert the lease is released and no heartbeat is emitted while
|
||||
`Suspended` — reuse T7.3's assertion (its step 8).
|
||||
3. Fetch the draft; edit one step; `POST /v1/plans/{id}/approve` with the
|
||||
edited draft.
|
||||
4. Assert the resulting `WorkflowVersion` hash equals hashing the same
|
||||
edited content loaded directly via `WorkflowFormat`/builder — front-door
|
||||
equivalence, extends T3.10.
|
||||
5. `POST /v1/plans/{id2}/reject` on a second draft; assert it reaches
|
||||
`Cancelled`, not stuck in `Suspended`.
|
||||
6. Draft referencing an unregistered verifier id; assert `plan-draft`'s
|
||||
validation step fails **before** ever reaching `Suspended` — the human
|
||||
never sees an unloadable plan.
|
||||
|
||||
**Command:** `cargo test -p orchestration plan_draft_approval`
|
||||
|
||||
**False pass:**
|
||||
- Skipping step 4's hash comparison and only checking "run resumed" — a
|
||||
subtly different canonicalization from door (a) would still resume, just
|
||||
diverge from what T3.1's guarantee promises.
|
||||
- Not testing reject — an approval-only test suite misses that `Suspended`
|
||||
must also support "no."
|
||||
|
||||
## Traps
|
||||
|
||||
- Having `/approve` construct a `WorkflowDef` directly in the HTTP handler
|
||||
instead of routing through the same loader as door (a) — this is exactly
|
||||
the two-IR-drift bug T3.10 exists to prevent, reintroduced one layer up.
|
||||
- Skipping the registry validation step "since a human will catch it." The
|
||||
whole point of validating before `Suspended` is that load errors are cheap
|
||||
pre-human, expensive post-human.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[../INDEX.md](../INDEX.md) (customization contract ~line 19-58) ·
|
||||
[T1.3-run-executor.md](T1.3-run-executor.md) ·
|
||||
[T7.3-leases-and-fencing.md](T7.3-leases-and-fencing.md) ·
|
||||
[T3.10-front-door-equivalence.md](T3.10-front-door-equivalence.md)
|
||||
@@ -0,0 +1,108 @@
|
||||
# T10.7 — Kafka topics and inbox relay
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P10 — Orchestration |
|
||||
| Size | L — over 3 days |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | T10.8 |
|
||||
|
||||
## Goal
|
||||
|
||||
`poimen.<tenant>.events` (outbound, existing outbox unchanged) and
|
||||
`poimen.<tenant>.dispatch` (inbound, new) — an inbox relay symmetric to
|
||||
T7.4's outbox, turning dispatch messages into spawn calls through the same
|
||||
admission path, never a raw kernel write.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
- Outbound is unchanged: T7.4's relay ships goal-tagged `WorkEvent`s exactly
|
||||
as it ships every other event. No new producer logic on this side.
|
||||
- Inbound is genuinely new: the orchestrator (T10.8) publishes to
|
||||
`.dispatch` after a `Reconciler` decision; this task builds only the relay
|
||||
that drains it, not the orchestrator itself.
|
||||
- The inbox relay's failure-domain rule mirrors T7.4's exactly: own
|
||||
process/task, own retry, own supervision, not co-located with the
|
||||
executor's lifecycle.
|
||||
- Every dispatch message becomes a call to the **same** `POST /v1/runs`
|
||||
(T10.4) the HTTP API exposes — the relay is an HTTP client to the local
|
||||
API surface, not a kernel-internal spawn path. This is what makes the
|
||||
"no bypass" guarantee mechanical, not a code-review convention.
|
||||
- Redelivery on `.dispatch` is handled entirely by T10.3's `dispatch_key`
|
||||
admission dedup — the relay passes `dispatch_key` through unchanged, no
|
||||
separate idempotency layer.
|
||||
- Partition key on both topics: `(TenantId, RunId)` per T7.5, consistent
|
||||
with the outbox.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Declare topics `poimen.<tenant>.events`, `poimen.<tenant>.dispatch` —
|
||||
partition key `(TenantId, RunId)` on both (T7.5).
|
||||
2. No change to T7.4's relay beyond including goal-tagged records (already
|
||||
covered since `GoalId` is a field on `WorkEvent`, T10.1).
|
||||
3. Build the inbox relay: consume `.dispatch`, for each message call
|
||||
`POST /v1/runs` with `{workflow_ref, input, goal_id, dispatch_key}`
|
||||
verbatim.
|
||||
4. Own retry/backoff on relay-to-API-call failure; never crash the relay on
|
||||
a single bad message — dead-letter it, keep draining.
|
||||
5. Own failure domain: separate process/task, no shared lifecycle with the
|
||||
executor or the HTTP API service.
|
||||
6. Emit inbox lag as a metric, alongside T7.4's existing outbox-lag metric.
|
||||
7. Test with the broker down for a stretch, and with forced redelivery of
|
||||
the same dispatch message.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- A message published to `.dispatch` results in exactly one run, even under
|
||||
forced redelivery.
|
||||
- Broker down for the inbox side does not affect runs already in flight;
|
||||
queued dispatch messages are processed once the broker returns.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** same broker-down/broker-up harness as T7.4, plus a second
|
||||
consumer/producer pair for the `.dispatch` direction.
|
||||
|
||||
**Integration test** — `tests/it_inbox_relay.rs`:
|
||||
1. Publish one message to `.dispatch`; assert exactly one run created via
|
||||
the same `POST /v1/runs` path — instrument the API, assert the call, not
|
||||
a direct kernel spawn.
|
||||
2. Force redelivery of the same message; assert still exactly one run
|
||||
(T10.3's admission dedup catches it) — no relay-side special-casing
|
||||
needed.
|
||||
3. Take the broker down; publish a message; bring the broker up; assert the
|
||||
run is eventually created, no message lost.
|
||||
4. Publish a malformed message; assert it's dead-lettered, relay keeps
|
||||
draining subsequent valid messages.
|
||||
5. Assert the relay never calls kernel spawn directly — instrument the
|
||||
kernel's internal admission entrypoint, assert all calls originate from
|
||||
the HTTP layer, none from relay threads.
|
||||
6. Assert inbox lag is emitted as a metric.
|
||||
|
||||
**Command:** `cargo test -p distribution inbox_relay -- --test-threads=1`
|
||||
|
||||
**False pass:**
|
||||
- Step 2 without step 1's exact-call-path instrumentation: a relay that
|
||||
spawns directly and also happens to dedupe correctly would still pass a
|
||||
coarser "one run" check while violating the no-bypass rule.
|
||||
- Skipping the malformed-message case: one bad message halting the whole
|
||||
relay is a silent single point of failure discovered only in production.
|
||||
|
||||
## Traps
|
||||
|
||||
- Relay calling an in-process spawn function "for performance" instead of
|
||||
going through `POST /v1/runs` — same trap T7.4 names for the outbound
|
||||
side, mirrored here.
|
||||
- Building custom dedup logic in the relay instead of trusting T10.3's
|
||||
admission-layer dedup. Two idempotency layers can disagree under partial
|
||||
failure.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[rust-agentic-sys.md](../../../rust-agentic-sys.md) §8.3, §9.2, §9.3, §13.3 ·
|
||||
[T7.4-outbox-relay.md](T7.4-outbox-relay.md) ·
|
||||
[T7.5-partition-keys-on-adapters.md](T7.5-partition-keys-on-adapters.md) ·
|
||||
[T10.3-idempotent-run-admission.md](T10.3-idempotent-run-admission.md)
|
||||
@@ -0,0 +1,108 @@
|
||||
# T10.8 — Orchestrator reference service
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P10 — Orchestration |
|
||||
| Size | M — 1 to 3 days |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | — |
|
||||
|
||||
## Goal
|
||||
|
||||
Reference process that consumes `.events`, invokes the registered
|
||||
`Reconciler` for goal-tagged terminal transitions, and publishes decisions
|
||||
to `.dispatch` — never spawns a run itself.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
- The orchestrator embeds poimen only via the HTTP API (T10.4) /
|
||||
`poimen-sdk`, exactly like any other external embedder — no special
|
||||
kernel access.
|
||||
- Its entire job: consume `.events`, filter for goal-tagged
|
||||
`Verified`/`Graded`/`Ungraded`, call `Reconciler` (T10.2), publish the
|
||||
resulting `SpawnNext` decision to `.dispatch`. `Stop`/`Retry` decisions
|
||||
are recorded but do not publish a dispatch message.
|
||||
- The default `Reconciler` implementation may itself be another poimen run
|
||||
(a `Model` step + a `Tool` step) — the orchestrator's job is identical
|
||||
regardless of whether the bound `Reconciler` is LLM-backed or a
|
||||
hand-written Rust impl.
|
||||
- Never publishes a run directly, never calls `POST /v1/runs` itself — that
|
||||
would reintroduce the bypass T10.7's inbox relay exists to prevent. Its
|
||||
only write is to `.dispatch`.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Consume `.events`, filter to records with a `GoalId` and a terminal run
|
||||
state (`Verified`/`Graded`/`Ungraded`).
|
||||
2. For each, build `ReconcileCtx` (T10.2) and call the `Reconciler` bound to
|
||||
that goal.
|
||||
3. On `SpawnNext`: publish `{workflow_ref, input, goal_id, dispatch_key}` to
|
||||
`.dispatch` — `dispatch_key` derived from the decision id so redelivery
|
||||
is naturally deduped downstream (T10.3).
|
||||
4. On `Retry`/`Stop`: record the decision (`WorkEvent` provenance per
|
||||
T10.2), publish nothing.
|
||||
5. Own failure domain: separate process, own consumer group, own
|
||||
retry/backoff on `Reconciler`-call failure — a `Reconciler` error does
|
||||
not crash the orchestrator, it skips and retries on next poll or leaves
|
||||
the goal stalled with a visible metric.
|
||||
6. Emit reconcile-latency and goal-stall metrics.
|
||||
7. Test the full loop against a stub `Reconciler`: events in, dispatch out,
|
||||
verified end to end including a two-hop goal chain (run A → reconcile →
|
||||
run B → reconcile → stop).
|
||||
|
||||
## Acceptance
|
||||
|
||||
- A two-run goal chain (A completes → reconcile → B spawns → B completes →
|
||||
reconcile → `Stop`) completes end to end through real Kafka topics, with
|
||||
the orchestrator never calling spawn directly.
|
||||
- `Reconciler` failure on one goal does not block reconciliation of other
|
||||
goals.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** real Kafka (or the board's chosen fault-injection substrate),
|
||||
stub `Reconciler` with scripted decisions, T10.7's inbox relay running
|
||||
alongside.
|
||||
|
||||
**Integration test** — `tests/it_orchestrator_e2e.rs`:
|
||||
1. Spawn run A with `goal_id=G`, no `dispatch_key`. Drive to
|
||||
`Verified{pass}`.
|
||||
2. Assert orchestrator calls `Reconciler` exactly once for A; stub returns
|
||||
`SpawnNext` for run B.
|
||||
3. Assert a message lands on `.dispatch`, and via T10.7's relay, run B is
|
||||
spawned with `goal_id=G`.
|
||||
4. Drive run B to `Graded`. Assert `Reconciler` called again; stub returns
|
||||
`Stop`.
|
||||
5. Assert no further `.dispatch` message is published; goal G's chain shows
|
||||
exactly `[A, B]` via T10.1's `GoalView`.
|
||||
6. Instrument `POST /v1/runs`; assert zero calls originate from the
|
||||
orchestrator process — only from the inbox relay.
|
||||
7. Second goal G2 with a `Reconciler` that always errors; assert G2 stalls
|
||||
(visible via goal-stall metric) while G's chain, run concurrently,
|
||||
proceeds unaffected.
|
||||
|
||||
**Command:** `cargo test -p orchestration orchestrator_e2e -- --test-threads=1`
|
||||
|
||||
**False pass:**
|
||||
- Step 6 omitted: an orchestrator that takes a shortcut and spawns directly
|
||||
under test-only conditions would still pass steps 1-5.
|
||||
- Testing only a single-hop chain (A → `Stop`) — the two-hop case is what
|
||||
proves goal-chain provenance actually threads through `GoalView`
|
||||
correctly across a full reconcile-dispatch-relay-spawn cycle.
|
||||
|
||||
## Traps
|
||||
|
||||
- Orchestrator holding an admission-layer shortcut "to save a network hop."
|
||||
Collapses the exact boundary T10.7 was built to enforce.
|
||||
- Coupling `Reconciler` failure handling to a single global retry policy.
|
||||
One broken goal's `Reconciler` should not starve reconciliation for every
|
||||
other goal in the tenant.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[T10.1-goalid-and-goal-scoped-query.md](T10.1-goalid-and-goal-scoped-query.md) ·
|
||||
[T10.2-reconciler-port.md](T10.2-reconciler-port.md) ·
|
||||
[T10.7-kafka-topics-and-inbox-relay.md](T10.7-kafka-topics-and-inbox-relay.md)
|
||||
@@ -0,0 +1,88 @@
|
||||
# T10.9 — CLI: runs list/episode/transcript
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Phase | P10 — Orchestration |
|
||||
| Size | S — under 1 day |
|
||||
| Status | Not started |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | — |
|
||||
|
||||
## Goal
|
||||
|
||||
Three thin CLI wrappers over the HTTP API — `poimen runs list --goal <id>`,
|
||||
`poimen runs episode <run_id>`, `poimen runs transcript <run_id> <step>
|
||||
<attempt>` — the whole troubleshooting UX, no dashboard.
|
||||
|
||||
## Facts (inlined — no spec read needed)
|
||||
|
||||
- Thin HTTP clients, not a second implementation of the query logic already
|
||||
in T10.1/T1.7/T1.6 — the CLI calls the same routes T10.4 exposes.
|
||||
- Rides the eventual shared `poimen` binary (T9.3); workable as a standalone
|
||||
binary if T9.3 hasn't landed yet — soft dependency, not a hard block.
|
||||
- `episode` output must include failed attempts by default — T1.7's own
|
||||
rule. A CLI that hides them defeats the purpose of the command.
|
||||
- `transcript` dereferences the `BlobRef` server-side, via the HTTP route
|
||||
(T10.4) — the CLI never talks to `BlobStore` directly.
|
||||
|
||||
## Steps
|
||||
|
||||
1. `poimen runs list [--goal <id>] [--status <state>]` → `GET /v1/runs`,
|
||||
table output (`run_id`, `status`, `goal_id`, age).
|
||||
2. `poimen runs episode <run_id>` → `GET /v1/runs/{id}/episode`,
|
||||
human-readable rendering of `EpisodeView` including failed attempts,
|
||||
clearly marked.
|
||||
3. `poimen runs transcript <run_id> <step> <attempt>` → `GET .../transcript`,
|
||||
prints prompt/output text, dereferenced server-side.
|
||||
4. Exit codes: non-zero on API error, distinct code for "run not found" vs
|
||||
"transport error" — operator scripting depends on this distinction.
|
||||
5. Test against a live embedded API instance (T10.4's test harness) with a
|
||||
run that has a retry and a rewind.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- `episode` on a run with two failed attempts and one success shows all
|
||||
three, visually distinguishable.
|
||||
- `transcript` returns the same text content as reading the `BlobRef`
|
||||
directly through `BlobStore` in a test assertion.
|
||||
|
||||
## Verify
|
||||
|
||||
**Harness:** T10.4's `axum` test instance, one run with a retry (two
|
||||
attempts, one failed) and a rewind.
|
||||
|
||||
**Integration test** — `tests/it_cli_runs_ux.rs`:
|
||||
1. `runs list --goal G`; assert output contains exactly the runs tagged G,
|
||||
none untagged.
|
||||
2. `runs episode <id>`; assert both the failed and successful attempt
|
||||
appear — count them explicitly, not just non-empty output.
|
||||
3. `runs transcript <id> <step> 1` (the failed attempt) and `... 2` (the
|
||||
retry); assert distinct content for each, matching their respective
|
||||
`BlobRef`s byte-for-byte.
|
||||
4. Run-not-found case; assert a distinct exit code from a transport-down
|
||||
case.
|
||||
|
||||
**Command:** `cargo test -p cli runs_ux`
|
||||
|
||||
**False pass:**
|
||||
- Step 2 checking only that output is non-empty — a CLI silently filtering
|
||||
to the winning attempt (T1.7's own named trap) passes a non-empty check
|
||||
while hiding exactly the debugging signal requested.
|
||||
|
||||
## Traps
|
||||
|
||||
- Filtering episode output to "just the successful attempt for clarity" —
|
||||
T1.7's trap restated at the CLI layer; the failed attempts are the reason
|
||||
this command exists.
|
||||
- Fetching transcripts client-side by resolving `BlobRef` directly instead
|
||||
of through the HTTP route. Bypasses the tenant auth checks the API
|
||||
enforces.
|
||||
|
||||
---
|
||||
|
||||
Background (not required to do this task):
|
||||
[T10.4-http-api-surface.md](T10.4-http-api-surface.md) ·
|
||||
[T1.7-episode-query-surface.md](T1.7-episode-query-surface.md) ·
|
||||
[T1.6-prompt-and-output-blob-capture.md](T1.6-prompt-and-output-blob-capture.md) ·
|
||||
[T8.2-metrics.md](T8.2-metrics.md)
|
||||
Reference in New Issue
Block a user