# Multi-Agent Dev Orchestrator (Temporal + Go) — Implementation Plan

See the full design doc at `/Users/rockliang/.claude/plans/considered-u-are-a-curried-reef.md`.

## Quick Summary

Build a Temporal-based orchestrator that drives multi-agent software dev work on arbitrary target repos. Three roles (Planner/reasoning, Judge/reasoning, Implementer/cheaper model) collaborate on hierarchical tasks (`T0` milestone split into `T0.1`-`T0.9` subtasks). Orchestrator owns config (system prompt, skills, activity timeouts/retries), live-updatable via signals. All code runs on shared FS where target repo sits; git concurrency handled via worktrees + advisory lock. System testable with mocked activities + real end-to-end against `temporal.riotpiao.com`.

## Implementation Track

**Milestone T0**: Nine subtasks, each with its own verification gate = completion criterion.

| Task | Scope | Status |
|---|---|---|
| [T0.1](#t01-repo-scaffold) | Repo scaffold: `go.mod`, `statemachine/`, `action/`, `cmd/`, `prompts/`, `internal/`, `tests/` | [ ] |
| [T0.2](#t02-shared-types) | Shared types: `ModelSpec`, `PromptSpec`, `OrchestratorConfig`, `ActivityTuning`, `PiRetryPolicy` | [ ] |
| [T0.3](#t03-git-and-locking) | Git & locking: `CloneRepoActivity`, worktrees, squash-merge, `orchestrator.lock` | [ ] |
| [T0.4](#t04-pi-and-error-classification) | `PrepareSkillsActivity`, `classifyPiErr` (4xx/5xx/504), stream timeout learning | [ ] |
| [T0.5](#t05-llm-agents-and-prompts) | Planner/Judge/Implementer activities, LLM client, prompt templates + live customization | [ ] |
| [T0.6](#t06-taskunit-workflow) | TaskUnit workflow: retry loops, timeout escalation, lessons injection | [ ] |
| [T0.7](#t07-orchestrator-workflow) | Orchestrator workflow: config state, signals, fan-out/fan-in, `continue-as-new`, 504 learning | [ ] |
| [T0.8](#t08-worker-and-starter) | Worker & starter CLIs, env/config loading, Temporal registration | [ ] |
| [T0.9](#t09-end-to-end) | Full e2e against real cluster + scratch repo: all 7 verification items | [ ] |

---

### T0.1: Repo Scaffold

Create directory structure, `go.mod`, empty stubs.

**Verification:** `go build ./...` succeeds; layout matches plan.

<details>
<summary>Details</summary>

```
/go.mod
/cmd/worker/main.go
/cmd/starter/main.go
/statemachine/types.go types.go signals.go orchestrator.go taskunit.go
/action/planner.go implementer.go judge.go git.go skills.go integration_test.go lessons.go llm/client.go
/prompts/registry.go planner/default.tmpl judge/default.tmpl implementer/default.tmpl
/internal/config/config.go lock/flock.go
/tests/taskunit_workflow_test.go orchestrator_workflow_test.go
```

</details>

---

### T0.2: Shared Types

Implement `statemachine/types.go` with all config/input/output structs. Document defaults.

**Verification:** Unit test asserts all defaults (5m / 2s / 30s / 2.0 / 30s stream / 2m stream-max).

<details>
<summary>Details</summary>

- `ModelSpec`: ModelID, Thinking, Effort
- `PromptSpec`: TemplateRef, RawTemplate, Variables, Model, LessonsRef
- `OrchestratorInput`, `OrchestratorOutput`
- `TaskUnitInput`, `TaskUnitOutput`
- `ActivityTuning`: ImplementerBaseTimeout, ImplementerMaxRetries, JudgeTimeout, PiRetry
- `PiRetryPolicy`: ScheduleToCloseTimeout (5m), InitialInterval (2s), MaximumInterval (30s), BackoffCoefficient (2.0), StreamTimeout (30s), StreamTimeoutMax (2m)
- `OrchestratorConfig`: SystemPrompt, Skills, RolePrompts, Tuning

</details>

---

### T0.3: Git & Locking

Implement `action/git.go` + `internal/lock/flock.go`.

**Verification:** Test against local scratch repo: clone-if-empty vs fetch-if-exists, worktree lifecycle (add/commit/remove), squash-merge produces exactly one commit on main.

<details>
<summary>Details</summary>

**Activities:**
- `CloneRepoActivity(ctx, {RemoteURL, TargetRepoPath}) error` — idempotent `git clone` or `git fetch`
- `GitWorktreeAddActivity(ctx, {RepoPath, TaskID}) (string, error)` — returns worktree path, guarded by lock
- `GitCommitActivity(ctx, {WorktreePath, Message}) error` — commits in worktree (no lock needed)
- `GitPushActivity(ctx, {RepoPath}) error` — guarded by lock
- `GitSquashMergeActivity(ctx, {RepoPath, Branches, Message}) error` — guarded by lock

**Lock helper (`internal/lock/`):**
- `Lock(path string) error`, `Unlock(path string) error` using `golang.org/x/sys/unix.Flock` or `fcntl` equivalent

**Squash-merge sequence:**
```
fetch origin main
checkout main && pull --ff-only origin main
for b in branches:
  merge --squash $b
commit -m "T0: squash merge subtasks..."
push origin main
for b in branches:
  worktree remove worktrees/$id --force
  branch -D $b
```

</details>

---

### T0.4: Pi & Error Classification

Implement `action/skills.go` with `PrepareSkillsActivity` and `classifyPiErr`.

**Verification:** Unit test all three error buckets (4xx/5xx/504) against a mocked `pi` HTTP client.

<details>
<summary>Details</summary>

**`PrepareSkillsActivity`:**
- Input: `{Skills []SkillRef, StreamTimeout time.Duration}`
- For each skill, `pi clone-or-fetch <skill-url>` (idempotent)
- Each skill guarded by its own lock

**Error classification:**
```go
func classifyPiErr(err error) error {
    // 4xx -> NonRetryableApplicationError "PiClientError"
    // 504 -> ApplicationError "PiStreamTimeout"
    // others -> retryable
}
```

</details>

---

### T0.5: LLM Agents & Prompts

Implement LLM activities + prompt templates.

**Verification:** Unit test renders a `PromptSpec` (system prompt + template override + raw template) and calls mock Anthropic client.

<details>
<summary>Details</summary>

**Activities:**
- `PlanningActivity(ctx, {OrchestratorConfig, BoardState}) (TaskDispatch, error)` — reads board/INDEX.md, calls Planner
- `ImplementerActivity(ctx, {PromptSpec, WortkreeePath, Lessons}) (ImplementOutput, error)` — tool-call agent loop
- `JudgeActivity(ctx, {PromptSpec, Diff, IntegrationTestResult}) (Verdict, Critique, error)` — reviews correctness
- `RunIntegrationTestActivity(ctx, {WortkreeePath, TestCmd}) (pass/fail, logs, error)` — shells out

**Prompt templates:**
- `planner/default.tmpl`: expects `{{.SystemPrompt}}`, `{{.TaskBoard}}`, etc.
- `judge/default.tmpl`: expects `{{.SystemPrompt}}`, `{{.Diff}}`, `{{.TestResult}}`
- `implementer/default.tmpl`: expects `{{.SystemPrompt}}`, `{{.Task}}`, `{{.Lessons}}`

**`prompts/registry.go`:**
- `go:embed prompts/*.tmpl`
- `Render(templateRef string, variables map[string]any) (string, error)`

**`action/llm/client.go`:**
- Thin Anthropic client wrapper
- Read `ANTHROPIC_API_KEY` from env
- Call `messages.Create` with model/thinking/effort from `ModelSpec`

</details>

---

### T0.6: TaskUnit Workflow

Implement `statemachine/taskunit.go` with retry loops & timeout escalation.

**Verification (Testsuite):**
- Pass-first-try
- Fail-then-pass-after-lesson-injection
- Retries-exhausted
- Timeout-escalation (both judges)

<details>
<summary>Details</summary>

**Flow:**
1. `GitWorktreeAddActivity` → get isolated working tree
2. Retry loop:
   - Track `timeoutAttempt`, `judgeAttempt` separately
   - `ImplementerActivity` with timeout = `BaseTimeout * timeoutAttempt`
   - If timeout, increment `timeoutAttempt` and retry (duration grows)
   - If success, call `RunIntegrationTestActivity`
   - Call `JudgeActivity`
   - If judge pass, commit in worktree and return
   - If judge fail, append to lessons, increment `judgeAttempt`, retry (lessons injected next time)
   - If retries exhausted, return fail verdict to orchestrator

**Key detail:** HeartbeatTimeout = (BaseTimeout * timeoutAttempt) / 4, scales with escalation.

</details>

---

### T0.7: Orchestrator Workflow

Implement `statemachine/orchestrator.go` with config state, signals, fan-out/fan-in, `continue-as-new`, 504 learning.

**Verification (Testsuite):**
- Fan-out/fan-in correctness
- Squash-merge triggers on submilestone complete
- `continue-as-new` at cycle cap, carries `OrchestratorConfig` forward
- `update-*` signals mutate config without touching in-flight TaskUnit
- `PiStreamTimeout` doubles `StreamTimeout` and persists it

<details>
<summary>Details</summary>

**Per-cycle logic:**
1. If `config.Skills` changed, call `PrepareSkillsActivity` once (wraps it for 504 learning)
2. Call `PlanningActivity` → get dispatch decision
3. Fan out: `workflow.ExecuteChildWorkflow(TaskUnitWorkflow, ...)` for each dispatched T0.x
4. Await all via `workflow.Selector`
5. Call `PlanningActivity` again to update board + commit + push
6. If submilestone complete, call `GitSquashMergeActivity`
7. Increment cycle count
8. If cycle count >= cap, `workflow.NewContinueAsNewError(ctx, ..., nextInput)`

**Signal handlers:**
- `pause`, `resume`: gate the cycle loop
- `abort-task(taskID)`: forward via `SignalExternalWorkflow` to TaskUnit
- `inject-lesson`: append to lessons store
- `update-system-prompt`, `update-skills`, `update-role-prompt`, `update-tuning`: mutate `config.*`

**504 learning wrapper (pseudo-code):**
```go
for {
    r := config.Tuning.PiRetry
    err := ExecuteActivity(..., PrepareSkillsActivity, Input{...StreamTimeout: r.StreamTimeout})
    if isPiStreamTimeout(err) && r.StreamTimeout < r.StreamTimeoutMax {
        config.Tuning.PiRetry.StreamTimeout *= 2
        continue
    }
    break
}
```

</details>

---

### T0.8: Worker & Starter CLIs

Implement `cmd/worker/main.go` and `cmd/starter/main.go`.

**Verification:**
- `go run ./cmd/worker` connects to `temporal.riotpiao.com:7233` without error
- `go run ./cmd/starter --dry-run` starts a workflow that appears in Temporal Web UI

<details>
<summary>Details</summary>

**`cmd/worker/main.go`:**
```go
config := loadConfig() // reads env: TEMPORAL_NAMESPACE, TEMPORAL_TLS_CERT, TEMPORAL_TLS_KEY, ANTHROPIC_API_KEY
c, err := client.Dial(client.Options{HostPort: "temporal.riotpiao.com:7233", ...TLS...})
w, err := worker.New(c, "default", worker.Options{})
// register both workflows
w.RegisterWorkflow(statemachine.OrchestratorWorkflow)
w.RegisterWorkflow(statemachine.TaskUnitWorkflow)
// register all activities
w.RegisterActivity(action.CloneRepoActivity)
w.RegisterActivity(action.GitWorktreeAddActivity)
// ... etc
w.Run()
```

**`cmd/starter/main.go`:**
```go
flag.String("repo", "", "target repo path")
flag.String("remote", "", "remote URL")
flag.String("milestone", "T0", "milestone ID")
flag.Bool("dry-run", false, "disable git push/merge")
flag.String("planner-model", "claude-opus-5", "planner model ID")
// ... judge, implementer models
// build OrchestratorInput, call client.ExecuteWorkflow
```

**`internal/config/config.go`:**
- Load Temporal settings from env
- Load ANTHROPIC_API_KEY from env
- Return filled config struct

</details>

---

### T0.9: End-to-End Test

Run against real `temporal.riotpiao.com` + disposable scratch repo.

**Verification (all 7 items in the plan):**
1. Clone bootstrap: fresh clone when repo path empty
2. Full cycle: dispatch subtasks, judge pass/fail, commit, squash-merge
3. Live signal updates: change prompt/skills mid-run, next dispatch sees them
4. 5xx retry-then-succeed + always-503 exhausts at 5m mark
5. 504 stream-timeout learning: doubles and is actually used, capped at max
6. `continue-as-new` history bounded
7. Squash-merge result: main has one squashed commit per submilestone

<details>
<summary>Details</summary>

**Fixture repo structure:**
```
tasks/
  INDEX.md (guidelines)
  board.json (task list, T0.1-T0.3 with trivial definitions)
```

**Example subtask:** "Create file `output.txt` with content 'hello world'"

**Run sequence:**
1. `go run ./cmd/starter --repo /tmp/fixture --remote git@forgejo.homelab:scratch/workflow-test.git --dry-run`
2. Monitor Temporal Web UI for workflow progress
3. Midway, send signals: `temporal workflow signal --workflow-id orch-... --name update-role-prompt ...`
4. Confirm next dispatch uses new prompt (assert marker in output file)
5. Confirm board updated, lessons file exists (if any failure happened)
6. Remove `--dry-run`, repeat against real remote
7. Assert final state: real commits on remote, squash-merge on main

</details>

---

## Next Steps

1. Approve this scaffold (PLAN.md + tasks/INDEX.md + board)
2. Start T0.1 → checkout branch `task/T0.1` → scaffold repo structure
3. Each task: implement, test locally, verify against criterion
4. Mark on board: [x] when verification passes
5. T0.9: final e2e run
6. Squash all T0.* branches into main, push