# M5.3 — Training corpus export

| Field | Value |
|---|---|
| Phase | M5 — Post-training |
| Size | M — 1–3 days |
| Status | ⬜ Not started |
| Flags | — |
| Spec | inlined below |
| Blocks | M5.1 |

## Goal

Turn the log plus labels into trajectories verl can train on — the boundary
between the Rust side and the Python side.

## Facts (inlined — no spec read needed)

**The JSONL log is the boundary.** Rust produces it; Python consumes it. Nothing
else crosses, which is what keeps the two halves independent.

A training example is a **trajectory**, not a turn: the paper's advantage mixes a
trajectory-level term with a turn-level one (`Â = α·Â_traj + (1−α)·Â_turn`,
α=0.9), so turns must stay grouped by run.

Per turn, verl needs: the exact prompt sent, the exact response, and the rewards.

```jsonl
{"trajectory_id":"01HXYZ","turns":[
  {"t":1,"prompt":"<full assembled prompt>","response":"<think>...</next>",
   "r_update":-1,"parsed":true},
  ...],
 "r_exit":-0.5,"r_format":1,"r_outcome":null}
```

Reward assembly, from the paper:

- `r_update_t` = +1 if the recorded `U_t` matches M5.1's label, −1 otherwise.
- `r_exit` — one value per trajectory: `0` if `t_exit == t_last_evidence`,
  `−0.75` if earlier, `−0.5` if later. `t_last_evidence` is the largest `t` whose
  label is true. **Note L1 runs never exit** (exit gate off), so every L1
  trajectory is a "late" exit at −0.5 unless the exit signal is taken from the
  recorded `E_t` rather than the loop's behaviour — take it from the record.
- `r_format` = 1 only if **every** turn in the trajectory parsed, 0 otherwise.
  Strict, because a malformed turn may be caused by the previous one.
- `r_outcome` is null. We have no answer-correctness signal; the paper's
  `is_equiv(A, Â)` has no analogue in extraction. Say so explicitly rather than
  fabricating one.

The prompt must be the **exact bytes sent**, not re-assembled. Re-assembly drifts
from what the model actually saw the moment M1.3's template changes.

## Steps

1. `mem export --project P --format verl --out corpus/`.
2. Join log turns to labels by `chunk_sha`.
3. Reconstruct each turn's prompt from the recorded request if `MEM_LLM_RECORD`
   captured it; otherwise fail loudly rather than re-assembling.
4. Compute rewards as above; carry `r_outcome: null` through.
5. Group by run into trajectories, ordered by `t`.
6. Emit a summary: trajectories, turns, positive/negative `r_update` split,
   `r_format` pass rate, `t_last_evidence` distribution.
7. Refuse to export if M5.2's κ is below the threshold or absent.

## Acceptance

- Turns are grouped into trajectories, ordered.
- Prompts are byte-exact recordings, never re-assembled.
- `r_format` is 0 for a trajectory with any unparsed turn.
- Export is refused without calibration.

## Verify

**Harness:** log fixture with known labels and a deliberately unparsed turn.

**Integration test** — `tests/it_export.rs`:
1. `a1_trajectory_grouping` — turns grouped by run, `t` ascending, none lost.
2. `a2_r_update_signs` — matching label → +1, mismatching → −1, checked per turn.
3. `a3_r_format_strict` — a trajectory with one unparsed turn scores 0 overall,
   not per turn.
4. `a4_r_exit_from_record` — assert `r_exit` derives from the recorded `E_t`, not
   from whether the loop stopped. A fixture where the gate said `end` at t=5 but
   the loop continued must score as an exit at 5.
5. `a5_prompt_is_recorded_bytes` — assert the exported prompt equals the recorded
   request body; corrupt the recording and assert export fails rather than
   silently re-assembling.
6. `a6_r_outcome_null` — assert the field is present and null, not omitted and not
   zero.
7. `a7_refuses_without_calibration` — no κ file → non-zero exit naming M5.2.
8. `a8_summary_counts` — reported splits match a hand count on the fixture.

**Command:** `cargo test -p mem-cli export`

**False pass:**
- Re-assembling prompts at export time. Every assertion except 5 passes, and the
  policy is trained on prompts the model never saw — which shows up as a training
  run that will not converge, with no obvious cause.
- Applying `r_format` per turn. It looks more granular and it is wrong: the paper
  is explicit that the strictness exists because a bad turn may be caused by the
  previous one.
- Emitting `r_outcome: 0` instead of null. Zero is a real reward value and the
  trainer will use it as signal.

## Traps

- Deriving `r_exit` from loop behaviour at L1. The gate is switched off there, so
  every trajectory scores as a late exit and the exit signal becomes constant
  noise the policy cannot learn from.
- Joining labels by `t`. Same failure as M5.1's trap — a re-chunk misaligns
  everything silently.

---

Background: [DESIGN.md](../DESIGN.md) — P6 · paper §3.2