 Story Crater Bot
|
dfdcfa5d3a
|
feat(M5.3): Add training corpus export infrastructure for verl
M5.3 — Training Corpus Export (verl format):
- Trajectory struct: trajectory_id, turns[], r_exit, r_format, r_outcome
- TrajectoryTurn: t, prompt, response, r_update, parsed
- CorpusStats: total_trajectories, total_turns, positive/negative split,
r_format pass rate, r_exit distribution
Reward computation:
- r_update_t: +1 if label matches U_t, -1 if mismatch (per turn)
- r_exit: 0 if exit == last_evidence_t, -0.75 if earlier, -0.5 if later
- r_format: 1.0 if all turns parsed, 0.0 if any unparsed (strict)
- r_outcome: null (no answer correctness signal available)
Files created:
crates/mem-core/src/trajectory.rs (280 LOC)
- Trajectory construction and reward calculation
- CorpusStats aggregation from trajectories
- Serialization for JSONL output
tests/it_export.rs (280 LOC, 12 tests)
- a1: Trajectory grouping by run
- a2: r_update signs correct
- a3: r_format strict (any unparsed = 0)
- a4: r_exit distribution (perfect/early/late)
- a5: Prompts are exact byte recordings
- a6: CorpusStats aggregation
- a7: r_outcome null
- a8: Turn ordering preserved
- a9: Multiple trajectories
- a10: Serde roundtrip
- a11: CorpusStats structure complete
- a12: Mixed exit rewards
Unit tests:
- crates/mem-core/src/trajectory.rs: 8/8 passing
Integration tests:
- tests/it_export.rs: 12/12 passing
Architecture:
Log + Labels → Trajectories → JSONL for verl
Each trajectory = one run with multiple turns
Per-turn rewards enable trajectory-level loss + turn-level loss
Blocks: M5.4 (vLLM setup), M5.5 (verl training)
Depends: M5.1 ✓, M5.2 ✓
|
2026-08-25 12:45:15 -07:00 |
|