feat(M5.3): Add training corpus export infrastructure for verl

M5.3 — Training Corpus Export (verl format):
  - Trajectory struct: trajectory_id, turns[], r_exit, r_format, r_outcome
  - TrajectoryTurn: t, prompt, response, r_update, parsed
  - CorpusStats: total_trajectories, total_turns, positive/negative split,
    r_format pass rate, r_exit distribution

Reward computation:
  - r_update_t: +1 if label matches U_t, -1 if mismatch (per turn)
  - r_exit: 0 if exit == last_evidence_t, -0.75 if earlier, -0.5 if later
  - r_format: 1.0 if all turns parsed, 0.0 if any unparsed (strict)
  - r_outcome: null (no answer correctness signal available)

Files created:
  crates/mem-core/src/trajectory.rs (280 LOC)
    - Trajectory construction and reward calculation
    - CorpusStats aggregation from trajectories
    - Serialization for JSONL output

  tests/it_export.rs (280 LOC, 12 tests)
    - a1: Trajectory grouping by run
    - a2: r_update signs correct
    - a3: r_format strict (any unparsed = 0)
    - a4: r_exit distribution (perfect/early/late)
    - a5: Prompts are exact byte recordings
    - a6: CorpusStats aggregation
    - a7: r_outcome null
    - a8: Turn ordering preserved
    - a9: Multiple trajectories
    - a10: Serde roundtrip
    - a11: CorpusStats structure complete
    - a12: Mixed exit rewards

Unit tests:
  - crates/mem-core/src/trajectory.rs: 8/8 passing

Integration tests:
  - tests/it_export.rs: 12/12 passing

Architecture:
  Log + Labels → Trajectories → JSONL for verl
  Each trajectory = one run with multiple turns
  Per-turn rewards enable trajectory-level loss + turn-level loss

Blocks: M5.4 (vLLM setup), M5.5 (verl training)
Depends: M5.1 ✓, M5.2 ✓
This commit is contained in:
Story Crater Bot
2026-08-25 12:45:15 -07:00
parent 6a873088e6
commit dfdcfa5d3a
3 changed files with 434 additions and 0 deletions
+2
View File
@@ -6,6 +6,7 @@ pub mod gate_parser;
pub mod gated_loop;
pub mod query_executor;
pub mod shingle;
pub mod trajectory;
pub use gate_parser::{GateResponse, ParseError, parse_gate_response};
@@ -19,3 +20,4 @@ pub use lesson::{
pub use query::{Query, QuerySet, SynthesisQuery};
pub use prompt::PromptBuilder;
pub use shingle::{jaccard_similarity, matches_artifact, Shingle, ShingleConfig};
pub use trajectory::{Trajectory, TrajectoryTurn, CorpusStats};