poimen-memory/M5-PROGRESS.md

# M5 Progress — Post-Training Infrastructure

**Status:** M5.1-M5.3 COMPLETE (Labeling, Calibration, Corpus Export)

Date: 2026-08-25

---

## What was accomplished

### M5.1 — Evidence Labeler (Distant Supervision)

**Goal:** Label chunks as containing evidence for Q using a 32B reasoning model.

**Implemented:**
```rust
// EvidenceLabel struct
pub struct EvidenceLabel {
    pub chunk_sha: String,      // Keyed by chunk SHA (survives re-chunking)
    pub t: usize,               // Turn number (for reference)
    pub label: bool,            // true = evidence, false = no evidence
    pub why: String,            // 1-sentence justification (for M5.2)
    pub model: String,          // "reasoning" (32B)
    pub ts: String,             // ISO 8601 timestamp
}

// Labeling pipeline
make_label_prompt()          // Assemble prompt in 16K budget
parse_label_response()       // Extract yes/no + justification
fits_context_budget()        // Verify 16K limit not exceeded
```

**Features:**
- Context budget checked (reasoning model limit: 16384 tokens)
- Justifications preserved (enables disagreement analysis in M5.2)
- No tools field (reasoning model rejects function calls)
- Resumable (skip already-labeled chunks by sha)

**Files:**
- `crates/mem-llm/src/labeler.rs` (250 LOC)
- Unit tests: 8/8 passing
- Integration tests: 11/11 passing (tests/it_labeler.rs)

---

### M5.2 — Labeler Calibration (Cohen's Kappa)

**Goal:** Measure labeler accuracy via hand-labeled holdout before training.

**Implemented:**
```rust
// Calibration results
pub struct CalibrationResults {
    pub tp: usize,           // True positives
    pub tn: usize,           // True negatives
    pub fp: usize,           // False positives
    pub fn_: usize,          // False negatives
    pub accuracy: f32,       // Raw agreement (misleading on class imbalance)
    pub kappa: f32,          // Cohen's kappa (corrects for chance)
    pub precision: f32,      // tp / (tp + fp)
    pub recall: f32,         // tp / (tp + fn)
    pub f1: f32,             // Harmonic mean
}

// Stratified sampling (50/50 positive/negative, not corpus-proportional)
pub fn stratified_sample() -> Vec<usize>

// Blind worksheet (hides labeler answers from human)
pub fn to_blind_json()

// Gate: kappa >= 0.6
pub fn passes_gate() -> bool
```

**Example:**
- 95% negative corpus: accuracy of "always say no" ≈ 95% (useless)
- But kappa ≈ 0.0 (Cohen's kappa correctly shows this is random)
- This is why accuracy is reported alongside kappa

**Files:**
- `crates/mem-llm/src/calibration.rs` (280 LOC)
- Unit tests: 6/6 passing
- Integration tests: 12/12 passing (tests/it_calibration.rs)

**Gate:**
- κ ≥ 0.6 required before labels are used for training
- κ < 0.6 blocks M5.3 corpus export and training

---

### M5.3 — Training Corpus Export (Verl Format)

**Goal:** Convert log + labels into trajectories for verl RL training.

**Implemented:**
```rust
// Trajectory = one run with multiple turns
pub struct Trajectory {
    pub trajectory_id: String,
    pub turns: Vec<TrajectoryTurn>,
    pub r_exit: f32,         // Exit reward (-0.75, 0.0, or -0.5)
    pub r_format: f32,       // 1.0 if all parsed, 0.0 if any unparsed
    pub r_outcome: Option<f32>, // null (no correctness signal)
}

// Per-turn reward
pub struct TrajectoryTurn {
    pub t: usize,
    pub prompt: String,       // Exact bytes sent to model
    pub response: String,     // Exact bytes from model
    pub r_update: i32,        // +1 if label matches, -1 if mismatch
    pub parsed: bool,
}

// Statistics summary
pub struct CorpusStats {
    pub total_trajectories: usize,
    pub total_turns: usize,
    pub positive_r_update: usize,
    pub negative_r_update: usize,
    pub r_format_pass_rate: f32,
    pub r_exit_distribution: HashMap<String, usize>,
}
```

**Reward Logic:**
- `r_update_t = +1` if M5.1's label matches recorded U_t, else -1 (per turn)
- `r_exit = 0` if exit turn == last_evidence_t (perfect)
- `r_exit = -0.75` if exit < last_evidence_t (missed evidence, bad)
- `r_exit = -0.5` if exit > last_evidence_t (continued, moderate)
- `r_format = 1.0` only if all turns parsed, 0 otherwise (strict)
- `r_outcome = null` (no answer-correctness signal available)

**Files:**
- `crates/mem-core/src/trajectory.rs` (280 LOC)
- Unit tests: 8/8 passing
- Integration tests: 12/12 passing (tests/it_export.rs)

---

## Test Results Summary

**M5.1 Tests (Evidence Labeler):**
```
it_labeler.rs
  ✓ a1_one_label_per_chunk
  ✓ a2_keyed_by_sha
  ✓ a3_context_budget_respected
  ✓ a4_justification_kept
  ✓ a5_label_structure
  ✓ a6_prompt_no_tools_field
  ✓ a7_parsing_handles_variations
  ✓ a8_empty_prompt_safe
  ✓ a9_large_chunk_exceeds_budget
  ✓ a10_label_rate_summary
  ✓ a11_evidence_label_serde
Total: 11/11 passing
```

**M5.2 Tests (Calibration):**
```
it_calibration.rs
  ✓ a1_worksheet_is_blind
  ✓ a2_stratified_sampling
  ✓ a3_kappa_perfect_agreement
  ✓ a4_kappa_vs_accuracy
  ✓ a5_confusion_matrix
  ✓ a6_precision_recall_separate
  ✓ a7_gate_threshold_kappa_06
  ✓ a8_f1_score_computed
  ✓ a9_calibration_sample_roundtrip
  ✓ a10_disagreement_analysis
  ✓ a11_sample_size_sufficient
  ✓ a12_kappa_formula_correct
Total: 12/12 passing
```

**M5.3 Tests (Corpus Export):**
```
it_export.rs
  ✓ a1_trajectory_grouping
  ✓ a2_r_update_signs
  ✓ a3_r_format_strict
  ✓ a4_r_exit_distribution
  ✓ a5_prompt_exact_bytes
  ✓ a6_corpus_stats_aggregation
  ✓ a7_r_outcome_null
  ✓ a8_trajectory_ordering
  ✓ a9_multiple_trajectories
  ✓ a10_trajectory_serde_roundtrip
  ✓ a11_corpus_stats_structure
  ✓ a12_mixed_exit_rewards
Total: 12/12 passing
```

**Unit Tests (Embedded):**
```
mem-llm/labeler.rs:       8/8 passing
mem-llm/calibration.rs:   6/6 passing
mem-core/trajectory.rs:   8/8 passing
```

**Grand Total: 57 tests passing, 0 failing**

---

## Architecture Overview

```
M5.1: Labeling Pipeline
  chunks + questions
    ↓
  reasoning model (32B)
    ↓
  labels + justifications

M5.2: Calibration
  labeler labels
    ↓
  hand-labeled holdout (100 samples, stratified 50/50)
    ↓
  κ, precision, recall → gate (κ ≥ 0.6)

M5.3: Corpus Export
  log + labels
    ↓
  trajectories (grouped by run)
    ↓
  rewards (r_update, r_exit, r_format, r_outcome)
    ↓
  JSONL for verl training
```

---

## What Remains for M5

**M5.4 — vLLM LoRA Setup** (Kubernetes infrastructure)
- Deploy vLLM with `--enable-lora`
- Configure Kong routes and timeouts
- Ready for LoRA adapter serving

**M5.5 — verl Training Loop** (Python track, can run in parallel)
- verl training loop with trajectory batching
- Policy gradient with α-blended loss
- Adapter checkpoint saving

**M5.6 — M5 Gate** (Full integration test)
- Train controller on exported corpus
- Measure return-over-baseline
- Verify improvement

---

## Integration Points

**From M4:**
- M4.1: Skill drafts → artifact manifest
- M4.2: Shingle filter → `derived: true` tag
- M4.3: Proven cycle remains open

**To M5.4+:**
- M5.3 exports JSONL trajectories
- M5.4 serves memory controller LoRA
- M5.5 trains on exported corpus

---

## Statistics

**Lines of Code:**
- M5.1 Labeler: 250 LOC
- M5.2 Calibration: 280 LOC
- M5.3 Trajectory: 280 LOC
- Tests: 900+ LOC
- **Total: ~1,700 LOC**

**Tests:**
- Unit tests: 22 passing
- Integration tests: 35 passing
- **Total: 57/57 passing**

**Key Data Structures:**
- EvidenceLabel (6 fields, Serde)
- CalibrationResults (9 fields, kappa formula)
- Trajectory (5 fields, rewards)
- CorpusStats (6 fields, aggregation)

---

## Gate Status

**M5.1 Complete:** No gate (labeling phase)

**M5.2 Gate:** κ ≥ 0.6
- Passes only if hand-labeled holdout shows agreement
- Blocks M5.3 corpus export if κ < 0.6
- Ensures low-quality labels don't corrupt training

**M5.3 Complete:** Trajectories ready for verl

---

## Build Status

✅ All code compiles
✅ All tests pass (57/57)
✅ No warnings or errors
✅ Cargo check clean

---

## Next Steps

1. **M5.4** — vLLM LoRA deployment (K8s)
2. **M5.5** — verl training loop (Python)
3. **M5.6** — M5 gate (integration test)
4. **M6** — Agent-manager migration (optional parallel track)

---

## Session Summary

**What was built this session (M4.2-M5.3):**
- M4.2: Shingle matching (cycle guard)
- M4.3: M4 gate tests
- M5.1: Labeler + tests
- M5.2: Calibration + tests
- M5.3: Trajectory export + tests

**Total commits:** 5
- 3 code commits (M4.2, M5.1-M5.2, M5.3)
- 2 documentation commits

**Progress:** 47/64 tasks complete (73%)
- M3: ✅ Complete
- M4: ✅ Complete (M4.1 CLI, M4.2 shingle guard, M4.3 gate)
- M5: 🟡 3/6 complete (M5.1, M5.2, M5.3 infrastructure)
- M5.4: ⏳ Ready (vLLM setup)
- M5.5: ⏳ Ready (verl training)
- M5.6: ⏳ Ready (gate)

**Estimated time to M5 complete:** 2-3 weeks (M5.4 parallel, M5.5 sequential)