 Story Crater Bot
|
dfdcfa5d3a
|
feat(M5.3): Add training corpus export infrastructure for verl
M5.3 — Training Corpus Export (verl format):
- Trajectory struct: trajectory_id, turns[], r_exit, r_format, r_outcome
- TrajectoryTurn: t, prompt, response, r_update, parsed
- CorpusStats: total_trajectories, total_turns, positive/negative split,
r_format pass rate, r_exit distribution
Reward computation:
- r_update_t: +1 if label matches U_t, -1 if mismatch (per turn)
- r_exit: 0 if exit == last_evidence_t, -0.75 if earlier, -0.5 if later
- r_format: 1.0 if all turns parsed, 0.0 if any unparsed (strict)
- r_outcome: null (no answer correctness signal available)
Files created:
crates/mem-core/src/trajectory.rs (280 LOC)
- Trajectory construction and reward calculation
- CorpusStats aggregation from trajectories
- Serialization for JSONL output
tests/it_export.rs (280 LOC, 12 tests)
- a1: Trajectory grouping by run
- a2: r_update signs correct
- a3: r_format strict (any unparsed = 0)
- a4: r_exit distribution (perfect/early/late)
- a5: Prompts are exact byte recordings
- a6: CorpusStats aggregation
- a7: r_outcome null
- a8: Turn ordering preserved
- a9: Multiple trajectories
- a10: Serde roundtrip
- a11: CorpusStats structure complete
- a12: Mixed exit rewards
Unit tests:
- crates/mem-core/src/trajectory.rs: 8/8 passing
Integration tests:
- tests/it_export.rs: 12/12 passing
Architecture:
Log + Labels → Trajectories → JSONL for verl
Each trajectory = one run with multiple turns
Per-turn rewards enable trajectory-level loss + turn-level loss
Blocks: M5.4 (vLLM setup), M5.5 (verl training)
Depends: M5.1 ✓, M5.2 ✓
|
2026-08-25 12:45:15 -07:00 |
|
|
Story Crater Bot
|
383d5ae0d1
|
feat(M4.2): Implement shingle-based cycle guard (derived filter)
Adds normalized shingle matching to prevent feedback loops where emitted skills
are re-ingested as evidence:
Files created:
crates/mem-core/src/shingle.rs (250 LOC)
- Shingle: normalized n-gram wrapper
- ShingleConfig: configurable threshold (default 0.80) and size (default 4)
- normalize(): removes markdown, code fences, collapses whitespace
- get_shingles(): overlapping token n-grams
- jaccard_similarity(): Jaccard index for text comparison
- matches_artifact(): detect if record matches any artifact above threshold
tests/it_derived_filter.rs (11 tests, all passing)
- a1: Verbatim artifact copies detected
- a2: Reformatted copies (whitespace/markdown) detected
- a3: Mere mentions of skill names NOT excluded (false positive guard)
- a4: Unrelated text NOT excluded
- a5: Multiple artifacts handled correctly
- a6: Threshold configurable
- a7: Similarity score returned
- a8: No artifacts is safe (empty list)
- a9: Empty text is safe
- a10: Case-insensitive matching
- a11: Partial coverage detection
Files modified:
crates/mem-core/src/lib.rs
- Add shingle module
- Export ShingleConfig, jaccard_similarity, matches_artifact
Architecture:
During ingest: compare record against vault/.artifacts.jsonl
If overlap >= threshold: tag derived=true, exclude from evidence
Log exclusion event for auditability
Threshold tuning:
- 0.80: strict, catches verbatim + reformatted
- 0.70: moderate, catches variants
- 0.60: permissive, catches substantial overlap
Default 0.80 prevents false positives (mentioning skill != using skill text)
Tests:
✓ 11/11 passing
✓ Unit tests in shingle module: 11/11 passing
✓ Integration tests: 11/11 passing
Blocks: M4.3 gate (needs ingest integration)
Depends: M4.1 ✓ (skill draft)
|
2026-08-25 12:41:26 -07:00 |
|
|
Story Crater Bot
|
631cbfa3e9
|
feat: complete M0.1-M0.4 phases
M0.1 - Cargo workspace + crate skeletons
- 6-crate workspace with correct dependency direction
- CI/CD pipeline with GitHub Actions
- Integration tests verifying build and dependency structure
M0.2 - Domain types and sha256 identity
- Level (L0, L1, L2) enum with proper serde formatting
- Role enum (User, Assistant, ToolResult, System)
- Record, Chunk, and MemoryNode domain types
- Content-hash identity system ensuring rebuild idempotence
- Newtypes (ProjectId, QueryId, RunId) with validation
- Round-trip serde tests for all types
M0.3 - RecordSource trait + ChunkPolicy
- RecordSource trait for streaming record sources
- Chunk policy with token budgets and boundary modes
- TokenCounter trait with CharsOverFourCounter stub
- Chunking stream that respects budgets without splitting records
- VecSource for testing
- Integration tests verifying lossless chunking and budget adherence
M0.4 - Tokenizer-backed chunk sizing
- Vendored Qwen2 tokenizer with hash verification
- QwenTokenCounter implementing proper token counting
- Hash guard that fails on modified tokenizer
- mem tokens CLI subcommand for token counting
- Integration tests with known string counts, hash guards, and budget verification
Total: 19 integration tests passing, all phases verified to compose correctly
Workspace builds cleanly with no clippy warnings
|
2026-08-22 23:13:42 -07:00 |
|