plan: add Magika ML classifier to content router

tasks/M3.8.1-context-optimizer.md

@@ -8,7 +8,7 @@
 | Flags | — |
 | Spec | `docs/CONTEXT_OPTIMIZER.md` |
 | Blocks | M3.8.4 |
-| Depends | M3.7.7 (lesson.rs patterns), M3.7.8 (stop words) |
+| Depends | M3.7.7 (lesson.rs patterns), M3.7.8 (stop words), magika crate |
 
 ## Goal
 
@@ -24,15 +24,26 @@ before they enter the GRU-Mem prompt. Search indexes stay untouched.
 - `crates/mem-core/src/optimizer/router.rs` — content type detection
 - `crates/mem-core/src/optimizer/log.rs` — log compression
 
-**ContentRouter** detects content type via heuristics:
+**ContentRouter** uses Google Magika (ML) + regex fallback:
 
-| Type | Signal |
-|---|---|
-| Json | starts with `{` or `[`, valid JSON parse |
-| Log | timestamp patterns, log levels, `error:`, `npm ERR!` |
-| Diff | `---`/`+++`/`@@` markers |
-| Code | `import`/`use`/`fn`/`def`/`class` + indentation |
-| Text | fallback |
+```rust
+// Primary: Magika ML classifier (ONNX, <1ms per classification)
+let magika = magika::Session::new()?;
+let result = magika.identify_content_sync(content.as_bytes())?;
+let label = result.info().label; // "json", "python", "shell", "yaml", etc.
+
+// Map Magika labels → our compressor types
+// Fallback to regex heuristics if Magika confidence < threshold
+```
+
+| Type | Magika Labels | Regex Fallback |
+|---|---|---|
+| Json | `json`, `jsonl` | starts with `{` or `[`, valid parse |
+| Log | `txt` + log heuristics | timestamp patterns, `error:`, `npm ERR!` |
+| Diff | `diff` | `---`/`+++`/`@@` markers |
+| Code | `python`, `javascript`, `rust`, `go`, `typescript`, `shell` | `import`/`use`/`fn`/`def` |
+| Config | `yaml`, `toml`, `ini`, `xml` | key-value patterns |
+| Text | fallback | default |
 
 **LogCompressor** reuses M3.7.7 `lesson.rs`:
 - `markers()` for error line detection
@@ -41,10 +52,12 @@ before they enter the GRU-Mem prompt. Search indexes stay untouched.
 - Keep: error lines, stack traces, exit codes
 - Drop: INFO/DEBUG noise, passing tests, repeated patterns
 
-**Tests (10):**
-- `detect_json`, `detect_log`, `detect_diff`, `detect_code`, `detect_text`
+**Tests (12):**
+- `magika_detects_json`, `magika_detects_python`, `magika_detects_diff`
+- `fallback_detects_log`, `fallback_detects_code`, `fallback_detects_text`
+- `router_maps_magika_to_compressor`, `router_low_confidence_uses_fallback`
 - `log_keeps_errors`, `log_drops_info_noise`, `log_keeps_stack_traces`
-- `log_compression_ratio_above_80pct`, `log_strips_ansi`
+- `log_compression_ratio_above_80pct`
 
 ### Phase 2: JsonCrusher + DiffCompressor (day 2)