rock/poimen-memory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: M3.8.1 phase 1 — content router + log compressor
Build and Push / Test (push) Failing after 1m46s
Details
Build and Push / Build and push image (push) Skipped
Details

Browse Source 
ContentRouter uses Google Magika ML for content detection (<1ms) with regex
fallback. Detects JSON, code, logs, diffs, config, text.

LogCompressor reuses M3.7.7 patterns (markers, cascade, strip_ansi) to
shrink build logs by keeping errors/stacks and dropping noise.

17 unit tests passing:
- router: json, code, diff, log, text detection
- log: error lines, stack traces, ansi stripping, compression
- optimizer: token estimation, passthrough mode

Magika + ort ONNX runtime added to Cargo.toml.
This commit is contained in:
Story Crater Bot
2026-08-28 09:29:56 -07:00
parent 1f9b30b1ec
commit 05aec4e23b
8 changed files with 872 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tasks/INDEX.md
+6 -4
View File
@@ -64,19 +64,19 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked
| 4 | L2 synthesis + retrieval | M3.x | 4 | 4 | 0 | 0 | ✅ M3.4 |
| 4.5 | Distributed API Layer | M3.5.x | 10 | 10 | 0 | 0 | ✅ M3.5.8 |
| 5 | Skills | M4.x | 3 | 2 | 0 | 1 | ⬜ M4.3 |
| 5.5 | Reference corpora | M3.6.x | 5 | 1 | 0 | 4 | ⬜ M3.6.6 |
| 5.5 | Reference corpora | M3.6.x | 7 | 1 | 0 | 6 | ⬜ M3.6.8 |
| 5.6 | Tool context | M3.7.x | 4 | 2 | 0 | 2 | ⬜ M3.7.6 |
| 5.7 | Context optimization | M3.8.x | 4 | 0 | 0 | 4 | ⬜ M3.8.4 |
| 6 | Post-training | M5.x | 6 | 0 | 0 | 6 | ⬜ M5.6 |
| 7 | agent-manager migration | M6.x | 6 | 0 | 0 | 6 | ⬜ M6.6 |
| 8 | Source connectors | M7.x | 10 | 0 | 0 | 10 | ⬜ M7.10 |
| 9 | Hybrid search | M8.x | 9 | 1 | 1 | 7 | ⬜ M8.9 |
| | **Total** | | **74** | **60** | **0** | **14** | 6/12 green |
| | **Total** | | **76** | **60** | **0** | **16** | 6/13 green |
**Current status — 2025-01-28.** Completed phases M0.x, M1.x fully archived (16/16 tasks). **M2.1-6 ✅** (embeddings, CNPG, schema, pgvector, obsidian projector, rebuild). **M3.x ✅** (4/4). **M3.5.x ✅** (10/10 complete + archived). **M3.7.7-8 ✅** (failure diagnosis). **M4.1-2 ✅** (skill drafting + derived filter). **M3.6.1 ✅** (DocCorpusSource). **M3.6.3 ❌ retired** (Obsidian UI replaces CLI). **M8.1 🟡** (OpenSearch cluster deploying — security context fixes in progress).
**Current status — 2025-01-28.** Completed phases M0.x, M1.x fully archived (16/16 tasks). **M2.1-6 ✅** (embeddings, CNPG, schema, pgvector, obsidian projector, rebuild). **M3.x ✅** (4/4). **M3.5.x ✅** (10/10 complete + archived). **M3.7.7-8 ✅** (failure diagnosis). **M4.1-2 ✅** (skill drafting + derived filter). **M3.6.1 ✅** (DocCorpusSource). **M3.6.3 ❌ retired** (Obsidian UI replaces CLI). **M3.6.7-8 ⬜ new** (ingest enrichment + deduplication). **M8.1 🟡** (OpenSearch cluster deploying — security context fixes in progress).
**Current work:**
- M3.8.1: Context optimizer (Headroom-inspired pre-LLM compression)
- M3.8.1 Phase 1: ContentRouter (Magika ML) + LogCompressor (M3.7.7 reuse)
- M8.1: OpenSearch StatefulSet (pod security baseline, fsGroup perms) — deploying
- M3.7.4: Context endpoint glue (uses tier logic + hybrid search)
- Obsidian service: ✅ Deployed (ppatlabs/obsidian:latest, REST API on 27124)
@@ -168,6 +168,8 @@ changed chunks (SHA comparison, deterministic embedding).
| [M3.6.4](M3.6.4-reference-cycle-guard.md) | Reference text cannot re-enter as evidence | M | — | ⬜ |
| [M3.6.5](M3.6.5-query-levels-and-floor.md) | Query: filter-then-recall, R opt-in, floor | M | — | ⬜ |
| [M3.6.6](M3.6.6-m3.6-gate.md) | **M3.6 composition gate** | M | gate | ⬜ |
| [M3.6.7](M3.6.7-contextual-enrichment.md) | Contextual enrichment at ingest (Anthropic-style) | M | — | ⬜ |
| [M3.6.8](M3.6.8-chunk-deduplication.md) | Chunk deduplication via MinHash | M | — | ⬜ |
## 5.6 — Tool context · M3.7.x