 Story Crater Bot
|
f528902098
|
feat: M3.8.1 phase 4a — TextCompressor + env config (12 tests)
TextCompressor (320 LOC, 10 tests):
- Token importance scoring with lazy_static STOP_WORDS
- Keeps: high-entropy tokens (IDs, hashes, error codes, numbers, symbols)
- Drops: stop words, filler words, low-information prose
- ID detection: UUID, SHA256, session IDs, underscored patterns
- Error marker detection: error, exception, panic, fail, warn, critical
- Configurable compression ratio (default 40% token retention)
ContextOptimizerConfig::from_env() (2 tests):
- MEM_CONTEXT_OPTIMIZER (on/off)
- MEM_MAGIKA_ENABLED, MEM_MAGIKA_THRESHOLD
- MEM_COMPRESS_JSON, MEM_COMPRESS_LOGS, MEM_COMPRESS_CODE, MEM_COMPRESS_DIFF, MEM_COMPRESS_TEXT
- MEM_TOKEN_BUDGET, MEM_CCR_ENABLED
ContextOptimizer::from_env() factory method
62 optimizer tests total:
Phase 1 (17) + Phase 2 (15) + Phase 3 (18) + Phase 4a (12) = 62 passing
|
2026-08-28 10:02:52 -07:00 |
|
|
Story Crater Bot
|
e96510d80d
|
feat: M3.8.1 phase 3 — CacheAligner + CCR Store (18 tests)
CacheAligner (180 LOC, 8 tests):
- Detects dynamic patterns: timestamps, UUIDs, session IDs, temp paths, SHAs
- Uses once_cell Lazy statics + Regex for pattern matching
- Separates stable prefix (cache-able) from dynamic tail (varies)
- Reports drift metrics (0.0-1.0 ratio of dynamic content)
- Preserves identical prefixes across calls for KV cache hits
CcrStore (170 LOC, 10 tests):
- LRU cache with IndexMap (preserves insertion order)
- SHA256 hashing for content identification
- TTL-based expiry (default 1hr, configurable)
- Thread-safe (Mutex-wrapped)
- Supports large content (tested 100KB+)
ContextOptimizer integration (2 tests):
- Wired CCR store into optimizer
- Stores originals when compression occurs + CCR enabled
- Returns hash for retrieval hints
50 optimizer tests total:
Phase 1 (17) + Phase 2 (15) + Phase 3 (18) = 50 passing
|
2026-08-28 09:39:31 -07:00 |
|
|
Story Crater Bot
|
b18932b10c
|
feat: M3.8.1 phase 1 — content router + log compressor
ContentRouter uses Google Magika ML for content detection (<1ms) with regex
fallback. Detects JSON, code, logs, diffs, config, text.
LogCompressor reuses M3.7.7 patterns (markers, cascade, strip_ansi) to
shrink build logs by keeping errors/stacks and dropping noise.
17 unit tests passing:
- router: json, code, diff, log, text detection
- log: error lines, stack traces, ansi stripping, compression
- optimizer: token estimation, passthrough mode
Magika + ort ONNX runtime added to Cargo.toml.
|
2026-08-28 09:29:56 -07:00 |
|
|
Story Crater Bot
|
695e115212
|
Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)
|
2026-08-22 23:13:42 -07:00 |
|
|
Story Crater Bot
|
33b7150f56
|
feat: complete M0.1-M0.4 phases
M0.1 - Cargo workspace + crate skeletons
- 6-crate workspace with correct dependency direction
- CI/CD pipeline with GitHub Actions
- Integration tests verifying build and dependency structure
M0.2 - Domain types and sha256 identity
- Level (L0, L1, L2) enum with proper serde formatting
- Role enum (User, Assistant, ToolResult, System)
- Record, Chunk, and MemoryNode domain types
- Content-hash identity system ensuring rebuild idempotence
- Newtypes (ProjectId, QueryId, RunId) with validation
- Round-trip serde tests for all types
M0.3 - RecordSource trait + ChunkPolicy
- RecordSource trait for streaming record sources
- Chunk policy with token budgets and boundary modes
- TokenCounter trait with CharsOverFourCounter stub
- Chunking stream that respects budgets without splitting records
- VecSource for testing
- Integration tests verifying lossless chunking and budget adherence
M0.4 - Tokenizer-backed chunk sizing
- Vendored Qwen2 tokenizer with hash verification
- QwenTokenCounter implementing proper token counting
- Hash guard that fails on modified tokenizer
- mem tokens CLI subcommand for token counting
- Integration tests with known string counts, hash guards, and budget verification
Total: 19 integration tests passing, all phases verified to compose correctly
Workspace builds cleanly with no clippy warnings
|
2026-08-22 23:13:42 -07:00 |
|