Story Crater Bot
f528902098
TextCompressor (320 LOC, 10 tests): - Token importance scoring with lazy_static STOP_WORDS - Keeps: high-entropy tokens (IDs, hashes, error codes, numbers, symbols) - Drops: stop words, filler words, low-information prose - ID detection: UUID, SHA256, session IDs, underscored patterns - Error marker detection: error, exception, panic, fail, warn, critical - Configurable compression ratio (default 40% token retention) ContextOptimizerConfig::from_env() (2 tests): - MEM_CONTEXT_OPTIMIZER (on/off) - MEM_MAGIKA_ENABLED, MEM_MAGIKA_THRESHOLD - MEM_COMPRESS_JSON, MEM_COMPRESS_LOGS, MEM_COMPRESS_CODE, MEM_COMPRESS_DIFF, MEM_COMPRESS_TEXT - MEM_TOKEN_BUDGET, MEM_CCR_ENABLED ContextOptimizer::from_env() factory method 62 optimizer tests total: Phase 1 (17) + Phase 2 (15) + Phase 3 (18) + Phase 4a (12) = 62 passing
25 lines
617 B
TOML
25 lines
617 B
TOML
[package]
|
|
name = "mem-core"
|
|
version = "0.1.0"
|
|
edition = "2021"
|
|
publish = false
|
|
|
|
[dependencies]
|
|
tokio = { workspace = true }
|
|
futures = { workspace = true }
|
|
serde = { workspace = true }
|
|
serde_json = { workspace = true }
|
|
serde_yaml = { workspace = true }
|
|
anyhow = { workspace = true }
|
|
thiserror = { workspace = true }
|
|
sha2 = { workspace = true }
|
|
tracing = { workspace = true }
|
|
hex = "0.4"
|
|
time = { version = "0.3", features = ["serde", "formatting", "parsing", "macros"] }
|
|
magika = "1.1.0"
|
|
ort = { version = "2.0.0-rc.12", default-features = true }
|
|
regex = "1.10"
|
|
once_cell = "1.19"
|
|
indexmap = "2.0"
|
|
lazy_static = "1.4"
|