Story Crater Bot
1991291bc9
PROBLEM: - PromptBuilder.build() puts everything in a single user message - System + query + memory + chunk all change together - LLM prompt caching gets 0% hits (entire message differs per call) - For a 50-chunk ingestion run, we pay full input price 50 times SOLUTION: PromptBuilder.build_cache_aligned() - Splits prompt into 3 separate messages: 1. SYSTEM: instructions (stable across ALL calls) → CACHED 2. USER[0]: query/problem (stable per run) → CACHED 3. USER[1]: memory + chunk (varies per call) → not cached - Cache prefix (system + query) reused across all chunks in a run - Estimated 30-70% cache hit ratio depending on chunk sizes - ~50% input token cost savings for multi-chunk ingestion TEMPLATES: - templates/gru-mem-system.txt (instructions only, 840B) - templates/gru-mem-query.txt (problem wrapper, 29B) - templates/gru-mem-turn.txt (memory + section, 57B) - templates/gru-mem.txt (legacy, unchanged) API: - PromptBuilder::build() — legacy, backward compatible - PromptBuilder::build_cache_aligned() → PromptMessages - PromptMessages.cache_prefix_tokens() — cacheable token count - PromptMessages.total_tokens() — total estimated tokens - PromptMessages.headroom() — tokens available for response TESTS: 11 unit + 3 integration = 14 new tests - test_cache_aligned_produces_two_user_messages - test_cache_prefix_is_stable_across_chunks - test_cache_prefix_is_stable_across_memory_changes - test_cache_prefix_tokens_positive - test_headroom_positive_under_budget - test_legacy_build_still_works - test_cache_aligned_contains_query - test_cache_aligned_memory/chunk_budget_exceeded - a8_cache_prefix_stable_across_50_chunks - a9_cache_aligned_headroom - a10_cache_savings_estimate TOTAL: 64 mem-core tests passing (52 unit + 12 integration)
10 lines
840 B
Plaintext
10 lines
840 B
Plaintext
You are a memory gate for a recurrent update system. You evaluate whether new evidence sections contain information relevant to a standing problem, and update the running memory accordingly.
|
|
|
|
Rules:
|
|
1. Retain all relevant details from the previous memory while adding new useful information.
|
|
2. Judge whether you have collected enough information to answer the problem.
|
|
3. Reason about the new section between <think> and </think>.
|
|
4. If the section contains useful information: output <check>yes</check>, then update memory between <update> and </update>.
|
|
5. If the section does NOT contain useful information: output <check>no</check>, then keep previous memory unchanged between <update> and </update>.
|
|
6. If more information is needed: return <next>continue</next>.
|
|
7. ONLY when enough information is collected: return <next>end</next>. |