Story Crater Bot a45263410f feat: add cache-aligned prompt builder for LLM API cost savings ...

PROBLEM:
- PromptBuilder.build() puts everything in a single user message
- System + query + memory + chunk all change together
- LLM prompt caching gets 0% hits (entire message differs per call)
- For a 50-chunk ingestion run, we pay full input price 50 times

SOLUTION: PromptBuilder.build_cache_aligned()
- Splits prompt into 3 separate messages:
  1. SYSTEM: instructions (stable across ALL calls) → CACHED
  2. USER[0]: query/problem (stable per run) → CACHED
  3. USER[1]: memory + chunk (varies per call) → not cached
- Cache prefix (system + query) reused across all chunks in a run
- Estimated 30-70% cache hit ratio depending on chunk sizes
- ~50% input token cost savings for multi-chunk ingestion

TEMPLATES:
- templates/gru-mem-system.txt (instructions only, 840B)
- templates/gru-mem-query.txt (problem wrapper, 29B)
- templates/gru-mem-turn.txt (memory + section, 57B)
- templates/gru-mem.txt (legacy, unchanged)

API:
- PromptBuilder::build() — legacy, backward compatible
- PromptBuilder::build_cache_aligned() → PromptMessages
- PromptMessages.cache_prefix_tokens() — cacheable token count
- PromptMessages.total_tokens() — total estimated tokens
- PromptMessages.headroom() — tokens available for response

TESTS: 11 unit + 3 integration = 14 new tests
- test_cache_aligned_produces_two_user_messages
- test_cache_prefix_is_stable_across_chunks
- test_cache_prefix_is_stable_across_memory_changes
- test_cache_prefix_tokens_positive
- test_headroom_positive_under_budget
- test_legacy_build_still_works
- test_cache_aligned_contains_query
- test_cache_aligned_memory/chunk_budget_exceeded
- a8_cache_prefix_stable_across_50_chunks
- a9_cache_aligned_headroom
- a10_cache_savings_estimate

TOTAL: 64 mem-core tests passing (52 unit + 12 integration)

2026-08-28 08:24:38 -07:00

..

it_chat_client.rs

Fix LLM gateway path, update M1.8 gate test to load real chunks (Option B)

2026-08-23 00:32:27 -07:00

it_chunking.rs

feat: complete M0.1-M0.4 phases

2026-08-22 23:13:42 -07:00

it_claude_source.rs

feat: complete M0 phase - read-only spine (8/51 tasks)

2026-08-22 23:13:42 -07:00

it_derived_filter.rs

Implement M4.2: Derived filter (shingle matcher + 10 tests, 239 total)

2026-08-26 13:55:37 -07:00

it_doc_corpus.rs

Implement M3.6.1: DocCorpusSource with heading-boundary chunking (196 tests)

2026-08-23 09:42:09 -07:00

it_dry_run.rs

fix: Add jwt_validator module declaration to main.rs

2026-08-27 12:54:50 -07:00

it_e2e_pipeline.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_embeddings.rs

feat: Implement M2.1 Embeddings client (768-dim batching @32)

2026-08-27 20:36:57 -07:00

it_endpoints.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_event_log.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_gate_parser.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_gated_loop.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_http_server.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_identity.rs

feat: complete M0.1-M0.4 phases

2026-08-22 23:13:42 -07:00

it_ingest_endpoint.rs

Implement M3.5.2: POST /ingest endpoint with idempotent async queue (204 tests)

2026-08-23 16:33:34 -07:00

it_jwt_auth.rs

feat: JWT auth validation with Authentik OIDC

2026-08-27 12:29:23 -07:00

it_jwt_integration.rs

feat: JWT auth validation with Authentik OIDC

2026-08-27 12:29:23 -07:00

it_l2.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_m0_gate.rs

feat: complete M0 phase - read-only spine (8/51 tasks)

2026-08-22 23:13:42 -07:00

it_m1_gate.rs

Fix LLM gateway path, update M1.8 gate test to load real chunks (Option B)

2026-08-23 00:32:27 -07:00

it_m2_gate.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_m3_5_gate.rs

feat: M3.5.8 complete - all endpoints, rate limiting, and deployment (253 tests)

2026-08-23 17:19:42 -07:00

it_m3_gate.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_pg_repo.rs

feat: Implement M2.4 pgvector repository with real Postgres

2026-08-27 20:48:37 -07:00

it_pgvector.rs

feat(core): implement full memory pipeline (#11)

2026-08-24 01:37:16 +00:00

it_pi_source.rs

feat: complete M0 phase - read-only spine (8/51 tasks)

2026-08-22 23:13:42 -07:00

it_projects_endpoint.rs

feat: M3.5.8 complete - all endpoints, rate limiting, and deployment (253 tests)

2026-08-23 17:19:42 -07:00

it_prompt.rs

feat: add cache-aligned prompt builder for LLM API cost savings

2026-08-28 08:24:38 -07:00

it_query_endpoint.rs

feat: M3.5.8 complete - all endpoints, rate limiting, and deployment (253 tests)

2026-08-23 17:19:42 -07:00

it_query_executor.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_query_federation.rs

feat: M3.5.8 complete - all endpoints, rate limiting, and deployment (253 tests)

2026-08-23 17:19:42 -07:00

it_query_loader.rs

Deploy Poimen Memory K8s cluster with ArgoCD tracking (M2.2, M3.5-M3.7)

2026-08-22 23:13:42 -07:00

it_rate_limiting.rs

Implement M3.5.7: Rate limiting + idempotency (20 tests)

2026-08-26 13:35:50 -07:00

it_rebuild.rs

feat: Implement M2.5 & M2.6 — Obsidian vault projector + rebuild orchestrator

2026-08-27 20:54:43 -07:00

it_rerank.rs

fix: resolve module imports and rerank test format (#12)

2026-08-24 01:45:47 +00:00

it_signature.rs

feat: M3.7.7 signature extraction CLI + integration tests (unit tests pass, integration tests pending mem-cli fix)

2026-08-28 07:46:36 -07:00

it_skill_draft.rs

Implement M4.1: Skill draft command + 10 tests (229 total)

2026-08-26 13:50:22 -07:00

it_skills_endpoint.rs

feat: M3.5.8 complete - all endpoints, rate limiting, and deployment (253 tests)

2026-08-23 17:19:42 -07:00

it_symptom_projection.rs

feat: implement M3.7.8 symptom projection (250 LOC) + 22 tests (10 unit + 12 integration)

2026-08-28 08:08:55 -07:00

it_tokens.rs

feat: complete M0.1-M0.4 phases

2026-08-22 23:13:42 -07:00

it_verify.rs

refactor: replace Obsidian projector with standalone service (ppatlabs/obsidian)

2026-08-27 21:35:07 -07:00

it_workspace.rs

feat: complete M0.1-M0.4 phases

2026-08-22 23:13:42 -07:00