 Story Crater Bot
|
52605bd73a
|
chore: Remove Kong references, make infrastructure examples generic
Replaced Kong-specific examples with generic infrastructure scenarios:
- Known-answer questions: database timeout, model loading, GPU memory
- K8s manifest: generic timeout annotations (cloud-provider agnostic)
- Removed Kong timeouts, routes, plugins
- Added gateway configuration guidance for various platforms
- Updated HF token secret management
Benefits:
- System is now cloud-provider agnostic
- Works with any gateway (Istio, Nginx, cloud LB, etc.)
- Examples are more universally applicable
- Easier to adapt to different infrastructure
Affected files:
- verify/known-answers.yaml (3 generic scenarios)
- M3.4-GATE.md (updated expected answers)
- k8s/apps/llm-serving/memory-isvc.yaml (cloud-agnostic setup)
|
2026-08-25 13:50:53 -07:00 |
|
|
Story Crater Bot
|
764bbf3452
|
feat(M3.4): Implement composition gate for M3 (L2 + rerank + query)
Adds gate verification that M3.1 (L2 synthesis) + M3.2 (rerank) + M3.3 (query) work together:
Files added:
verify/known-answers.yaml
- 3 known-answer questions from real infrastructure findings
- Expected node texts and source substrings
- Gate thresholds: hit_rate ≥ 0.8, precision ≥ 0.9
verify/m3.4.sh (executable)
- Runs known-answer questions through mem query
- Measures hit rate at k=5
- Verifies provenance precision (90%+ of citations contain facts)
- Checks mem verify for level consistency
- Checks L2→L1→L0 edge resolution
- Exit 0 if all thresholds met, 1 if any fail
tests/it_m3_gate.rs
- 8 integration tests, 6 marked #[ignore] (need live DB)
- a1-a2: Known-answer Kong buffer / auth header
- a3: L2→L1→L0 two-hop provenance walks
- a4: Reranking improves order
- a5: No cross-project leakage
- a6: Level consistency check
- a7: Query command exists (✅ passes)
- a8: Verify command works (✅ passes)
Gate criteria (M3 passes when):
- Hit rate at k=5 ≥ 0.8
- Provenance precision ≥ 0.9
- mem verify clean
- L2→L1→L0 edges resolve
- Reranking maintains/improves accuracy
Status:
✅ Tests compile
✅ Smoke tests pass (a7, a8)
⏳ Full gate ready for seeded database
Blocks: M4 (skills implementation)
Depends: M3.1 ✅, M3.2 ✅, M3.3 ✅
|
2026-08-25 12:26:06 -07:00 |
|