feat(phase3-4): Complete hybrid retrieval + LLM optimization pipeline
Phase 3: Hybrid Retrieval - HybridRetriever: TF-IDF prefilter + semantic rerank + RRF fusion - WikiScopedFilter: BFS wiki-graph traversal - RetrievalRoute: Direct | WikiScoped | ReferenceOnly - 10 unit tests Phase 4: LLM Call Optimization - ChunkOptimizer: unified pipeline (threshold + budget + dedup) - ScoreThresholdFilter: configurable min_score (default 0.6) - BudgetSelector: greedy selection within byte budget - ShingleDeduplicator: Jaccard similarity dedup - 8 unit tests QueryRouter (Phase 3+4 Integration) - Bridges WikiLinkGraph + HybridRetriever + ChunkOptimizer - RouterConfig: max_hops, thresholds, budget, RRF weights - WikiGraphBuilder: construct graph from markdown docs - 11 unit tests Integration Tests (it_phase3_phase4.rs) - 19 end-to-end tests covering full pipeline - Wiki-link parsing, graph traversal, route selection - TF-IDF prefilter, RRF fusion, chunk optimization - Edge cases (empty, no matches, config customization) Total: 107 tests passing (was 32)
This commit is contained in:
@@ -25,6 +25,7 @@ pub mod query_filter;
|
||||
pub mod advanced_ranking;
|
||||
pub mod result_compressor;
|
||||
pub mod federation;
|
||||
pub mod query_router;
|
||||
|
||||
pub use endpoints::{IngestQueue, IngestRequest, JobStatus};
|
||||
pub use ingest_worker::IngestWorker;
|
||||
@@ -35,3 +36,4 @@ pub use chunk_metadata::{MetadataExtractor, MetadataBooster, ChunkMetadata, Chun
|
||||
pub use cache_alignment::{LruChunkCache, KvCacheAligner, CacheLocalityAnalyzer, RetrievalProfiler, CacheMetrics};
|
||||
pub use query_orchestrator::{QueryOrchestrator, QueryResult, OptimizedChunk, QueryContext, MemoryProjection};
|
||||
pub use query_filter::{QueryFilter, FilterableDocument, FilterEngine, FilterStatistics};
|
||||
pub use query_router::{QueryRouter, RouterConfig, RoutedResult, SelectedChunk, WikiGraphBuilder};
|
||||
|
||||
Reference in New Issue
Block a user