poimen-memory/docs/IMPLEMENTATION_NOTES.md

Implementation Notes: Query Optimization Engine

Status

✅ Design Phase COMPLETE

• Query Optimizer (query_optimizer.rs) — READY
• Hybrid Query Worker (hybrid_query_worker.rs) — STUB (needs API integration)
• Design Documentation (QUERY_OPTIMIZATION_ENGINE.md) — COMPLETE

⚠️ API Integration Notes (for Phase 2)

---

VectorStore API Corrections

Current Methods (Confirmed)

The actual VectorStore has level-based search methods:

// NOT available:
vector_store.search(&embedding, project, limit, None)
vector_store.search_with_ids(&embedding, project, limit, chunk_ids)

// ACTUALLY available:
vector_store.search_l1(project, &embedding, limit)
vector_store.search_l2(project, &embedding, limit)
vector_store.search_l3(project, &embedding, limit)

Updated Semantic Retrieval

async fn retrieve_semantic(
    &self,
    project: &str,
    query_ctx: &QueryContext,
    limit: i64,
) -> Result<Vec<(String, f32)>> {
    let embedding = query_ctx
        .embedding
        .as_ref()
        .ok_or_else(|| anyhow::anyhow!("no embedding"))?;

    // Use L1 (most specific level)
    let results = self.vector_store.search_l1(project, embedding, limit).await?;

    // Convert to (id, score) tuples
    let scored: Vec<(String, f32)> = results
        .into_iter()
        .map(|r| (r.id, r.score))
        .collect();

    Ok(scored)
}

---

OpenSearchClient API Corrections

Issue: Method Visibility

The lexical_search method in opensearch_client.rs is private:

// NOT public (private):
async fn lexical_search(...)

// NEEDED:
pub async fn lexical_search(...)

Fix

Make the method public:

// In crates/mem-cli/src/opensearch_client.rs
pub async fn lexical_search(
    &self,
    query: &str,
    limit: usize,
    jwt_token: &str,
) -> Result<Vec<(String, f32, String, String, Vec<String>)>>

---

Simplified Phase 2 Implementation

For Phase 2, instead of modifying http_server.rs extensively, create a wrapper:

/// In crates/mem-cli/src/http_server.rs

async fn query_handler(...) -> HttpResponse {
    let (claims, token) = match validate_auth(&req, &state).await {
        Ok(c) => c,
        Err(e) => return e,
    };

    // ... existing checks ...

    // Try hybrid if available
    #[cfg(feature = "hybrid_search")]
    {
        match state.hybrid_query_worker.query(
            &project,
            &question,
            limit,
            &token,
        ).await {
            Ok(response) => return HttpResponse::Ok().json(response),
            Err(e) => {
                tracing::warn!("Hybrid query failed: {}, falling back", e);
            }
        }
    }

    // Fallback: existing semantic search
    match state.query_worker.query(&project, &question, Some(limit)).await {
        Ok(results) => HttpResponse::Ok().json(json!({
            "query": question,
            "project": project,
            "results": results,
            "method": "semantic_only"  // Indicate fallback
        })),
        Err(e) => {
            tracing::error!("Query failed: {}", e);
            HttpResponse::InternalServerError().json(json!({"error": "query_failed"}))
        }
    }
}

---

Build Status

Current Issues (Non-Blocking Design)

1. hybrid_query_worker.rs uses placeholder VectorStore API

   • Fix: Use search_l1() instead of search()
   • Status: TRIVIAL (rename methods)

2. OpenSearchClient::lexical_search is private

   • Fix: Remove async fn, change to pub async fn
   • Status: TRIVIAL (add pub)

3. Embedding type mismatch in line 89

   • Fix: Use correct return type from embeddings crate
   • Status: TRIVIAL (type annotation)

Estimated Fix Time

15-20 minutes to update API calls and make methods public.

No Design Changes Needed

All architectural decisions are sound:

• ✅ QueryOptimizer (6-stage pipeline) — No API dependency
• ✅ RRF Fusion algorithm — No API dependency
• ✅ Search strategy routing — No API dependency
• ⚠️ HybridQueryWorker — Needs VectorStore + OpenSearchClient API fixes

---

Phase 2 Checklist (1 Week)

Day 1-2: Fix Compilation

• Update hybrid_query_worker.rs to use actual VectorStore API
• Make OpenSearchClient::lexical_search public
• Fix type mismatches in embedding handling
• cargo check passes without errors

Day 3-4: Integration

• Update /memory/query handler to attempt hybrid search
• Add fallback strategy (hybrid → semantic → error)
• Forward JWT token through query pipeline
• Update response format to include metrics + score breakdown

Day 5: Testing

• Write 10+ integration tests
• Test fallback scenarios (OpenSearch unavailable)
• Measure latency (hybrid vs semantic)
• Verify score breakdown accuracy

Day 6-7: Buffer + Deployment

• Performance profiling
• Documentation updates
• Deploy to staging
• Manual E2E testing

---

Reference: Actual API Signatures

VectorStore (from mem-store)

pub async fn search_l1(
    &self,
    project: &str,
    embedding: &[f32],
    limit: i64,
) -> Result<Vec<SearchResult>>;

pub struct SearchResult {
    pub id: String,
    pub score: f32,
    pub item: ChunkWithMetadata,
}

pub struct ChunkWithMetadata {
    pub id: String,
    pub content: String,
    pub source: String,
    pub level: Option<String>,
    pub breadcrumb: Option<Vec<String>>,
}

EmbeddingsClient (from mem-llm)

pub async fn embed(&self, text: &str) -> Result<Vec<f32>>;

// Returns: 384-dim (all-MiniLM) or 1536-dim (OpenAI)

OpenSearchClient (current, needs pub)

pub async fn lexical_search(
    &self,
    query: &str,
    limit: usize,
    jwt_token: &str,
) -> Result<Vec<(String, f32, String, String, Vec<String>)>>;
//                (id,    score, chunk, source, breadcrumb)

---

Code Diff Preview (Phase 2)

Fix 1: Make OpenSearch method public

- async fn lexical_search(
+ pub async fn lexical_search(

Fix 2: Update HybridQueryWorker to use real API

- let results = self.vector_store.search(&embedding, project, limit, None).await?;
+ let results = self.vector_store.search_l1(project, embedding, limit).await?;

- let scored: Vec<(String, f32)> = results
-     .into_iter()
-     .map(|(id, score, _)| (id, score))
-     .collect();

+ let scored: Vec<(String, f32)> = results
+     .into_iter()
+     .map(|r| (r.id, r.score))
+     .collect();

Fix 3: Update http_server.rs to use HybridQueryWorker

- match state.query_worker.query(&project, &question, Some(limit)).await {
+ // Try hybrid first
+ match state.hybrid_query_worker.query(
+     &project,
+     &question,
+     limit,
+     &jwt_token,
+ ).await {
      Ok(response) => return HttpResponse::Ok().json(response),
+     Err(_) => {
+         // Fallback to semantic
      }
+ }
+ 
+ match state.query_worker.query(...).await {

---

Design Validation

✅ All design decisions validated:

Component	Status	Notes
Query Optimizer (6-stage)	✅ READY	No API dependency
RRF Algorithm	✅ READY	No API dependency
Query Routing	✅ READY	Pure logic
Cascading Strategy	✅ READY	Uses existing APIs
Hybrid Strategy	⚠️ STUB	Needs VectorStore API fix
Fallback Pattern	✅ READY	Uses existing query_worker

None of these require architectural changes.

---

Next: Week 2 Task

"Implement Phase 2: Hybrid Integration"

Time estimate: 3-4 days (15-20 min for compilation fixes + 2-3 days for integration + testing)

Deliverables:

1. ✅ /memory/query now attempts hybrid search
2. ✅ Score breakdown + metrics in response
3. ✅ Fallback to semantic if OpenSearch unavailable
4. ✅ 10+ integration tests
5. ✅ Latency measurements (hybrid vs semantic)

---