poimen-memory/docs/IMPLEMENTATION_NOTES.md

# Implementation Notes: Query Optimization Engine

## Status

✅ **Design Phase COMPLETE**
- Query Optimizer (query_optimizer.rs) — READY
- Hybrid Query Worker (hybrid_query_worker.rs) — STUB (needs API integration)
- Design Documentation (QUERY_OPTIMIZATION_ENGINE.md) — COMPLETE

⚠️ **API Integration Notes** (for Phase 2)

---

## VectorStore API Corrections

### Current Methods (Confirmed)

The actual `VectorStore` has level-based search methods:

```rust
// NOT available:
vector_store.search(&embedding, project, limit, None)
vector_store.search_with_ids(&embedding, project, limit, chunk_ids)

// ACTUALLY available:
vector_store.search_l1(project, &embedding, limit)
vector_store.search_l2(project, &embedding, limit)
vector_store.search_l3(project, &embedding, limit)
```

### Updated Semantic Retrieval

```rust
async fn retrieve_semantic(
    &self,
    project: &str,
    query_ctx: &QueryContext,
    limit: i64,
) -> Result<Vec<(String, f32)>> {
    let embedding = query_ctx
        .embedding
        .as_ref()
        .ok_or_else(|| anyhow::anyhow!("no embedding"))?;

    // Use L1 (most specific level)
    let results = self.vector_store.search_l1(project, embedding, limit).await?;

    // Convert to (id, score) tuples
    let scored: Vec<(String, f32)> = results
        .into_iter()
        .map(|r| (r.id, r.score))
        .collect();

    Ok(scored)
}
```

---

## OpenSearchClient API Corrections

### Issue: Method Visibility

The `lexical_search` method in `opensearch_client.rs` is private:

```rust
// NOT public (private):
async fn lexical_search(...)

// NEEDED:
pub async fn lexical_search(...)
```

### Fix

Make the method public:

```rust
// In crates/mem-cli/src/opensearch_client.rs
pub async fn lexical_search(
    &self,
    query: &str,
    limit: usize,
    jwt_token: &str,
) -> Result<Vec<(String, f32, String, String, Vec<String>)>>
```

---

## Simplified Phase 2 Implementation

For Phase 2, instead of modifying http_server.rs extensively, create a wrapper:

```rust
/// In crates/mem-cli/src/http_server.rs

async fn query_handler(...) -> HttpResponse {
    let (claims, token) = match validate_auth(&req, &state).await {
        Ok(c) => c,
        Err(e) => return e,
    };

    // ... existing checks ...

    // Try hybrid if available
    #[cfg(feature = "hybrid_search")]
    {
        match state.hybrid_query_worker.query(
            &project,
            &question,
            limit,
            &token,
        ).await {
            Ok(response) => return HttpResponse::Ok().json(response),
            Err(e) => {
                tracing::warn!("Hybrid query failed: {}, falling back", e);
            }
        }
    }

    // Fallback: existing semantic search
    match state.query_worker.query(&project, &question, Some(limit)).await {
        Ok(results) => HttpResponse::Ok().json(json!({
            "query": question,
            "project": project,
            "results": results,
            "method": "semantic_only"  // Indicate fallback
        })),
        Err(e) => {
            tracing::error!("Query failed: {}", e);
            HttpResponse::InternalServerError().json(json!({"error": "query_failed"}))
        }
    }
}
```

---

## Build Status

### Current Issues (Non-Blocking Design)

1. **hybrid_query_worker.rs** uses placeholder VectorStore API
   - Fix: Use `search_l1()` instead of `search()`
   - Status: TRIVIAL (rename methods)

2. **OpenSearchClient::lexical_search** is private
   - Fix: Remove `async fn`, change to `pub async fn`
   - Status: TRIVIAL (add `pub`)

3. **Embedding type mismatch** in line 89
   - Fix: Use correct return type from embeddings crate
   - Status: TRIVIAL (type annotation)

### Estimated Fix Time

**15-20 minutes** to update API calls and make methods public.

### No Design Changes Needed

All architectural decisions are sound:
- ✅ QueryOptimizer (6-stage pipeline) — No API dependency
- ✅ RRF Fusion algorithm — No API dependency
- ✅ Search strategy routing — No API dependency
- ⚠️ HybridQueryWorker — Needs VectorStore + OpenSearchClient API fixes

---

## Phase 2 Checklist (1 Week)

### Day 1-2: Fix Compilation

- [ ] Update `hybrid_query_worker.rs` to use actual VectorStore API
- [ ] Make `OpenSearchClient::lexical_search` public
- [ ] Fix type mismatches in embedding handling
- [ ] `cargo check` passes without errors

### Day 3-4: Integration

- [ ] Update `/memory/query` handler to attempt hybrid search
- [ ] Add fallback strategy (hybrid → semantic → error)
- [ ] Forward JWT token through query pipeline
- [ ] Update response format to include metrics + score breakdown

### Day 5: Testing

- [ ] Write 10+ integration tests
- [ ] Test fallback scenarios (OpenSearch unavailable)
- [ ] Measure latency (hybrid vs semantic)
- [ ] Verify score breakdown accuracy

### Day 6-7: Buffer + Deployment

- [ ] Performance profiling
- [ ] Documentation updates
- [ ] Deploy to staging
- [ ] Manual E2E testing

---

## Reference: Actual API Signatures

### VectorStore (from mem-store)

```rust
pub async fn search_l1(
    &self,
    project: &str,
    embedding: &[f32],
    limit: i64,
) -> Result<Vec<SearchResult>>;

pub struct SearchResult {
    pub id: String,
    pub score: f32,
    pub item: ChunkWithMetadata,
}

pub struct ChunkWithMetadata {
    pub id: String,
    pub content: String,
    pub source: String,
    pub level: Option<String>,
    pub breadcrumb: Option<Vec<String>>,
}
```

### EmbeddingsClient (from mem-llm)

```rust
pub async fn embed(&self, text: &str) -> Result<Vec<f32>>;

// Returns: 384-dim (all-MiniLM) or 1536-dim (OpenAI)
```

### OpenSearchClient (current, needs pub)

```rust
pub async fn lexical_search(
    &self,
    query: &str,
    limit: usize,
    jwt_token: &str,
) -> Result<Vec<(String, f32, String, String, Vec<String>)>>;
//                (id,    score, chunk, source, breadcrumb)
```

---

## Code Diff Preview (Phase 2)

### Fix 1: Make OpenSearch method public

```diff
- async fn lexical_search(
+ pub async fn lexical_search(
```

### Fix 2: Update HybridQueryWorker to use real API

```diff
- let results = self.vector_store.search(&embedding, project, limit, None).await?;
+ let results = self.vector_store.search_l1(project, embedding, limit).await?;

- let scored: Vec<(String, f32)> = results
-     .into_iter()
-     .map(|(id, score, _)| (id, score))
-     .collect();

+ let scored: Vec<(String, f32)> = results
+     .into_iter()
+     .map(|r| (r.id, r.score))
+     .collect();
```

### Fix 3: Update http_server.rs to use HybridQueryWorker

```diff
- match state.query_worker.query(&project, &question, Some(limit)).await {
+ // Try hybrid first
+ match state.hybrid_query_worker.query(
+     &project,
+     &question,
+     limit,
+     &jwt_token,
+ ).await {
      Ok(response) => return HttpResponse::Ok().json(response),
+     Err(_) => {
+         // Fallback to semantic
      }
+ }
+ 
+ match state.query_worker.query(...).await {
```

---

## Design Validation

✅ **All design decisions validated:**

| Component | Status | Notes |
|-----------|--------|-------|
| Query Optimizer (6-stage) | ✅ READY | No API dependency |
| RRF Algorithm | ✅ READY | No API dependency |
| Query Routing | ✅ READY | Pure logic |
| Cascading Strategy | ✅ READY | Uses existing APIs |
| Hybrid Strategy | ⚠️ STUB | Needs VectorStore API fix |
| Fallback Pattern | ✅ READY | Uses existing query_worker |

**None of these require architectural changes.**

---

## Next: Week 2 Task

**"Implement Phase 2: Hybrid Integration"**

**Time estimate:** 3-4 days (15-20 min for compilation fixes + 2-3 days for integration + testing)

**Deliverables:**
1. ✅ `/memory/query` now attempts hybrid search
2. ✅ Score breakdown + metrics in response
3. ✅ Fallback to semantic if OpenSearch unavailable
4. ✅ 10+ integration tests
5. ✅ Latency measurements (hybrid vs semantic)

---