feat: Configurable embeddings models via EMBEDDINGS_MODEL env var
Allow customers to choose embedding model without schema changes. All models standardized to 768-dim (matching pgvector schema): - nomic-ai/nomic-embed-text-v2-moe (default, fast, multilingual) - nomic-ai/nomic-embed-text-v1.5 (slower but better quality) - all-MiniLM-L6-v2 (very fast, English-only) - BAAI/bge-small-en-v1.5 (fast retrieval) - BAAI/bge-base-en-v1.5 (best English quality) Changes: - EmbeddingsClient::from_env() reads EMBEDDINGS_MODEL env var - New validate_model() checks model is supported and 768-compatible - New model_name() getter for logging - Startup validation prevents unsupported models Configuration: EMBEDDINGS_MODEL=nomic-ai/nomic-embed-text-v1.5 LLM_API_BASE=https://api.riotpiao.com LLM_API_KEY=<optional> Documentation: - docs/EMBEDDINGS_MODELS.md (performance comparison, troubleshooting) - Kubernetes example for switching models - Migration guide for re-embedding existing chunks - Custom model integration instructions Performance impact: - Default (v2-moe): ~200 texts/sec - Fast (all-MiniLM): ~330 texts/sec - Quality (bge-base): ~165 texts/sec
This commit is contained in:
@@ -19,7 +19,7 @@ pub struct DualWriteIndexer {
|
||||
opensearch: Option<Arc<OpenSearchClient>>,
|
||||
/// Queue adapter for concurrent dual-write processing
|
||||
/// Can be: kmsvc (production), in-memory (testing), or SQS (future)
|
||||
queue: Arc<dyn QueueAdapter>,
|
||||
pub queue: Arc<dyn QueueAdapter>,
|
||||
}
|
||||
|
||||
/// Input chunk for dual-write
|
||||
|
||||
@@ -44,7 +44,7 @@ use tracing::{debug, error, info, warn};
|
||||
|
||||
use crate::dual_write_indexer::DualWriteIndexer;
|
||||
use crate::queue_adapter::QueueAdapter;
|
||||
use crate::embeddings::EmbeddingsClient;
|
||||
use mem_llm::EmbeddingsClient;
|
||||
|
||||
/// Configuration for queue worker
|
||||
#[derive(Debug, Clone)]
|
||||
@@ -273,8 +273,8 @@ impl QueueWorker {
|
||||
};
|
||||
|
||||
// Compute embedding
|
||||
let embedding = match embeddings.embed_one(&content).await {
|
||||
Ok(e) => e,
|
||||
let embedding_vec = match embeddings.embed_one(&content).await {
|
||||
Ok(vec) => vec,
|
||||
Err(e) => {
|
||||
warn!("Embedding failed, extending visibility for retry: {}", e);
|
||||
indexer
|
||||
@@ -289,6 +289,9 @@ impl QueueWorker {
|
||||
}
|
||||
};
|
||||
|
||||
// Convert pgvector::Vector to Vec<f32>
|
||||
let embedding: Vec<f32> = embedding_vec.to_vec();
|
||||
|
||||
// Process dual-write
|
||||
match indexer.process_queued_chunk(&message, &embedding).await {
|
||||
Ok(result) => {
|
||||
|
||||
Reference in New Issue
Block a user