poimen-memory/tasks/M2.1-embeddings-client.md

M2.1 — Embeddings client

Field	Value
Phase	M2 — Projections
Size	S — under 1 day
Status	⬜ Not started
Flags	—
Spec	inlined below
Blocks	M1.1

Goal

Turn text into 768-dim vectors, batched, against the gateway's TEI endpoint.

Facts (inlined — no spec read needed)

POST /v1/embeddings
{"model":"nomic-ai/nomic-embed-text-v2-moe","input":["..."]}
-> {"object":"list","data":[{"embedding":[...768 floats...]}],"usage":{...}}

768 dimensions, probed and confirmed. It is the vector(768) in the schema and in the HNSW index; a model swap is a schema migration, not a config change.

Batch limit is 32. Verified: 1200 inputs returned {"message":"batch size 1200 > maximum allowed batch size 32","code":413}. Chunk the input list accordingly.

This route currently has no auth — no konghq.com/plugins annotation, so model-key-auth never attaches. Send the apikey header anyway: the route should be fixed, and a client that only works while auth is broken breaks when it is fixed.

Bodies here are large (many texts × long strings) and the Kong buffer is 16m, so batching also keeps requests well inside it.

Steps

1. EmbeddingsClient::embed(texts: &[String]) -> Result<Vec<Vec<f32>>> in mem-llm.
2. Split into batches of ≤32, preserving input order in the output.
3. Assert every returned vector is exactly 768 long; a mismatch is an error naming the model, not a silent pad or truncate.
4. Reuse M1.1's client config: apikey header, retry on 5xx only, generous timeout.
5. embed_one convenience wrapper.

Acceptance

• 100 texts return 100 vectors in input order.
• Every vector is 768-dim.
• A dimension mismatch errors loudly.

Verify

Harness: wiremock offline, one #[ignore] live test.

Integration test — tests/it_embeddings.rs:

1. a1_batches_at_32 — 100 inputs produce exactly 4 requests.
2. a2_order_preserved — mock returns identifiable vectors; assert output order matches input order across batch boundaries.
3. a3_dimension_asserted — mock returns a 512-dim vector; assert an error naming the model.
4. a4_apikey_sent — assert the header is present even though the route does not require it.
5. a5_live_dims — #[ignore]; real gateway, assert 768.

Command: cargo test -p mem-llm embeddings (add -- --ignored for a5)

False pass:

• Testing with ≤32 inputs. The batching path never runs and order-across-batches — the thing most likely to be wrong — is never exercised.
• Trusting the response order within a batch without asserting it. Assertion 2 must use distinguishable vectors, not a length check.

Traps

• Hardcoding 768 in three places. Put it in one constant that the schema migration also references, so a model change is one edit and one migration.
• Assuming no auth is needed because it currently works without a key. That route is missing its plugin annotation, which is a bug scheduled to be fixed.

---

Background: DESIGN.md — Verified facts, pgvector