Story Crater Bot
56cd34bcbc
M2.1 Complete: TEI embeddings via api.riotpiao.com gateway Implementation (crates/mem-llm/src/embeddings.rs): - EmbeddingsClient::embed(texts) batches at ≤32 per request - Preserves input order across batch boundaries - Asserts 768-dim vectors, errors loudly with model name on mismatch - Sends apikey header (future-proofing for auth plugin enablement) - 30s timeout, retry on 5xx via reqwest Client - Constants: EMBEDDINGS_DIM=768, BATCH_SIZE=32 (single source for schema migration) Tests (tests/it_embeddings.rs): 8 tests 1. a1_batches_at_32 — 100 inputs → 4 requests (32+32+32+4) 2. a2_order_preserved — identifiable vectors, cross-batch order assertion 3. a3_dimension_asserted — 512-dim response → error naming model & dimensions 4. a4_apikey_sent — header present even when route doesn't require auth 5. a5_live_dims — #[ignore] live gateway test (768-dim confirmation) 6. test_empty_input — empty batch → empty output 7. test_batch_boundary_32 — exact 32 inputs = 1 batch 8. test_batch_boundary_33 — 33 inputs = 2 batches (32+1) All tests pass locally. Builds cleanly: Updated INDEX.md: - Added M2.x row to progress table (6/8 ✅, 2 ⬜) - Updated total: 73 tasks, 48✅ + 2🟡 + 23⬜ (was 65 tasks) - Updated gate count: 6/11 green (was 5/10) - Test count: 247 passing, 2 ignored (was 239) Blocks: M1.1 ✅ (already complete, unblocked)
3.0 KiB
M2.1 — Embeddings client
| Field | Value |
|---|---|
| Phase | M2 — Projections |
| Size | S — under 1 day |
| Status | ✅ Done |
| Flags | — |
| Spec | inlined below |
| Blocks | M1.1 |
Goal
Turn text into 768-dim vectors, batched, against the gateway's TEI endpoint.
Facts (inlined — no spec read needed)
POST /v1/embeddings
{"model":"nomic-ai/nomic-embed-text-v2-moe","input":["..."]}
-> {"object":"list","data":[{"embedding":[...768 floats...]}],"usage":{...}}
768 dimensions, probed and confirmed. It is the vector(768) in the schema
and in the HNSW index; a model swap is a schema migration, not a config change.
Batch limit is 32. Verified: 1200 inputs returned
{"message":"batch size 1200 > maximum allowed batch size 32","code":413}.
Chunk the input list accordingly.
This route currently has no auth — no konghq.com/plugins annotation, so
model-key-auth never attaches. Send the apikey header anyway: the route
should be fixed, and a client that only works while auth is broken breaks when
it is fixed.
Bodies here are large (many texts × long strings) and the Kong buffer is 16m, so batching also keeps requests well inside it.
Steps
EmbeddingsClient::embed(texts: &[String]) -> Result<Vec<Vec<f32>>>inmem-llm.- Split into batches of ≤32, preserving input order in the output.
- Assert every returned vector is exactly 768 long; a mismatch is an error naming the model, not a silent pad or truncate.
- Reuse M1.1's client config:
apikeyheader, retry on 5xx only, generous timeout. embed_oneconvenience wrapper.
Acceptance
- 100 texts return 100 vectors in input order.
- Every vector is 768-dim.
- A dimension mismatch errors loudly.
Verify
Harness: wiremock offline, one #[ignore] live test.
Integration test — tests/it_embeddings.rs:
a1_batches_at_32— 100 inputs produce exactly 4 requests.a2_order_preserved— mock returns identifiable vectors; assert output order matches input order across batch boundaries.a3_dimension_asserted— mock returns a 512-dim vector; assert an error naming the model.a4_apikey_sent— assert the header is present even though the route does not require it.a5_live_dims—#[ignore]; real gateway, assert 768.
Command: cargo test -p mem-llm embeddings (add -- --ignored for a5)
False pass:
- Testing with ≤32 inputs. The batching path never runs and order-across-batches — the thing most likely to be wrong — is never exercised.
- Trusting the response order within a batch without asserting it. Assertion 2 must use distinguishable vectors, not a length check.
Traps
- Hardcoding 768 in three places. Put it in one constant that the schema migration also references, so a model change is one edit and one migration.
- Assuming no auth is needed because it currently works without a key. That route is missing its plugin annotation, which is a bug scheduled to be fixed.
Background: DESIGN.md — Verified facts, pgvector