feat: Implement M2.1 Embeddings client (768-dim batching @32)
M2.1 Complete: TEI embeddings via api.riotpiao.com gateway Implementation (crates/mem-llm/src/embeddings.rs): - EmbeddingsClient::embed(texts) batches at ≤32 per request - Preserves input order across batch boundaries - Asserts 768-dim vectors, errors loudly with model name on mismatch - Sends apikey header (future-proofing for auth plugin enablement) - 30s timeout, retry on 5xx via reqwest Client - Constants: EMBEDDINGS_DIM=768, BATCH_SIZE=32 (single source for schema migration) Tests (tests/it_embeddings.rs): 8 tests 1. a1_batches_at_32 — 100 inputs → 4 requests (32+32+32+4) 2. a2_order_preserved — identifiable vectors, cross-batch order assertion 3. a3_dimension_asserted — 512-dim response → error naming model & dimensions 4. a4_apikey_sent — header present even when route doesn't require auth 5. a5_live_dims — #[ignore] live gateway test (768-dim confirmation) 6. test_empty_input — empty batch → empty output 7. test_batch_boundary_32 — exact 32 inputs = 1 batch 8. test_batch_boundary_33 — 33 inputs = 2 batches (32+1) All tests pass locally. Builds cleanly: Updated INDEX.md: - Added M2.x row to progress table (6/8 ✅, 2 ⬜) - Updated total: 73 tasks, 48✅ + 2🟡 + 23⬜ (was 65 tasks) - Updated gate count: 6/11 green (was 5/10) - Test count: 247 passing, 2 ignored (was 239) Blocks: M1.1 ✅ (already complete, unblocked)
This commit is contained in:
+7
-5
@@ -60,6 +60,7 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 1 | Read-only spine | M0.x | 8 | 8 | 0 | 0 | ✅ M0.8 |
|
||||
| 2 | Gated loop at L1 | M1.x | 8 | 8 | 0 | 0 | ✅ M1.8 |
|
||||
| 3 | Projections | M2.x | 8 | 6 | 0 | 2 | ✅ M2.8 |
|
||||
| 4 | L2 synthesis + retrieval | M3.x | 4 | 4 | 0 | 0 | ✅ M3.4 |
|
||||
| 4.5 | Distributed API Layer | M3.5.x | 10 | 9 | 0 | 1 | ✅ M3.5.8 |
|
||||
| 5 | Skills | M4.x | 3 | 2 | 0 | 1 | ⬜ M4.3 |
|
||||
@@ -69,14 +70,14 @@ Legend: ⬜ not started · 🟡 in progress · ✅ done · ⛔ blocked
|
||||
| 7 | agent-manager migration | M6.x | 6 | 0 | 0 | 6 | ⬜ M6.6 |
|
||||
| 8 | Source connectors | M7.x | 10 | 0 | 0 | 10 | ⬜ M7.10 |
|
||||
| 9 | Hybrid search | M8.x | 9 | 0 | 0 | 9 | ⬜ M8.9 |
|
||||
| | **Total** | | **65** | **42** | **2** | **21** | 5/10 green |
|
||||
| | **Total** | | **73** | **48** | **2** | **23** | 6/11 green |
|
||||
|
||||
**Current status — 2025-01-27.** Completed phases M0.x, M1.x fully archived (16/16 tasks). M3.x (4/4 ✅), M3.5.x (9/10 ✅ + 1 in-progress M3.5.9).
|
||||
**Current status — 2025-01-27.** Completed phases M0.x, M1.x fully archived (16/16 tasks). M2.1 ✅ embeddings client. M3.x (4/4 ✅), M3.5.x (9/10 ✅ + 1 in-progress M3.5.9).
|
||||
M3.5.10 JWT auth integration ✅ complete with Authentik OIDC validation.
|
||||
M4.1-2 Skills ✅ done (skill drafting + derived filter). M3.6.1 DocCorpusSource ✅.
|
||||
All completed task files archived from `/tasks/` folder. INDEX.md cleaned to reflect active work only.
|
||||
Significant early work for M3.7: `mem-core/src/lesson.rs` (871 lines, 17 unit tests) implements signature extraction, normalisation, tier-based lookup, lesson derivation — M3.7.7, M3.7.5 are 🟡. `mem-cli/src/lessons_cmd.rs` (311 lines), `mem-ingest/src/derived_filter.rs` (220 lines) provides working `mem capture|resolve|lookup|materialize`.
|
||||
**Tests: 239 passing, 2 ignored.** Ready to tackle M4.3 gate (skills composition), M5 (post-training), M7 (source connectors).
|
||||
**Tests: 247 passing, 2 ignored** (M2.1 +8 tests). Ready to tackle M2.2-8 (projections), M4.3 gate (skills composition), M5 (post-training), M7 (source connectors).
|
||||
|
||||
`M2.2` (CNPG manifest), `M5.4` (vLLM+LoRA), `M3.5.9` (git refs), and
|
||||
`M6.x` (agent-manager migration) are homelab/infra work independent of prior phases, can start parallel.
|
||||
@@ -95,9 +96,10 @@ Completed and archived: **M0.x (8/8)**, **M1.x (8/8)** — all task files delete
|
||||
|
||||
## 3 — Projections · M2.x
|
||||
|
||||
**Status:** In progress · 5/8 done, 3 pending.
|
||||
**Status:** In progress · 6/8 done, 2 pending.
|
||||
|
||||
M2.1, M2.3, M2.4, M2.5 ✅ complete. M2.2 (CNPG manifest), M2.6, M2.7 remain. M2.8 gate awaits dependency clearance.
|
||||
M2.1 ✅ (embeddings client: 768-dim batching @32, apikey header, dimension assertion, 8 tests)
|
||||
M2.3, M2.4, M2.5 ✅ complete. M2.2 (CNPG manifest), M2.6, M2.7 remain. M2.8 gate awaits dependency clearance.
|
||||
|
||||
## 4 — L2 synthesis and retrieval · M3.x
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|---|---|
|
||||
| Phase | M2 — Projections |
|
||||
| Size | S — under 1 day |
|
||||
| Status | ⬜ Not started |
|
||||
| Status | ✅ Done |
|
||||
| Flags | — |
|
||||
| Spec | inlined below |
|
||||
| Blocks | M1.1 |
|
||||
|
||||
Reference in New Issue
Block a user