plan: add M3.6.7 contextual enrichment + M3.6.8 deduplication

2026-08-28 09:28:10 -07:00
parent c20f8f9a9f
commit 1f9b30b1ec
2 changed files with 94 additions and 0 deletions
@@ -0,0 +1,42 @@
+# M3.6.7 — Contextual Enrichment at Ingest
+
+| Field | Value |
+|---|---|
+| Phase | M3.6 — Reference corpora |
+| Size | M — 1–2 days |
+| Status | ⬜ Not started |
+| Depends | M3.6.1 (DocCorpusSource) |
+| Blocks | — |
+
+## Goal
+
+At ingest time, prepend each chunk with its context in the document hierarchy.
+This improves semantic search because queries using different terminology can
+still find relevant chunks.
+
+Inspired by Anthropic's Contextual Retrieval paper.
+
+## Example
+
+```
+BEFORE (raw chunk from heading "npm install"):
+  "Use 'npm ci' instead of 'npm install' for reproducible builds"
+
+AFTER (contextualized):
+  "From the Node.js Dependency Management guide, section npm install:
+   Use 'npm ci' instead of 'npm install' for reproducible builds"
+```
+
+The context gets embedded alongside the chunk's text, improving vector search.
+
+## Deliverables
+
+- `DocCorpusSource` enhanced to include breadcrumb path + section summary
+- Chunk rendering includes context header (auto-generated or manual)
+- Embedding happens on (context + chunk), not just chunk
+- Rebuild idempotence preserved
+
+## Tests
+
+- 5 unit tests (context generation, formatting, idempotence)
+- 3 integration tests (rebuild with enrichment, search improvement)