poimen-memory/tasks/M3.6.7-contextual-enrichment.md

M3.6.7 — Contextual Enrichment at Ingest

Field	Value
Phase	M3.6 — Reference corpora
Size	M — 1–2 days
Status	⬜ Not started
Depends	M3.6.1 (DocCorpusSource)
Blocks	—

Goal

At ingest time, prepend each chunk with its context in the document hierarchy. This improves semantic search because queries using different terminology can still find relevant chunks.

Inspired by Anthropic's Contextual Retrieval paper.

Example

BEFORE (raw chunk from heading "npm install"):
  "Use 'npm ci' instead of 'npm install' for reproducible builds"

AFTER (contextualized):
  "From the Node.js Dependency Management guide, section npm install:
   Use 'npm ci' instead of 'npm install' for reproducible builds"

The context gets embedded alongside the chunk's text, improving vector search.

Deliverables

• DocCorpusSource enhanced to include breadcrumb path + section summary
• Chunk rendering includes context header (auto-generated or manual)
• Embedding happens on (context + chunk), not just chunk
• Rebuild idempotence preserved

Tests

• 5 unit tests (context generation, formatting, idempotence)
• 3 integration tests (rebuild with enrichment, search improvement)