From 939c1436a28035fad083babb4e3d218eed14d3e0 Mon Sep 17 00:00:00 2001
From: rock <locartrock@gmail.com>
Date: Sat, 5 Sep 2026 01:14:32 -0700
Subject: [PATCH] Phase 7: Temporal-RAGA-Ingest Architecture Design (Complete)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

📋 DESIGN DOCUMENT (18.7 KB)

Architecture:
  ├─ Temporal-aware knowledge graph (versioning)
  ├─ RAGA ingest pipeline (Retrieval-Augmented Graph Architecture)
  ├─ Chunk editing with immutable audit trail
  └─ Multi-signal ranking (4 signals, 25% each)

Key Sections:

1. Chunk Editing Semantics (Immutable versions)
   ├─ chunk_versions table (version 1, 2, 3...)
   ├─ is_current flag (which version is active)
   ├─ edited_by, edit_reason, confidence tracking
   └─ Example: Kubernetes entity v1 → v2 (added CNCF affiliation)

2. Ranking Formula (4 Equal Signals)
   ├─ Signal 1: Confidence (LLM extraction, 0.0-1.0)
   ├─ Signal 2: Recency (exponential decay, τ=30d)
   ├─ Signal 3: Community (PageRank + in-degree)
   ├─ Signal 4: BM25 (lexical relevance, normalized)
   └─ final_score = 0.25*conf + 0.25*recency + 0.25*community + 0.25*bm25

3. Audit Trail (Append-only immutable log)
   ├─ audit_events table (partitioned by timestamp)
   ├─ Every mutation logged: chunk_edited, created, verified, deleted
   ├─ Cryptographic signing (SHA256 for tamper detection)
   ├─ Queryable: Who changed what, when, why
   └─ Archive: Daily batch to S3 cold storage

4. Schema Extensions
   ├─ chunk_versions: id, chunk_id, version, content, confidence, is_current
   ├─ audit_events: id, timestamp, event_type, actor, resource_id, action, reason
   ├─ ranking_signals: id, entity_id, signal_type, signal_value
   └─ query_rankings: query_id, chunk_id, rank, final_score, signal_breakdown

5. Deterministic Rebuild (Parity Check - M2.8 extended)
   ├─ Snapshot current state
   ├─ Replay audit events in order
   ├─ Recompute all signals
   ├─ Verify: checksum_before == checksum_after
   └─ Detects corruption in O(1) time

6. Metrics Emission & Prometheus Scraping
   ├─ GET /metrics endpoint (Authentik protected)
   ├─ Real-time Prometheus format (OpenMetrics)
   ├─ Prometheus scrapes every 15s
   ├─ Grafana dashboard tracks Phase 7 SLOs
   └─ Alerts for M7.1-M7.5 gates

Phase 7 Metrics (Prometheus):
  ├─ memory_chunk_edits_total (counter: create/update/delete)
  ├─ memory_edit_latency_seconds (histogram: P50/P99)
  ├─ memory_audit_events_total (counter: by event_type)
  ├─ memory_audit_signature_failures_total (counter: must be 0)
  ├─ memory_rebuild_checksum_matches_total (counter: parity checks)
  ├─ memory_ranking_ndcg_weighted (gauge: weighted accuracy)
  ├─ memory_storage_overhead_ratio (gauge: 1.5x max)
  ├─ memory_confidence_distribution (histogram: score buckets)
  ├─ memory_recency_score_* (gauge: avg/p50/p99)
  └─ memory_community_score_* (gauge: avg/p50/p99)

SLO Alerts (Prometheus Rules):
  ├─ M7.1_RebuildParityCheckFailed (critical)
  ├─ M7_2_AuditSignatureFailure (critical)
  ├─ M7_3_RankingAccuracyDegraded (warning: NDCG < 0.88)
  ├─ M7_4_EditLatencyHigh (warning: P99 > 2s)
  └─ M7_5_StorageOverheadHigh (warning: ratio > 1.5x)

Implementation Roadmap:
  ├─ Phase 7.1: Schema & Migrations (Week 1, ~200 LOC)
  ├─ Phase 7.2: Versioning API (Week 2, ~400 LOC, 50+ tests)
  ├─ Phase 7.3: Audit Trail (Week 2, ~300 LOC, 30+ tests)
  ├─ Phase 7.4: Multi-Signal Ranking (Week 3, ~350 LOC, 40+ tests)
  ├─ Phase 7.5: Deterministic Rebuild (Week 3, ~200 LOC, 20+ tests)
  └─ Phase 7.6: Documentation & SLOs (Week 4, ~500 LOC docs)

Success Criteria:
  ✅ All 5 composition gates pass (M7.1-M7.5)
  ✅ 150+ tests (unit + integration)
  ✅ NDCG@10 weighted >= 0.88 (M7.3)
  ✅ Edit latency P99 < 2s (M7.4)
  ✅ Storage overhead <= 1.5x (M7.5)
  ✅ Audit trail 100% immutable (M7.2)
  ✅ Rebuild parity 100% (M7.1)
  ✅ Full documentation + runbooks

Key Design Decisions:
  ├─ Versioning: Immutable (Option A, not Option B soft deletes)
  ├─ Signals: 4 equal weights (25% each, not weighted differently)
  ├─ Audit: Append-only JSONL + S3 (not mutable log)
  ├─ Rebuild: Signature verification (O(1), not full replay)
  ├─ Confidence: From LLM pipeline (Phase 5)
  ├─ Recency: Exponential decay τ=30d (standard info theory)
  ├─ Community: PageRank + in-degree (graph-theoretic)
  └─ Edit latency: P99 < 2s (real-time UX)

Risks & Mitigations:
  ├─ Version explosion: Compression + archival + TTL cleanup
  ├─ Audit log query slowness: Partitioning + materialized views
  ├─ Signature false positives: Comprehensive testing + HSM backup
  ├─ Community signal staleness: Recompute PageRank daily
  └─ Concurrent edits: Optimistic locking via version number

Integration Points:
  ├─ Phase 4 (Retrieval) → Multi-signal ranking
  ├─ Phase 5 (Synthesis) → Confidence extraction
  ├─ Phase 6 (Agents) → Metrics emission
  └─ Phase 7 (Versioning) → Deterministic rebuild

References:
  ├─ Git model (immutable commits)
  ├─ Okapi BM25 + PageRank (arXiv:1802.05365)
  ├─ NIST SP 800-92 (audit logs)
  ├─ Riak parity checks (deterministic replay)
  └─ ISO 8601 (temporal semantics)

Next: Architecture review, then Phase 7.1 (migrations)
---
 docs/PHASE_7_TEMPORAL_RAGA_INGEST_DESIGN.md | 989 ++++++++++++++++++++
 1 file changed, 989 insertions(+)
 create mode 100644 docs/PHASE_7_TEMPORAL_RAGA_INGEST_DESIGN.md

diff --git a/docs/PHASE_7_TEMPORAL_RAGA_INGEST_DESIGN.md b/docs/PHASE_7_TEMPORAL_RAGA_INGEST_DESIGN.md
new file mode 100644
index 0000000..d06c941
--- /dev/null
+++ b/docs/PHASE_7_TEMPORAL_RAGA_INGEST_DESIGN.md
@@ -0,0 +1,989 @@
+# Phase 7: Temporal-RAGA-Ingest Architecture Design
+
+**Status**: 🔵 Design Phase (No code yet)
+
+---
+
+## 1. Executive Summary
+
+**Phase 7** extends Poimen Memory with:
+1. **Temporal-Aware Knowledge Graph** (time-series semantic versioning)
+2. **RAGA Ingest Pipeline** (Retrieval-Augmented Graph Architecture)
+3. **Chunk Editing & Versioning** (immutable append-only audit trail)
+4. **Multi-Signal Ranking** (4 signals: confidence, recency, community, BM25)
+
+**Key Innovation**: Enable interactive **chunk editing** with full **temporal lineage** while maintaining **deterministic rebuilds** and **append-only audit logs**.
+
+---
+
+## 2. Architecture Overview
+
+### Current State (Phase 6)
+```
+┌─────────────────┐
+│ Input Content   │
+└────────┬────────┘
+         │
+         ↓
+┌─────────────────────────────────────┐
+│ Extraction Pipeline (Phase 3)        │
+│ ├─ Entity Linking (Phase 5.1)       │
+│ ├─ Inference (Phase 5.2)            │
+│ └─ Query Reasoning (Phase 5.3)      │
+└────────┬────────────────────────────┘
+         │
+         ↓
+┌─────────────────────────────────────┐
+│ Memory Storage (Phase 2)             │
+│ ├─ memory_entity (L1/L2)            │
+│ ├─ memory_edge (relationships)      │
+│ └─ memory_vector (embeddings)       │
+└────────┬────────────────────────────┘
+         │
+         ↓
+┌─────────────────────────────────────┐
+│ Query & Retrieval (Phase 4 + 6)     │
+│ ├─ Semantic search                  │
+│ ├─ Lexical search                   │
+│ └─ Hybrid ranking (RRF)             │
+└─────────────────────────────────────┘
+```
+
+### Phase 7 Extension
+```
+┌──────────────────────────────────────────────────────────────┐
+│ TEMPORAL-RAGA-INGEST ARCHITECTURE                            │
+└──────────────────────────────────────────────────────────────┘
+
+INPUT LAYER (Temporal-Aware):
+  ├─ Streaming ingest (Kafka)
+  ├─ Batch ingest (S3/Git)
+  ├─ Interactive edits (UI)
+  └─ Time-tagged: {content, timestamp, editor, reason}
+
+                    ↓
+
+EXTRACTION LAYER (LLM-Gated):
+  ├─ Entity extraction (with confidence)
+  ├─ Relationship inference
+  ├─ Temporal fact alignment (when/how long)
+  └─ Conflict detection (contradictions)
+
+                    ↓
+
+VERSIONING LAYER (Append-Only):
+  ├─ chunk_versions: {chunk_id, version, content, editor, timestamp}
+  ├─ entity_versions: {entity_id, version, properties, confidence}
+  ├─ edge_versions: {edge_id, version, source, relation, target}
+  └─ audit_trail: All mutations logged immutably
+
+                    ↓
+
+STORAGE LAYER (Multi-Signal Ranking):
+  ├─ memory_entity: entity + confidence + temporal_bounds
+  ├─ memory_edge: edge + recency_score + community_signal
+  ├─ memory_vector: embedding + ranking_signals
+  └─ chunk_metadata: version_history, edit_count, quality_score
+
+                    ↓
+
+QUERY LAYER (Context-Aware Retrieval):
+  ├─ Semantic search (Phase 4.1)
+  ├─ Lexical search (Phase 4.2)
+  ├─ Temporal filtering (when is this true?)
+  ├─ Community detection (Phase 4.3)
+  ├─ Multi-signal ranking (Phase 7 NEW)
+  └─ Deterministic tie-breaking (version + timestamp)
+```
+
+---
+
+## 3. Chunk Editing Semantics
+
+### Current: Read-Only
+```
+memory_entity (immutable)
+  ├─ id: UUID
+  ├─ name: String
+  ├─ properties: JSON (frozen)
+  └─ created_at: Timestamp
+```
+
+### Phase 7: Versioned + Editable
+
+#### Option A: Immutable Versions (Recommended)
+```sql
+-- Chunks are immutable; edits create new versions
+CREATE TABLE chunk_versions (
+  id UUID,
+  chunk_id UUID,              -- Stable reference
+  version INT,                -- 1, 2, 3...
+  content TEXT,
+  properties JSONB,
+  edited_by VARCHAR(255),
+  edit_reason TEXT,
+  confidence FLOAT,           -- 0.0-1.0
+  temporal_start TIMESTAMP,
+  temporal_end TIMESTAMP,
+  is_current BOOLEAN,
+  created_at TIMESTAMP,
+  PRIMARY KEY (chunk_id, version)
+);
+
+-- Active chunk always = is_current = true
+SELECT * FROM chunk_versions WHERE chunk_id = $1 AND is_current = true;
+
+-- Audit trail = historical versions
+SELECT * FROM chunk_versions WHERE chunk_id = $1 ORDER BY version;
+```
+
+**Example**:
+```
+chunk_id: ent-123 (Kubernetes entity)
+
+Version 1:
+  content: "Kubernetes: Container orchestrator by Google"
+  confidence: 0.85
+  edited_by: alice
+  created_at: 2025-01-30 10:00:00
+  is_current: false
+
+Version 2:
+  content: "Kubernetes: Container orchestrator by Google, open-source"
+  confidence: 0.92
+  edited_by: bob
+  created_at: 2025-01-31 14:30:00
+  is_current: true
+```
+
+#### Option B: Soft Deletes (Alternative)
+```sql
+CREATE TABLE chunk_versions (
+  id UUID,
+  chunk_id UUID,
+  version INT,
+  content TEXT,
+  is_deleted BOOLEAN,        -- soft delete
+  deleted_by VARCHAR(255),
+  PRIMARY KEY (chunk_id, version)
+);
+```
+
+**Verdict**: Use **Option A (Immutable)** — clearer lineage, no "tombstone" ambiguity.
+
+---
+
+## 4. Ranking Formula (4 Signals)
+
+### The 4 Signals
+
+#### Signal 1: Confidence (LLM Assessment)
+```
+confidence ∈ [0.0, 1.0]
+
+Source: LLM extraction pipeline (Phase 5.2)
+  ├─ 0.0-0.3: Uncertain (likely noise)
+  ├─ 0.3-0.6: Moderate confidence
+  ├─ 0.6-0.85: High confidence
+  └─ 0.85-1.0: Verified/human-edited
+
+Weight: 25%
+```
+
+#### Signal 2: Recency (Temporal Freshness)
+```
+recency_score = exp(-(now() - last_verified) / τ)
+  where τ = 30 days (half-life)
+
+Examples:
+  ├─ Last verified today: 1.00
+  ├─ Last verified 7 days ago: 0.78
+  ├─ Last verified 30 days ago: 0.50
+  ├─ Last verified 90 days ago: 0.12
+  └─ Last verified 365 days ago: 0.0001
+
+Weight: 25%
+```
+
+#### Signal 3: Community Signal (Graph Centrality)
+```
+community_score = pagerank_score * in_degree_normalized
+
+Where:
+  ├─ pagerank_score: Eigenvector centrality in entity graph
+  │  └─ High if many entities link to this entity
+  ├─ in_degree_normalized: (in_degree / max_in_degree) * 0.5
+  │  └─ Capped at 0.5 to avoid dominance
+  └─ Result: [0.0, 1.0]
+
+Examples:
+  ├─ "Kubernetes": 0.92 (highly connected)
+  ├─ "Docker": 0.87 (related to 15 entities)
+  └─ "ObscureLibrary": 0.15 (1-2 links only)
+
+Weight: 25%
+```
+
+#### Signal 4: Lexical Ranking (BM25 from OpenSearch)
+```
+bm25_score = OpenSearch BM25 normalized
+
+Source: Full-text index (existing Phase 4.2)
+  ├─ Raw BM25: [0, ∞)
+  ├─ Normalized: [0, 1.0]
+  │  └─ sigmoid(bm25 / 10.0) for smoothing
+  └─ Higher: More textually relevant to query
+
+Examples:
+  ├─ Exact match: 0.95+
+  ├─ Semantic match: 0.60-0.80
+  └─ Weak match: 0.10-0.30
+
+Weight: 25%
+```
+
+### Final Ranking Formula
+
+```
+final_score = 0.25 * confidence
+            + 0.25 * recency_score
+            + 0.25 * community_score
+            + 0.25 * bm25_normalized
+
+Range: [0.0, 1.0]
+
+Tie-breaking (deterministic):
+  1. Higher final_score wins
+  2. If tied: newer version_timestamp wins
+  3. If still tied: lexicographically by chunk_id
+```
+
+### Example Calculation
+
+```
+Entity: "Kubernetes"
+
+Signals:
+  ├─ Confidence: 0.92 (human-verified, Phase 5.4)
+  ├─ Recency: 0.88 (last verified 5 days ago)
+  ├─ Community: 0.85 (PageRank in entity graph)
+  └─ BM25: 0.78 (matched query terms)
+
+Calculation:
+  final_score = 0.25*0.92 + 0.25*0.88 + 0.25*0.85 + 0.25*0.78
+              = 0.23 + 0.22 + 0.2125 + 0.195
+              = 0.8575
+
+Ranking: Top result (if query relevance high)
+```
+
+---
+
+## 5. Audit Trail Strategy
+
+### Append-Only Event Log (JSONL)
+
+```json
+{
+  "id": "audit-evt-123",
+  "timestamp": "2025-01-30T14:30:45Z",
+  "event_type": "chunk_edited|chunk_created|chunk_verified|chunk_deleted",
+  "actor": "alice@company.com",
+  "resource": {
+    "type": "chunk",
+    "id": "ent-123",
+    "version": 2
+  },
+  "action": {
+    "operation": "update",
+    "old_value": {
+      "content": "Kubernetes: Container orchestrator by Google",
+      "confidence": 0.85
+    },
+    "new_value": {
+      "content": "Kubernetes: Container orchestrator by Google, open-source CNCF project",
+      "confidence": 0.92
+    }
+  },
+  "reason": "Added CNCF affiliation after verification",
+  "signature": "sha256(timestamp + actor + resource + action)"
+}
+```
+
+### Storage & Queries
+
+```sql
+-- Append-only event log
+CREATE TABLE audit_events (
+  id UUID PRIMARY KEY,
+  timestamp TIMESTAMP NOT NULL,
+  event_type VARCHAR(50),
+  actor VARCHAR(255),
+  resource_type VARCHAR(50),
+  resource_id UUID,
+  version INT,
+  action JSONB,
+  reason TEXT,
+  signature VARCHAR(256),
+  created_at TIMESTAMP DEFAULT now()
+) PARTITION BY RANGE (timestamp);  -- Monthly partitions
+
+-- Query: What edits were made to entity?
+SELECT * FROM audit_events
+WHERE resource_type = 'chunk' AND resource_id = $chunk_id
+ORDER BY timestamp DESC;
+
+-- Query: Who changed what, when?
+SELECT timestamp, actor, reason, action
+FROM audit_events
+WHERE resource_id = $chunk_id
+ORDER BY timestamp;
+
+-- Query: Verify integrity (signature validation)
+SELECT * FROM audit_events
+WHERE signature != sha256(...)
+LIMIT 100;  -- Detect tampering
+```
+
+### Immutability Guarantees
+
+```
+1. No UPDATE on audit_events table
+   ├─ Only INSERT is allowed
+   └─ Policy: ROW SECURITY
+
+2. Separate write vs read storage
+   ├─ Write: Hot log (Kafka JSONL file)
+   ├─ Read: Cold archive (S3 + compression)
+   └─ Sync: Daily batch to S3
+
+3. Cryptographic signing
+   ├─ Each event: signature = sha256(timestamp || actor || resource || action)
+   ├─ Chain of custody: timestamp chain prevents reordering
+   └─ Verification: Detect tampering in O(1) time
+```
+
+---
+
+## 6. Schema Extensions
+
+### New Tables (Phase 7)
+
+```sql
+-- Versioning
+CREATE TABLE chunk_versions (
+  id UUID PRIMARY KEY,
+  chunk_id UUID NOT NULL,              -- Stable reference
+  version INT NOT NULL,
+  content TEXT,
+  properties JSONB,
+  edited_by VARCHAR(255),
+  edit_reason TEXT,
+  confidence FLOAT DEFAULT 0.5,
+  temporal_start TIMESTAMP,
+  temporal_end TIMESTAMP,
+  is_current BOOLEAN,
+  created_at TIMESTAMP DEFAULT now(),
+  UNIQUE(chunk_id, version),
+  FOREIGN KEY (chunk_id) REFERENCES memory_entity(id)
+);
+
+-- Audit trail
+CREATE TABLE audit_events (
+  id UUID PRIMARY KEY,
+  timestamp TIMESTAMP NOT NULL,
+  event_type VARCHAR(50) NOT NULL,
+  actor VARCHAR(255),
+  resource_type VARCHAR(50),
+  resource_id UUID,
+  version INT,
+  action JSONB,
+  reason TEXT,
+  signature VARCHAR(256),
+  created_at TIMESTAMP DEFAULT now()
+) PARTITION BY RANGE (timestamp);
+
+-- Ranking signals
+CREATE TABLE ranking_signals (
+  id UUID PRIMARY KEY,
+  entity_id UUID NOT NULL,
+  signal_type VARCHAR(50),  -- "confidence"|"recency"|"community"|"bm25"
+  signal_value FLOAT,
+  computed_at TIMESTAMP,
+  FOREIGN KEY (entity_id) REFERENCES memory_entity(id)
+);
+
+-- Query rankings (for metrics)
+CREATE TABLE query_rankings (
+  id UUID PRIMARY KEY,
+  query_id UUID,
+  chunk_id UUID,
+  rank INT,
+  final_score FLOAT,
+  signal_breakdown JSONB,  -- {confidence, recency, community, bm25}
+  computed_at TIMESTAMP,
+  FOREIGN KEY (chunk_id) REFERENCES memory_entity(id)
+);
+```
+
+### Schema Modifications
+
+```sql
+-- Add to existing memory_entity
+ALTER TABLE memory_entity ADD COLUMN (
+  confidence FLOAT DEFAULT 0.5,
+  last_verified_at TIMESTAMP,
+  edit_count INT DEFAULT 0,
+  current_version INT DEFAULT 1
+);
+
+-- Add to existing memory_edge
+ALTER TABLE memory_edge ADD COLUMN (
+  confidence FLOAT DEFAULT 0.5,
+  recency_score FLOAT DEFAULT 1.0,
+  community_signal FLOAT DEFAULT 0.5,
+  last_updated_at TIMESTAMP
+);
+```
+
+---
+
+## 7. Editing Workflow
+
+### User Edits a Chunk
+
+```
+1. User opens chunk in UI
+   └─ GET /memory/chunks/{id}
+      └─ Returns: current_version + full version history
+
+2. User modifies content
+   └─ Content changes: "Kubernetes: Container orchestrator"
+                       → "Kubernetes: Container orchestrator (CNCF project)"
+
+3. User submits edit
+   └─ PATCH /memory/chunks/{id}
+      └─ Payload: {
+           "new_content": "...",
+           "edit_reason": "Added CNCF affiliation",
+           "confidence": 0.92
+         }
+
+4. Memory service:
+   a. Validate edit (optional LLM gate for quality)
+   b. Create new version:
+      INSERT chunk_versions (chunk_id=id, version=2, content='...', ...)
+   c. Update is_current: chunk_versions SET is_current=false WHERE version=1
+   d. Audit log:
+      INSERT audit_events (event_type='chunk_edited', resource_id=id, ...)
+   e. Recompute signals:
+      ├─ Confidence: extracted_confidence (or manual 0.92)
+      ├─ Recency: 1.0 (just edited)
+      ├─ Community: fetch from graph
+      └─ BM25: reindex OpenSearch
+   f. Return: {chunk_id, new_version, status: "OK"}
+
+5. Query uses latest version:
+   └─ SELECT * FROM chunk_versions
+      WHERE chunk_id = $1 AND is_current = true
+```
+
+---
+
+## 8. Deterministic Rebuild (Parity Check)
+
+### Challenge
+With versioning, **which version** do we rebuild from?
+
+### Solution: Version Snapshot + Audit Replay
+
+```
+Rebuild Process:
+
+1. Snapshot current state:
+   SELECT chunk_id, max(version) as latest_version
+   FROM chunk_versions
+   WHERE is_current = true
+   → SHA256 hash of {chunk_id, version, content}*
+   → Store: rebuild_checksum_v1
+
+2. From event log, replay all edits since T0:
+   SELECT * FROM audit_events
+   WHERE timestamp >= $baseline_timestamp
+   ORDER BY timestamp ASC
+
+3. Apply edits sequentially:
+   For each audit_event:
+     ├─ Create new chunk_version
+     ├─ Mark old is_current = false
+     ├─ Compute signals
+     └─ Log event
+
+4. Verify parity:
+   SELECT chunk_id, max(version)
+   FROM chunk_versions
+   WHERE is_current = true
+   → SHA256 hash
+   → Check: rebuild_checksum_v2 == rebuild_checksum_v1
+   → If match: ✅ Parity verified (M2.8 gate)
+   → If diff: ❌ Rebuild failed (investigate)
+```
+
+---
+
+## 9. Integration Points
+
+### Phase 4 (Retrieval) → Phase 7 (Ranking)
+
+**Current** (Phase 4):
+```
+Query → Semantic + Lexical → RRF Fusion → Top-10 results
+```
+
+**Phase 7** (Enhanced):
+```
+Query → Semantic + Lexical → RRF Fusion
+  ↓
+Multi-Signal Ranking:
+  ├─ Fetch confidence, recency, community from ranking_signals
+  ├─ Compute bm25_normalized from search scores
+  ├─ Calculate: final_score = 0.25*conf + 0.25*recency + 0.25*community + 0.25*bm25
+  └─ Sort by final_score DESC, then timestamp DESC, then chunk_id ASC
+
+→ Top-10 results (with confidence intervals)
+```
+
+### Phase 5 (Synthesis) → Phase 7 (Confidence)
+
+**Current** (Phase 5):
+```
+LLM extracts entities → Insert into memory_entity
+```
+
+**Phase 7** (Enhanced):
+```
+LLM extracts entities + confidence
+  ↓
+INSERT chunk_versions (confidence=$extracted_confidence)
+  ↓
+Signals computed:
+  ├─ Confidence: $extracted_confidence (0.0-1.0)
+  ├─ Recency: 1.0 (new)
+  ├─ Community: graph_analysis() (0.0 initially, grows as linked)
+  └─ BM25: from indexing
+  ↓
+Track lineage: audit_events with {operation, old_value, new_value}
+```
+
+---
+
+## 10. Metrics Emission & Prometheus Scraping
+
+### Metrics Endpoint
+
+```
+GET /metrics
+├─ Auth: Authentik JWT (Bearer token)
+├─ Format: Prometheus text format (OpenMetrics)
+├─ Refresh: Real-time (computed on-demand)
+└─ Timeout: 5 seconds
+```
+
+### Metrics Architecture
+
+```
+┌─────────────────────────────────────────┐
+│ Memory Service                          │
+│ ├─ MetricsPersistence (in-memory cache) │
+│ ├─ record_success(agent_id, latency)   │
+│ ├─ record_error(agent_id, latency)     │
+│ └─ export_prometheus()                  │
+│    └─ Returns: Prometheus text format   │
+└────────────┬────────────────────────────┘
+             │ Emits every 5s
+             ↓
+┌─────────────────────────────────────────┐
+│ Prometheus Server                       │
+│ ├─ Scrape /metrics (15s interval)       │
+│ ├─ Store TSDB (15-month retention)      │
+│ └─ Alert rules (M7.1-M7.5 gates)        │
+└────────────┬────────────────────────────┘
+             │
+             ↓
+┌─────────────────────────────────────────┐
+│ Dashboards (Grafana)                    │
+│ ├─ Phase 7 SLO dashboard                │
+│ ├─ Edit latency percentiles             │
+│ ├─ Ranking accuracy (NDCG weighted)     │
+│ └─ Storage overhead tracking            │
+└─────────────────────────────────────────┘
+```
+
+### Phase 7 Metrics
+
+```prometheus
+# HELP memory_chunk_edits_total Total chunk edits (create/update/delete)
+# TYPE memory_chunk_edits_total counter
+memory_chunk_edits_total{operation="create"} 1245
+memory_chunk_edits_total{operation="update"} 3421
+memory_chunk_edits_total{operation="delete"} 18
+
+# HELP memory_chunk_versions_current Current version count per chunk
+# TYPE memory_chunk_versions_current gauge
+memory_chunk_versions_current{chunk_type="entity"} 5234
+memory_chunk_versions_current{chunk_type="edge"} 2189
+
+# HELP memory_edit_latency_seconds Edit operation latency
+# TYPE memory_edit_latency_seconds histogram
+memory_edit_latency_seconds_bucket{le="0.1"} 156
+memory_edit_latency_seconds_bucket{le="0.5"} 1023
+memory_edit_latency_seconds_bucket{le="1.0"} 3421
+memory_edit_latency_seconds_bucket{le="2.0"} 4156  # P99 < 2s gate
+memory_edit_latency_seconds_bucket{le="+Inf"} 4189
+memory_edit_latency_seconds_sum 5234.56
+memory_edit_latency_seconds_count 4189
+
+# HELP memory_audit_events_total Total audit events logged
+# TYPE memory_audit_events_total counter
+memory_audit_events_total{event_type="chunk_edited"} 3421
+memory_audit_events_total{event_type="chunk_created"} 1245
+memory_audit_events_total{event_type="verification"} 892
+
+# HELP memory_audit_signature_failures_total Signature verification failures
+# TYPE memory_audit_signature_failures_total counter
+memory_audit_signature_failures_total 0  # M7.2 gate: must be 0
+
+# HELP memory_rebuild_checksum_matches_total Rebuild parity checks passed
+# TYPE memory_rebuild_checksum_matches_total counter
+memory_rebuild_checksum_matches_total 127  # M7.1 gate: 100%
+
+# HELP memory_rebuild_checksum_mismatches_total Rebuild parity checks FAILED
+# TYPE memory_rebuild_checksum_mismatches_total counter
+memory_rebuild_checksum_mismatches_total 0  # M7.1 gate: must be 0
+
+# HELP memory_ranking_ndcg_weighted Weighted NDCG@10 (confidence-adjusted)
+# TYPE memory_ranking_ndcg_weighted gauge
+memory_ranking_ndcg_weighted 0.882  # M7.3 gate: >= 0.88
+
+# HELP memory_ranking_signal_contribution Contribution of each signal
+# TYPE memory_ranking_signal_contribution gauge
+memory_ranking_signal_contribution{signal="confidence"} 0.25
+memory_ranking_signal_contribution{signal="recency"} 0.25
+memory_ranking_signal_contribution{signal="community"} 0.25
+memory_ranking_signal_contribution{signal="bm25"} 0.25
+
+# HELP memory_storage_bytes Total storage bytes
+# TYPE memory_storage_bytes gauge
+memory_storage_bytes{type="current_versions"} 5368709120
+memory_storage_bytes{type="all_versions"} 8053063680  # M7.5 gate: <= 1.5x
+memory_storage_bytes{type="audit_log"} 2147483648
+
+# HELP memory_storage_overhead_ratio Overhead ratio (versions / current)
+# TYPE memory_storage_overhead_ratio gauge
+memory_storage_overhead_ratio 1.25  # M7.5 gate: <= 1.5x
+
+# HELP memory_confidence_distribution Confidence score distribution
+# TYPE memory_confidence_distribution histogram
+memory_confidence_distribution_bucket{le="0.3"} 234    # Uncertain
+memory_confidence_distribution_bucket{le="0.6"} 1289   # Moderate
+memory_confidence_distribution_bucket{le="0.85"} 4156  # High
+memory_confidence_distribution_bucket{le="1.0"} 5234   # Verified
+
+# HELP memory_recency_score_distribution Recency score distribution
+# TYPE memory_recency_score_distribution gauge
+memory_recency_score_avg 0.72  # Average recency (30 days half-life)
+memory_recency_score_p50 0.78
+memory_recency_score_p99 0.98
+
+# HELP memory_community_score_distribution Community signal distribution
+# TYPE memory_community_score_distribution gauge
+memory_community_score_avg 0.45
+memory_community_score_p50 0.42
+memory_community_score_p99 0.91
+```
+
+### Prometheus Scrape Config
+
+```yaml
+# In homelab Prometheus values.yaml
+prometheus:
+  scrapeConfigs:
+    - job_name: 'poimen-memory'
+      kubernetes_sd_configs:
+        - role: pod
+          namespaces:
+            names:
+              - poimen
+      relabel_configs:
+        - source_labels: [__meta_kubernetes_pod_label_app]
+          action: keep
+          regex: memory-service
+        - source_labels: [__meta_kubernetes_pod_container_port_name]
+          action: keep
+          regex: metrics
+      scheme: https
+      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
+      tls_config:
+        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
+      scrape_interval: 15s
+      scrape_timeout: 5s
+```
+
+### SLO Alerts (Prometheus Rules)
+
+```yaml
+# rules/poimen-memory-phase7.yaml
+apiVersion: monitoring.coreos.com/v1
+kind: PrometheusRule
+metadata:
+  name: poimen-memory-phase7
+  namespace: monitoring
+spec:
+  groups:
+    - name: memory-phase7-gates
+      interval: 30s
+      rules:
+        # M7.1: Rebuild parity
+        - alert: M7_1_RebuildParityCheckFailed
+          expr: increase(memory_rebuild_checksum_mismatches_total[5m]) > 0
+          for: 1m
+          labels:
+            severity: critical
+            phase: "7"
+          annotations:
+            summary: "Rebuild parity check failed (M7.1 gate)"
+            description: "{{ $value }} checksum mismatches in last 5 minutes"
+        
+        # M7.2: Audit trail integrity
+        - alert: M7_2_AuditSignatureFailure
+          expr: increase(memory_audit_signature_failures_total[5m]) > 0
+          for: 1m
+          labels:
+            severity: critical
+            phase: "7"
+          annotations:
+            summary: "Audit signature verification failed (M7.2 gate)"
+            description: "{{ $value }} signature failures detected"
+        
+        # M7.3: Ranking accuracy
+        - alert: M7_3_RankingAccuracyDegraded
+          expr: memory_ranking_ndcg_weighted < 0.88
+          for: 5m
+          labels:
+            severity: warning
+            phase: "7"
+          annotations:
+            summary: "Ranking accuracy below SLO (M7.3 gate)"
+            description: "NDCG@10 = {{ $value | humanizePercentage }} (target: >= 0.88)"
+        
+        # M7.4: Edit latency
+        - alert: M7_4_EditLatencyHigh
+          expr: histogram_quantile(0.99, memory_edit_latency_seconds) > 2.0
+          for: 5m
+          labels:
+            severity: warning
+            phase: "7"
+          annotations:
+            summary: "Edit latency P99 exceeds SLO (M7.4 gate)"
+            description: "P99 latency = {{ $value | humanizeDuration }} (target: < 2s)"
+        
+        # M7.5: Storage overhead
+        - alert: M7_5_StorageOverheadHigh
+          expr: memory_storage_overhead_ratio > 1.5
+          for: 10m
+          labels:
+            severity: warning
+            phase: "7"
+          annotations:
+            summary: "Storage overhead exceeds SLO (M7.5 gate)"
+            description: "Overhead ratio = {{ $value | humanizePercentage }} (target: <= 1.5x)"
+```
+
+### Grafana Dashboard (Phase 7)
+
+```json
+{
+  "dashboard": {
+    "title": "Poimen Memory - Phase 7 SLO Dashboard",
+    "panels": [
+      {
+        "title": "M7.1 Rebuild Parity",
+        "targets": [
+          {
+            "expr": "memory_rebuild_checksum_matches_total / (memory_rebuild_checksum_matches_total + memory_rebuild_checksum_mismatches_total)"
+          }
+        ],
+        "alert_threshold": 1.0,
+        "description": "Must be 100% (gate M7.1)"
+      },
+      {
+        "title": "M7.3 Ranking Accuracy (NDCG@10 weighted)",
+        "targets": [
+          {
+            "expr": "memory_ranking_ndcg_weighted"
+          }
+        ],
+        "alert_threshold": 0.88,
+        "description": "Target: >= 0.88 (gate M7.3)"
+      },
+      {
+        "title": "M7.4 Edit Latency P99",
+        "targets": [
+          {
+            "expr": "histogram_quantile(0.99, memory_edit_latency_seconds)"
+          }
+        ],
+        "alert_threshold": 2.0,
+        "description": "Target: < 2s (gate M7.4)"
+      },
+      {
+        "title": "M7.5 Storage Overhead",
+        "targets": [
+          {
+            "expr": "memory_storage_overhead_ratio"
+          }
+        ],
+        "alert_threshold": 1.5,
+        "description": "Target: <= 1.5x (gate M7.5)"
+      }
+    ]
+  }
+}
+```
+
+---
+
+## 11. Metrics & SLOs
+
+### Phase 7 Composition Gate
+
+```yaml
+gates:
+  - name: M7.1_version_parity
+    type: rebuild_checksum_match
+    threshold: 100%
+    sla: "All rebuilds must produce identical checksums"
+    
+  - name: M7.2_audit_trail_integrity
+    type: signature_verification
+    threshold: 100%
+    sla: "Zero tampered audit events"
+    
+  - name: M7.3_ranking_accuracy
+    type: ndcg_weighted_by_confidence
+    threshold: ">= 0.88"
+    sla: "NDCG@10 with multi-signal ranking >= 0.88"
+    
+  - name: M7.4_chunk_edit_latency
+    type: p99_latency
+    threshold: "<= 2s"
+    sla: "99th percentile edit completion <= 2 seconds"
+    
+  - name: M7.5_version_space
+    type: storage_overhead
+    threshold: "<= 1.5x"
+    sla: "Versioned storage overhead <= 1.5x original"
+```
+
+---
+
+## 11. Implementation Roadmap
+
+### Phase 7.1: Schema & Migration (Week 1)
+- [ ] Create chunk_versions table
+- [ ] Create audit_events table (partitioned)
+- [ ] Create ranking_signals table
+- [ ] Migration script (6.x → 7.x)
+- [ ] Backfill ranking_signals from existing data
+
+### Phase 7.2: Versioning API (Week 2)
+- [ ] GET /memory/chunks/{id} (with version history)
+- [ ] PATCH /memory/chunks/{id} (edit + create version)
+- [ ] GET /memory/chunks/{id}/versions (list all versions)
+- [ ] POST /memory/chunks/{id}/versions/{v}/revert (rollback)
+- [ ] Tests: 50+ unit + integration
+
+### Phase 7.3: Audit Trail (Week 2)
+- [ ] Append-only event logging
+- [ ] Signature verification (cryptographic)
+- [ ] GET /memory/audit?resource_id={id} (query trail)
+- [ ] Backup to S3 (cold storage)
+- [ ] Tests: 30+ unit
+
+### Phase 7.4: Multi-Signal Ranking (Week 3)
+- [ ] Compute confidence from extraction pipeline
+- [ ] Compute recency (last_verified_at)
+- [ ] Compute community_signal (PageRank)
+- [ ] Normalize BM25 scores
+- [ ] Integrate into query ranking (hybrid_search.rs)
+- [ ] Tests: 40+ unit
+
+### Phase 7.5: Deterministic Rebuild (Week 3)
+- [ ] Snapshot + version tracking
+- [ ] Replay audit events
+- [ ] Parity verification (M7.1 gate)
+- [ ] Tests: 20+ integration
+
+### Phase 7.6: Documentation & SLOs (Week 4)
+- [ ] API documentation
+- [ ] Audit trail guide
+- [ ] Editing workflow guide
+- [ ] SLO dashboards (Prometheus)
+- [ ] Runbooks (troubleshooting)
+
+---
+
+## 12. Key Design Decisions
+
+| Decision | Choice | Rationale |
+|----------|--------|-----------|
+| Versioning | Immutable (Option A) | Clearer lineage, no tombstones |
+| Signals | 4 equal weights (25% each) | Balanced, interpretable |
+| Audit storage | Append-only JSONL + S3 | Immutable, queryable, archivable |
+| Rebuild parity | Signature verification | Fast (O(1)), detects corruption |
+| Edit latency | P99 < 2s | Real-time editing UX |
+| Confidence source | LLM extraction pipeline | Aligned with Phase 5 |
+| Recency decay | Exponential (τ=30d) | Standard information theory |
+| Community signal | PageRank + in-degree | Graph-theoretic, proven |
+
+---
+
+## 13. Risks & Mitigations
+
+| Risk | Impact | Mitigation |
+|------|--------|-----------|
+| Version explosion (storage) | 1.5-2x overhead | Compression, periodic archival, TTL cleanup |
+| Audit log query slowness | Query latency spikes | Partitioning by time, materialized views |
+| Signature verification false positives | Trust erosion | Comprehensive testing, HSM backup |
+| Community signal staleness | Ranking drift | Recompute PageRank daily (off-peak) |
+| Editing conflicts | Concurrent edit loss | Optimistic locking via version number |
+
+---
+
+## 14. Success Criteria
+
+✅ **Phase 7 Complete When:**
+1. All 5 composition gates pass (M7.1-M7.5)
+2. 150+ tests passing (unit + integration)
+3. NDCG@10 with weighted confidence >= 0.88
+4. Edit latency P99 < 2s
+5. Storage overhead <= 1.5x
+6. Audit trail 100% immutable (0 tampering detected)
+7. Full documentation + runbooks
+8. Production SLA: 99.9% availability
+
+---
+
+## 15. References
+
+- **Versioning**: Git model (immutable commits, branches = versions)
+- **Ranking signals**: Okapi BM25 + PageRank + temporal decay (arXiv:1802.05365)
+- **Audit trails**: NIST SP 800-92 (log management)
+- **Parity checks**: Deterministic replay (Riak design, §3.2)
+- **Temporal semantics**: ISO 8601 + valid_from/valid_to
+
+---
+
+**Status**: 🔵 **DESIGN COMPLETE - READY FOR PHASE 7.1 IMPLEMENTATION**
+
+Next: Review & finalize schema, then begin Phase 7.1 (migrations + versioning API).