poimen-memory/docs/api/T7_REBUILD_API.md

T7.5: Deterministic Rebuild API Reference

Overview

Verify rebuild determinism with SHA-256 checksums. Daily CI integration with zero drift tolerance.

---

Endpoints

POST /memory/rebuild

Trigger rebuild with optional verification

Request:

curl -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "project": "poimen",
    "verify": true,
    "dry_run": false
  }' \
  http://localhost:8080/memory/rebuild

Response (200 OK - Success):

{
  "status": "success",
  "rebuild_id": "rebuild-2025-01-30-100000",
  "records_processed": 45230,
  "duration_ms": 142500,
  "checksum": {
    "before": "a3f8c9d2e1b4...",
    "after": "a3f8c9d2e1b4...",
    "match": true
  },
  "incremental": false,
  "from_checkpoint": null,
  "diff_summary": null
}

Response (409 Conflict - Checksum Mismatch):

{
  "status": "failed",
  "rebuild_id": "rebuild-2025-01-30-100001",
  "records_processed": 45230,
  "duration_ms": 145000,
  "checksum": {
    "before": "a3f8c9d2e1b4...",
    "after": "7f2e1a9c8b3d...",
    "match": false
  },
  "incremental": false,
  "from_checkpoint": null,
  "diff_summary": {
    "entities_added": 3,
    "entities_removed": 0,
    "entities_modified": 12,
    "edges_added": 5,
    "edges_modified": 8
  }
}

---

GET /memory/rebuild/status

Get last rebuild result and checkpoints

Request:

curl -H "Authorization: Bearer $TOKEN" \
  http://localhost:8080/memory/rebuild/status

Response (200 OK):

{
  "last_rebuild": {
    "rebuild_id": "rebuild-2025-01-30-100000",
    "started_at": "2025-01-30T10:00:00Z",
    "completed_at": "2025-01-30T10:02:22Z",
    "status": "success",
    "checksum": "a3f8c9d2e1b4...",
    "records_processed": 45230
  },
  "checkpoints": [
    {
      "id": "cp-2025-01-29",
      "created_at": "2025-01-29T00:00:00Z",
      "event_count": 44000,
      "checksum": "b4c9d8e7f6a5..."
    },
    {
      "id": "cp-2025-01-28",
      "created_at": "2025-01-28T00:00:00Z",
      "event_count": 43000,
      "checksum": "c5d8e7f6a4b3..."
    }
  ],
  "health": {
    "event_log_size": 45230,
    "last_event_at": "2025-01-30T09:55:00Z",
    "estimated_rebuild_time_ms": 145000
  }
}

---

Request Options

verify (Boolean, default: true)

Compute checksums before/after rebuild. Fails if mismatch.

# With verification (recommended)
curl -X POST http://localhost:8080/memory/rebuild \
  -d '{"project": "poimen", "verify": true}'

# Without verification (fast path, less safe)
curl -X POST http://localhost:8080/memory/rebuild \
  -d '{"project": "poimen", "verify": false}'

dry_run (Boolean, default: false)

Preview rebuild without applying. Returns what would happen.

curl -X POST http://localhost:8080/memory/rebuild \
  -d '{"project": "poimen", "verify": true, "dry_run": true}'

# Response:
{
  "status": "dry_run",
  "message": "Rebuild would succeed",
  "result": {...}
}

from_checkpoint (String, optional)

Resume incremental rebuild from checkpoint.

# Full rebuild
curl -X POST http://localhost:8080/memory/rebuild \
  -d '{"project": "poimen", "from_checkpoint": null}'

# Incremental from checkpoint
curl -X POST http://localhost:8080/memory/rebuild \
  -d '{"project": "poimen", "from_checkpoint": "cp-2025-01-29"}'

---

Use Cases

Daily Verification (CI)

#!/bin/bash
# Run every day at 2 AM UTC

TOKEN=$(get_jwt_token)
RESULT=$(curl -s -X POST http://localhost:8080/memory/rebuild \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "project": "poimen",
    "verify": true,
    "dry_run": false
  }')

STATUS=$(echo $RESULT | jq -r '.status')
MATCH=$(echo $RESULT | jq -r '.checksum.match')

if [ "$MATCH" != "true" ]; then
  echo "CRITICAL: Rebuild parity failed!"
  echo $RESULT | jq '.diff_summary'
  alert_team
  exit 1
fi

echo "OK: Rebuild is deterministic"

Dry-Run Before Production

# Test the rebuild without committing
curl -X POST http://localhost:8080/memory/rebuild \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "project": "prod",
    "verify": true,
    "dry_run": true
  }'

# Response shows what would happen
# If satisfied, run again without dry_run

Incremental Rebuild

# For large datasets, resume from checkpoint
curl -X POST http://localhost:8080/memory/rebuild \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "project": "poimen",
    "verify": true,
    "from_checkpoint": "cp-2025-01-29"
  }'

Post-Incident Recovery

# After fixing corruption, verify system recovers
curl -X POST http://localhost:8080/memory/rebuild \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "project": "poimen",
    "verify": true
  }'

# Check status
curl -H "Authorization: Bearer $TOKEN" \
  http://localhost:8080/memory/rebuild/status

---

Checksum Details

Computation

checksum = SHA256(
  sorted_entity_ids ||
  sorted_edge_ids ||
  event_metadata
)

Deterministic because:

1. Entities sorted by ID (no random order)
2. Edges sorted by ID
3. Event replay sequential (no async variation)
4. Embedding model pinned (same version)
5. JSON serialization canonical (sorted keys)

Interpretation

Checksums match (match: true):

• ✅ Rebuild is byte-identical
• ✅ No hidden non-determinism
• ✅ Data is consistent

Checksums differ (match: false):

• ❌ Non-determinism detected
• ❌ Investigate: embedding model change? Event log corruption?
• ❌ BLOCK further rebuilds until root cause fixed

---

Error Codes

Code	Meaning
200	Rebuild succeeded (or dry_run verified)
201	Checkpoint created
409	Checksum mismatch (parity failed)
401	Unauthorized (missing/invalid JWT)
403	Forbidden (insufficient permissions)
408	Request timeout (rebuild took > 60s)
503	Service unavailable (DB connection failed)

---

Rate Limits

Endpoint	Limit
/rebuild	10/day (prevent spam)
/rebuild/status	100/hr

---

Monitoring

Prometheus Metrics

# Rebuild success rate (daily)
sum(rate(memory_rebuild_total{status="success"}[1d]))
/
sum(rate(memory_rebuild_total[1d]))

# Average rebuild time
avg(memory_rebuild_duration_seconds)

# Checksum matches (should be 100%)
memory_rebuild_parity_check{status="success"} > 0

Alerts

- alert: RebuildParityFailed
  expr: memory_rebuild_parity_check{status="failed"} > 0
  for: 0m
  annotations:
    summary: "Deterministic rebuild checksum mismatch"
    severity: critical

---

CI Integration

GitHub Actions Workflow

# .github/workflows/rebuild-verify.yml
name: Verify Deterministic Rebuild
on:
  schedule:
    - cron: '0 2 * * *'  # 2 AM UTC daily
jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - name: Rebuild with verification
        env:
          API_URL: ${{ secrets.MEMORY_API_URL }}
          TOKEN: ${{ secrets.MEMORY_API_TOKEN }}
        run: |
          RESULT=$(curl -s -X POST $API_URL/memory/rebuild \
            -H "Authorization: Bearer $TOKEN" \
            -d '{"project":"poimen","verify":true}')
          
          MATCH=$(echo $RESULT | jq -r '.checksum.match')
          [ "$MATCH" = "true" ] || exit 1
      
      - name: Notify on failure
        if: failure()
        uses: slackapi/slack-github-action@v1
        with:
          payload: |
            {"text": "🚨 Rebuild parity check FAILED!"}