poimenandrock 4169effd8a
CI / CI (push) Successful in 12m36s
Deploy / Tag & Push Latest (push) Successful in 1m56s
feat: complete observability stack (O1-O13) (#52)
## Complete Observability Stack (O1-O13)

Implements all 13 observability issues in a single PR. 119 metrics total.

### Commits (one per issue)

| Issue | Title | Metrics |
|-------|-------|---------|
| **O10** | Prometheus metrics module + /metrics endpoint | Foundation |
| **O1** | Instrument ingest handler | I1-I12 (12) |
| **O2** | Instrument query handler | Q1-Q12 (12) |
| **O3** | Instrument context endpoint | C1-C8 (8) |
| **O4** | Relevance judge | R1-R9 (9) |
| **O5** | Write volume + storage metrics | W1-W12 (12) |
| **O6** | Pod resource observability | P1-P13 |
| **O7** | Availability + dependency health | A1-A10 (10) |
| **O8** | Ingest rate pattern tracking | IR1-IR10 (10) |
| **O9** | Postgres internal observability | PG1-PG33 |
| **O11** | Grafana dashboard | 12 panels |
| **O12** | Prometheus alerting rules | 11 alerts |
| **O13** | Relevance evaluation CronJob | K8s manifest |

### Key Changes

- **metrics.rs**: Zero-dependency Prometheus metrics (Counter, Gauge, Histogram, Timer)
- **GET /metrics**: Prometheus text exposition format endpoint
- **Ingest/Query/Context handlers**: Instrumented with latency, errors, auth failures
- **Health check**: DB dependency check with latency tracking
- **Background task**: Periodic DB stats collection (entity/edge counts, pool stats)
- **Relevance judge**: Threshold-based eval with precision/recall/F1 tracking
- **Grafana dashboard**: 12 panels covering all metric groups
- **Alert rules**: 11 PrometheusRule alerts (availability, latency, errors, quality)
- **CronJob**: Periodic relevance evaluation with sample queries

### Testing

- 506 tests passing (0 failures)
- All metrics modules have unit tests
- Relevance judge: 4 tests

### Deploy

```bash
# Grafana dashboard
kubectl apply -f k8s/infra/grafana-dashboard.json

# Prometheus alerts
kubectl apply -f k8s/infra/prometheus-alerts.yaml

# Relevance eval CronJob
kubectl apply -f k8s/infra/relevance-eval-cronjob.yaml
```

Closes #27 #28 #29 #30 #31 #32 #33 #34 #35 #36 #37 #38 #39

---------

Co-authored-by: rock <[email protected]>
Reviewed-on: #52
Co-authored-by: poimen <[email protected]>
2026-09-13 13:53:50 +00:00
2026-08-22 23:13:42 -07:00

Poimen Memory System

Production-grade knowledge graph RAG system with semantic search, temporal filtering, community detection, path finding, and faceted search.

Quick Start

# Build
cargo build --release

# Run
cargo run --release -- --config config/default.toml

API Documentation

See API.md for complete endpoint specifications, request/response formats, and usage examples.

Core Endpoints

  • POST /memory/query/semantic/entities — Semantic search with optional community detection, path finding, facet discovery
  • POST /memory/query/semantic/edges — Relation search with temporal and facet filters
  • POST /memory/query/hybrid — Combined semantic + lexical search (RRF fusion)

Optional Features (via query parameters)

  • Temporal Filtering: start_time, end_time (ISO 8601 datetime)
  • Community Detection: detect_communities=true, min_community_size=N
  • Path Finding: find_paths=true, target_entity_id=<id>, max_path_depth=N, k_hops=N
  • Faceted Search: discover_facets=true, facet_filters={...}

Architecture

crates/mem-cli/src/
├── query/
│   ├── semantic_retriever.rs    (vector + lexical search)
│   ├── community_detector.rs    (Louvain algorithm)
│   ├── path_finder.rs           (BFS/DFS graph traversal)
│   └── faceted_search.rs        (multi-dimension filtering)
├── handlers/
│   └── semantic.rs              (HTTP endpoints)
└── http_server.rs               (Actix-web server)

crates/mem-core/src/
├── domain.rs                    (data structures)
├── entity.rs, edge.rs           (graph entities)
└── scoring.rs                   (relevance metrics)

crates/mem-store/src/
└── *_repo.rs                    (database persistence)

Testing

# Run all tests
cargo test --lib

# Run specific test suite
cargo test --lib query::semantic
cargo test --lib handlers::semantic

# With output
cargo test --lib -- --nocapture

Configuration

See config/default.toml for:

  • Database connection strings
  • JWT authentication settings
  • Rate limiting thresholds
  • Embeddings model configuration

Production Deployment

  1. Build release binary: cargo build --release
  2. Set environment: JWT_SECRET, DATABASE_URL, OPENAI_API_KEY
  3. Run: ./target/release/mem-cli
  4. Health check: GET http://localhost:8080/health

Development

Quality Standards:

  • CRAP score < 3.2 (low complexity)
  • DRY > 98% (minimal duplication)
  • SOLID 5.0/5 (excellent design)
  • 230+ comprehensive tests (100% pass rate)
  • Performance: P50 latency < 500ms

Adding New Features:

  1. Create core module in crates/mem-cli/src/query/
  2. Add optional parameters to request struct
  3. Extend response with optional field (use skip_serializing_if)
  4. Add handler logic (delegate to core module)
  5. Write 25-35 tests (unit + integration)
  6. Document in API.md

See CLAUDE.md for project context and constraints.

CI test 1788759975

S
Description
Agent-ready Graph-RAG system with hallucination prevention and enterprise RBAC
https://forgejo.riotpiao.com/rock/poimen-memory
Readme
3.3 MiB
Languages
Rust 98.6%
Shell 0.8%
Python 0.4%
PLpgSQL 0.2%