riotpiao-poimen/poimen-memory
2
0
Fork 0
Code Issues 13 Pull Requests Actions 3 Packages Projects Releases Wiki Activity

O7: Availability metrics and dependency health (A1-A10) #33

New Issue
Open
opened 2026-09-08 05:01:04 +00:00 by rock · 0 comments
rock commented 2026-09-08 05:01:04 +00:00
Owner
Copy Link
Copy Source

Implement three-tier availability tracking.

Metrics:

  • A1: http_requests_total (Counter, labeled method/endpoint/status_code)
  • A2: http_requests_duration_seconds (Histogram, labeled endpoint)
  • A3: http_5xx_total (Counter)
  • A4: http_4xx_total (Counter, labeled status_code)
  • A5: availability_ratio (Gauge) — 1 - (5xx / total), SLO >= 99.9%
  • A6: successful_query_ratio (Gauge) — 200 with results within 500ms, SLO >= 95%
  • A7: health_check_consecutive_failures (Gauge)
  • A8: dependency_up (Gauge, labeled: postgres/opensearch/obsidian/embedding_model)
  • A9: graceful_degradation_total (Counter, labeled degraded_component)
  • A10: circuit_breaker_state (Gauge, labeled backend: 0=closed/1=half-open/2=open)

Tasks:

  1. Add Actix-web middleware for A1/A2 (request counting + latency)
  2. Implement dependency health checker (periodic ping to pg, opensearch, embedding model)
  3. Add circuit breaker pattern for OpenSearch and Obsidian
  4. Compute A5/A6 as recording rules in Prometheus

Reference: docs/OBSERVABILITY.md Section 7

Implement three-tier availability tracking. Metrics: - A1: http_requests_total (Counter, labeled method/endpoint/status_code) - A2: http_requests_duration_seconds (Histogram, labeled endpoint) - A3: http_5xx_total (Counter) - A4: http_4xx_total (Counter, labeled status_code) - A5: availability_ratio (Gauge) — 1 - (5xx / total), SLO >= 99.9% - A6: successful_query_ratio (Gauge) — 200 with results within 500ms, SLO >= 95% - A7: health_check_consecutive_failures (Gauge) - A8: dependency_up (Gauge, labeled: postgres/opensearch/obsidian/embedding_model) - A9: graceful_degradation_total (Counter, labeled degraded_component) - A10: circuit_breaker_state (Gauge, labeled backend: 0=closed/1=half-open/2=open) Tasks: 1. Add Actix-web middleware for A1/A2 (request counting + latency) 2. Implement dependency health checker (periodic ping to pg, opensearch, embedding model) 3. Add circuit breaker pattern for OpenSearch and Obsidian 4. Compute A5/A6 as recording rules in Prometheus Reference: docs/OBSERVABILITY.md Section 7
rock added the memory-observability label 2026-09-08 05:01:04 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
memory-observability
Observability implementation tasks
No labels memory-observability
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
poimen rock (Rock)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: riotpiao-poimen/poimen-memory#33