riotpiao-poimen/poimen-memory
2
0
Fork 0
Code Issues 13 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

O5: Instrument write volume and storage metrics (W1-W12) #31

New Issue
Open
opened 2026-09-08 05:00:37 +00:00 by rock · 0 comments
rock commented 2026-09-08 05:00:37 +00:00
Owner
Copy Link
Copy Source

Track storage growth, parity drift, and compaction effectiveness.

Metrics:

  • W1: storage_pgvector_rows_total (Gauge)
  • W2: storage_pgvector_bytes (Gauge)
  • W3: storage_opensearch_docs_total (Gauge)
  • W4: storage_opensearch_bytes (Gauge)
  • W5: storage_parity_drift (Gauge) — abs(W1 - W3), target = 0
  • W6: write_rate_per_hour (Gauge)
  • W7: write_rate_per_project (Gauge, labeled)
  • W8: storage_level_distribution (Gauge, labeled L0/L1/L2/R)
  • W9: compaction_runs_total (Counter)
  • W10: compaction_dedup_removed_total (Counter)
  • W11: compaction_stale_gc_removed_total (Counter)
  • W12: compaction_space_freed_bytes (Counter)

Implement periodic collector (every 60s) querying pg and OpenSearch for counts/sizes.

Reference: docs/OBSERVABILITY.md Section 5

Track storage growth, parity drift, and compaction effectiveness. Metrics: - W1: storage_pgvector_rows_total (Gauge) - W2: storage_pgvector_bytes (Gauge) - W3: storage_opensearch_docs_total (Gauge) - W4: storage_opensearch_bytes (Gauge) - W5: storage_parity_drift (Gauge) — abs(W1 - W3), target = 0 - W6: write_rate_per_hour (Gauge) - W7: write_rate_per_project (Gauge, labeled) - W8: storage_level_distribution (Gauge, labeled L0/L1/L2/R) - W9: compaction_runs_total (Counter) - W10: compaction_dedup_removed_total (Counter) - W11: compaction_stale_gc_removed_total (Counter) - W12: compaction_space_freed_bytes (Counter) Implement periodic collector (every 60s) querying pg and OpenSearch for counts/sizes. Reference: docs/OBSERVABILITY.md Section 5
rock added the memory-observability label 2026-09-08 05:00:38 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
memory-observability
Observability implementation tasks
No labels memory-observability
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
poimen rock (Rock)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: riotpiao-poimen/poimen-memory#31