O6: Pod resource observability (P1-P13) #32

Open
opened 2026-09-08 05:00:50 +00:00 by rock · 0 comments
Owner

Ensure pod-level resource metrics are collected and alerted on.

Tasks:

  1. Verify kube-state-metrics and cAdvisor are scraping memory pod
  2. Set resource requests/limits in deployment manifest (baseline: 2Gi memory, 1 CPU)
  3. Create Grafana panels for:
    • P1: container_memory_working_set_bytes (with limit line overlay)
    • P5: container_cpu_usage_seconds_total (rate, with limit line)
    • P6: container_cpu_throttled_seconds_total
    • P11: kube_pod_container_status_restarts_total
    • P12/P13: container_network_receive/transmit_bytes_total
  4. Create alerts:
    • Memory pressure P1/P4 > 0.85 for 5min (Critical)
    • Pod restart P11 increments (Critical)
    • CPU throttle rate(P6) > 0 for 10min (Warning)
    • Memory monotonic growth over 24h (Warning — leak detection)

Reference: docs/OBSERVABILITY.md Section 6

Ensure pod-level resource metrics are collected and alerted on. Tasks: 1. Verify kube-state-metrics and cAdvisor are scraping memory pod 2. Set resource requests/limits in deployment manifest (baseline: 2Gi memory, 1 CPU) 3. Create Grafana panels for: - P1: container_memory_working_set_bytes (with limit line overlay) - P5: container_cpu_usage_seconds_total (rate, with limit line) - P6: container_cpu_throttled_seconds_total - P11: kube_pod_container_status_restarts_total - P12/P13: container_network_receive/transmit_bytes_total 4. Create alerts: - Memory pressure P1/P4 > 0.85 for 5min (Critical) - Pod restart P11 increments (Critical) - CPU throttle rate(P6) > 0 for 10min (Warning) - Memory monotonic growth over 24h (Warning — leak detection) Reference: docs/OBSERVABILITY.md Section 6
rock added the memory-observability label 2026-09-08 05:00:50 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: riotpiao-poimen/poimen-memory#32