fix(monitoring): cap Prometheus retention below the PVC size #68

2026-10-25T04:43:13Z

rock commented

2026-10-25 04:43:13 +00:00

retentionSize was 95GB against a 25Gi volume, so the limit never fired and the volume filled. Prometheus then crashlooped on write /prometheus/queries.active: no space left on device.

Sets 18GB, keeping ~7GiB headroom for WAL and block overhead.

`retentionSize` was `95GB` against a `25Gi` volume, so the limit never fired and the volume filled. Prometheus then crashlooped on `write /prometheus/queries.active: no space left on device`. Sets 18GB, keeping ~7GiB headroom for WAL and block overhead.

rock added 1 commit 2026-10-25 04:43:14 +00:00

fix(monitoring): cap Prometheus retention below the PVC size b7fc6ed60b

retentionSize was 95GB against a 25Gi volume, so the limit never fired. The
volume filled, and Prometheus crashlooped with:

  write /prometheus/queries.active: no space left on device
  failed to initialize active query tracker

18GB keeps roughly 7GiB of headroom for the WAL and block overhead.

rock merged commit 7251032081 into main

2026-10-25 04:43:23 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: riotpiao-poimen/homelab#68