Skip to content

Commit

Permalink
docs: update KubePersistentVolumeFillingUp with prerequisite for stor…
Browse files Browse the repository at this point in the history
…e-gateways
  • Loading branch information
NickAnge committed Nov 18, 2024
1 parent 645e317 commit 657ab23
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions docs/sources/mimir/manage/mimir-runbooks/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1411,6 +1411,15 @@ How to **investigate** and **fix** it:
- Check the number of in-memory series shown on the `Mimir / Tenants` dashboard for an approximation of the number of series that will be compacted once these blocks are shipped from ingesters.
- Check the configured `compactor_split_and_merge_shards` for the tenant. A reasonable rule of thumb is 8-10 million series per compactor shard - if the number of series per shard is above this range, increase `compactor_split_and_merge_shards` for the affected tenant(s) accordingly.
##### Persistent Volume Resizing Prequisite - store-gateways
In case persistent volume resizing is needed for store-gateways, ensure the following steps are taken before proceeding, if automatic downscaling and/or autoscaling, have been enabled:
- Disable automatic downscaling for store-gateways.
- Set autoscaling min/max limits for the Zone-A Horizontal Pod Autoscaler (HPA) based on current usage to ensure that the minimum number of replicas matches the current count and does not decrease.
Both of these steps are necessary to prevent any downscaling issues during the resizing process.
## Mimir ingest storage (experimental)
This section contains runbooks for alerts related to experimental Mimir ingest storage.
Expand Down

0 comments on commit 657ab23

Please sign in to comment.