The number of index chunks currently scales with index vdev size, even when chunk_num is specified. This can lead to excessive memory usage by the allocator, with each chunk consuming around 7MB. In extreme cases, this can result in over 10GB of memory usage, increasing the risk of OOM.
For Example:
In some SH test cases, due to Tess scheduling a HDD to the index PVC, an index vdev size is calculated to be over 6TB, requiring 1.64K chunks, using more than 13GB of memory, and then causing OOM.
Proposal:
- Limit chunk count for index vdev
- Optimize fix allocator to reduce the memory required per chunk.