Following up on Unable to allocate cuda memory, when there is enough of cached memory 71, while there is no way to defrag nvidia GPU RAM, is there a way to get the memory allocation map? I’m asking in the simple context of just having one process using the GPU exclusively.
Using free memory info from nvml can be very misleading due to fragmentation, so it would be useful to be able to have some sort of estimation of fragmentation, e.g. 1GB free, 0.3G of which is in chunks less than 50MB.
And it would help to optimize code by comparing fragmentation maps before and after it’s run.
I spent quite some time reading nvidia forums but couldn’t find anything useful, perhaps someone here has knowledge of any resources that could be useful to do diagnostics for gpu ram fragmentation.
And a question about pytorch gpu ram allocation process - does pytorch have a way to choose which free segment to use? e.g. given the free memory list sequence is (a) 200MB (b) 50MB  and pytorch needs to allocate 20MB - will it search for the smallest free chunk that can fit 20MB and pick (b), or will it pick the first available chunk that fits the requirement (a)?
Thank you.