colesbury:


But then it won’t let me allocate memory smaller than a page if that page is already partially used, am I correct?

No – memory smaller than a page may use a partially filled page. It just won’t combine multiple partially filled pages together.


OK, then I could write an app that would go and fill out all the little nooks and crannies and then dump a summary. It sounds like it’d be the easiest to do directly in C to avoid any overheads that might get incurred by using pytorch.



 colesbury:


How would you recommend I then try to diagnose a situation that lead me to start on this path? My idea was to write a sort of memory mapper which I would bisect/splice my code with and through its printouts find the guilty parts that caused that much fragmentation.

I would just record the sizes of the malloc and free calls to a log. (Edit THCCachingAllocator.cpp).


I’m not sure how could that help with detecting fragmentation. Unless you mean looking at mallocs that are lesser than a page size and then accounting the remaining fragments, but that sounds like it’d be very unreliable/difficult to accomplish since it’d require an exact understanding of how CUDA does things internally. Basically this is what I did in my 2nd post, except on the macro-level.



 colesbury:


Does the content of this error come from CUDA or pytorch?

It comes from PyTorch. Cached in this case is cached but not allocated.


I think this is diverging from the topic now, so I will create a new one for that and keep this one focused.