@LuckyIceland @Umais @abhinavdhere see this issue which gives more details

  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      OOM error where ~50% of the GPU RAM cannot be utilised/reserved 385
    

    
      
        opened 
        
          
        
        Apr 2, 2020
      
      


      
        
          
          david-macleod
        
      
    
  



  🐛 Bug
I am experiencing the classic out of memory error where my GPU appears to have a lot of memory still...



    module: cuda
    topic: memory usage
    triaged


  
  
    
    
  
  


It seems that “reserved in total” is memory “already allocated” to tensors + memory cached by PyTorch. When a new block of memory is requested by PyTorch, it will check if there is sufficient memory left in the pool of memory which is not currently utilized by PyTorch (i.e. total gpu memory - “reserved in total”). If it doesn’t have enough memory the allocator will try to clear the cache and return it to the GPU which will lead to a reduction in “reserved in total”, however it will only be able to clear blocks on memory in the cache of which no part is currently allocated. If any of the block is allocated to a tensor it won’t be able to return it to GPU. Thus you can have scenarios where the tensor allocated memory + the gpu free memory is much less than the gpu total memory, because the cache is holding some which it cannot release