Hi all,
I recognized that my code consumes more and more memory with each epoch that I am training my model. I have tracked the memory leak down by commenting out parts of my code and its seems that torch.choleksy is the culprit (If i only comment out the line with torch.cholesky my code has no increase in memory usage).
I am training on CPU, so this is not an issure of the GPU memory but of the actual main memory of the pc. I have seen that in the past there has already been problems with memory leakage and torch.choleksy, however, with respect to gpu ram:

  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      Memory leak in torch.cholesky 2
    

    
      
        opened 
        
          
        
        Dec 26, 2018
      
      

        
          closed 
        
          
        
        Feb 7, 2019
      
        

      
        
          
          ozzzp
        
      
    
  



  🐛 Bug
There exists memory leak of GPU memory in torch.cholesky
Code
import torch
def main():
 def test_cholesky():
 A = torch.rand([4, 1280, 1280]).cuda()
 H =...





  
  
    
    
  
  


Unfortunately, my model is far too large to provide the code here. So I just wanted to ask if someone has had similar issues?
I check the memory consumption with:
77 def memory_usage_psutil():
78     # return the memory usage in MB
79     process = psutil.Process(os.getpid())
80     mem = process.memory_info().rss / float(10 ** 6) # /float(2 ** 20)
81     return mem