Hi,
As stated in the title, what memory is released by using loss.backward()? Why more memory is used for model.eval().
I know torch.no_grad may be able to reduce the memory for evaluation, but I couldn’t update my pytorch version yet. (I am using pytorch 0.4.0). From the documents, this function should be there, but it cannot be imported.
Is there any other way to release the memory in the way what loss.backward has done?
I found the following one, but it seems only works for parameters not requiring grad.

  
      github.com/pytorch/pytorch
  
  
      
    
  


  Improve autograd memory usage 21



  by apaszke
  on 09:45PM - 26 Feb 17



  1 commits
  changed 4 files
  with 25 additions
  and 7 deletions.


  
  
    
    
  
  


I found the answer from the following thread. It is because of the scoping rules in python and two graphs will be generated.

  
    
    
    Calling loss.backward() reduce memory usage? 
  
  
    It’s because of the scoping rules in Python. When you do this: 
while True:
    loss = model(output)

it will always use 2x the memory that is needed to compute the model, because the reference to the loss form the previous iteration won’t be overwritten (and thus the graph with all the buffers it holds won’t be freed) until this iteration completes. So you’ll effectively end up holding to two graphs. This is why you should use volatile=True inputs when only doing inference. Once you add .backwa…
  


Thanks.