Hi,
I’m facing the issue when I want to do backward() with 2 models, action_model and value_model. I’ve already searched related topic. They said that ‘pytorch 1.15’  always automatically check the ‘inplace’ when using backward(). However, it still report the same problem. How can I do backward() without missing previous parameter in model.
Thanks~~

  
      github.com/pytorch/pytorch
  
  
    
  
	  
  

  
    
      RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation? 196
    

    
      
        opened 
        
          
        
        May 28, 2020
      
      

        
          closed 
        
          
        
        Jun 1, 2020
      
        

      
        
          
          honeytidy
        
      
    
  



  I am using pytorch-1.5 to do some gan test. My code is very simple gan code which just fit the sin(x)...



    module: autograd
    triaged


  
  
    
    
  
  


This is error message:
File "train_test.py", line 234, in <module>
    flatten_imaginated_gru_hiddens).view(imagination_horizon+1, -1)
  File "/home/jeff/VirtualEnv/openai/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/jeff/dockers/pick_and_place/models.py", line 168, in forward
    state_value = self.fc4(hidden)
  File "/home/jeff/VirtualEnv/openai/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/jeff/VirtualEnv/openai/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 87, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/jeff/VirtualEnv/openai/lib/python3.7/site-packages/torch/nn/functional.py", line 1610, in linear
    ret = torch.addmm(bias, input, weight.t())
 (print_stack at /pytorch/torch/csrc/autograd/python_anomaly_mode.cpp:60)
Traceback (most recent call last):
  File "train_test.py", line 255, in <module>
    action_loss.backward()
  File "/home/jeff/VirtualEnv/openai/lib/python3.7/site-packages/torch/tensor.py", line 198, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/jeff/VirtualEnv/openai/lib/python3.7/site-packages/torch/autograd/__init__.py", line 100, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [400, 1]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

The problems are from action_model and value_model when I want to do backward() :
imaginated_values = value_model(flatten_imaginated_states,
                                          flatten_imaginated_gru_hiddens).view(imagination_horizon+1, -1)
         
## compute lambda target
lambda_target_values = utils.lambda_target(imaginated_rewards, imaginated_values,
                                                   gamma, lambda_)

## update_value model
value_loss = 0.5 * mse_loss(imaginated_values, lambda_target_values.detach())
value_optimizer.zero_grad()
value_loss.backward(retain_graph=True) 
#--------------------------------------------------------------
 # retain_graph=True : 
# reserve the parameters in the process of backpropagation 
 # when encountering two more outputs from the forward processing.
#--------------------------------------------------------------
clip_grad_norm_(value_model.parameters(), clip_grad_norm)
value_optimizer.step()

 ## update action model (multiply -1 for gradient ascent)
action_loss = -1*(lambda_target_values.mean())
action_optimizer.zero_grad()
action_loss.backward()
clip_grad_norm_(action_model.parameters(), clip_grad_norm)
action_optimizer.step()