The boxes tensor is performing inplace operations that can be fixed with a .clone() call on the right hand side:
boxes[:, :, 0] = torch.clamp(boxes.clone()[:, :, 0], min=0)
boxes[:, :, 1] = torch.clamp(boxes.clone()[:, :, 1], min=0)
boxes[:, :, 2] = torch.clamp(boxes.clone()[:, :, 2], max=width)
boxes[:, :, 3] = torch.clamp(boxes.clone()[:, :, 3], max=height)

Some more discussion here:

  
    
    
    Autograd runtime error with operations involving ffts autograd
  
  
    Fixed it. Key takeaways in fixing autograd problems like these: 

Use torch.autograd.set_detect_anomaly(True) to isolate the line with the problem.
Call .clone() on the variables on the right hand side of the update.

had to replace the line: 
dx = 2.0 * torch.stack([torch.rfft(x_r[0], 2), torch.rfft(x_r[1], 2)], dim=0)

with: 
dx = 2.0 * torch.stack([torch.rfft(x_r[0].clone(), 2), torch.rfft(x_r[1].clone(), 2)], dim=0)

Found the following threads / blogs useful: 


https://www.yongfengli.tk/20…
  



  
    
    
    Encounter the RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation 
  
  
    I am going to define my layer. How ever, I encounter the RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation while running backward(). 
I found that if i commented the second for loop ‘for j in range(self.number_person):’ or make ‘u_i[:,j,:] = (1 - self.lumbda)*u_i[:,j,:]’, the backward() was fine. 
I wonder where is inplace operation and why it does not work? ‘p_rnn_feature’ and ‘u_sum’ has been compute before. 
BTW, this code is run on p…