Solved by albanD in post #4 
              
              
                Ho right you can’t use it inplace… The first one is definitely best. 
Or use you original code and check that total_loss.requires_grad before calling backward on it.