I thought similary but as you can see I tried this in my 3rd. try and this throws runtime exceptions. I think this happens because If i create a tensor with .detach() the gradients are no longer tracked. Although I need this in the next for loop.