Your training seems to get stuck after 3 epochs of training, so you might need to play around with some hyperparameters.
I verified that the loss decreases at the beginning using your code.
However, I’m not sure, why you want to delete the input tensor as well as the intermediate tensors in your forward.