Thanks for the info!
I tried to debug your code and stumbled upon this line of code 5. You are detaching image_energy by calling .item(). Later in your get_loss method, you are using outputs to calculate the gradients.
However, since image_energy was detached, the computation graph shouldn’t compute any valid gradients before this point.
I’m not completely understanding the code, so let me know, if I’m on the wrong path.