Hi Piotr,
Thank you very much for your quick reply! I had been stuck on this runtime error for two days! Glad to have some pointers.

No, this would only be needed, if you really want to calculate gradients in the input (e.g. for adversarial attacks).

But in my custom dataset class, I specifically set inputs’s require_grad=False. If I don’t reset it to True before passing into the network, doesn’t it mean that there will be no grad_fn being tracked?

Are you calling backward() on this averaged loss? If so, this won’t work since you are recreating a new tensor and will thus break the computation graph. Try to use torch.stack or torch.cat instead.

Yes, I believe the backward() is called. The idea of using an average loss is because in the case where batch_size >= 2, the network architecture will be analyzing multiple batches at once, hence multiple losses.
I assumed that there is no way that we can call backward() on these losses one by one, hence I took an average loss. What’s your suggestions? If we “return” a vector of three losses (say we have batch_size = 3), how is backward() being called on this vector of losses?
return torch.mean(torch.FloatTensor(loss))
To be more specific about this line, “loss” is a list of Tensor floats resulting from the map() function. Then we convert it to a Torch Float Tensor vector and then we take average.
Looking forward to your reply! Many thanks!