Hello,
I want to check if my model is calculating the gradient ok or not.
I have used nn.BCELoss for the loss function. Based on the Pytorch Website, it is mentioned that the output of the loss will be clipped to -100 if it is smaller than this value.
My question is that what will happen in the backward? If the y is zero, because of the gradient it will be in denominator, and the gradient would be inf. There is no explanation about the backward in the site. Can you explain how should I calculate it? Should I add it with epsilon? or …
Also, I have seen a way for getting the gradient of loss respect to the prediction is using hook, but there are small examples and I do not know how should I do that? Can someone say what is the procedure?
Thanks,