Thank you for your reply. Im using torch so the second one applies to my case. Then I guess that the backward operations of the operations used in my loss function do not align with the backward used in here https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/Loss.cpp#L272 1. Does the backward of a division also work with an epsilon?