Adding a small eps value might work.



 chantk:

but that would produce some difference with the value calculated with nn.BCELoss


Note that nn.BCELoss also clamps its log function outputs as described in the docs 8:

For one, if either y_n = or (1 - y_n) = 0, then we would be multiplying 0 with infinity. Secondly, if we have an infinite loss value, then we would also have an infinite term in our gradient, since […]
Our solution is that BCELoss clamps its log function outputs to be greater than or equal to -100. This way, we can always have a finite loss value and a linear backward method.