Hi Roy and 杜明軒!



 RoySadaka:

That said, they say that using the official torch.nn.BCEWithLogitsLoss() is better, cause although it’s doing sigmoid and then BCE, it does it in such a way that it is numerically stable (i believe the optimization is on the c++ level)


As a minor note, you can implement your own BCEWithLogitsLoss
with the same numerical benefits as pytorch’s by replacing the
separate calls to sigmoid() and log() with a call to logsigmoid().
Best.
K. Frank