Hi Meme!



 aynmeme:

This would make BCELoss’s backward method nonlinear with respect to xnx_nxn​ , and using it for things like linear regression would not be straight-forward."
my question is on the last sentence; how does a loss term and its gradient going to infinity translate into to a nonlinear backward method and its incompatibility with linear regression?


I’ve noticed that sentence too.  I think it’s pretty much nonsense, where
either the author was just sloppy or didn’t understand what the words
he was using meant.
The best I can come up with is that he wanted to say something like:
“When BCELoss becomes inf, its backward() method – and the
whole backpropagation – will become polluted with infs and nans,
causing training to fail.  We therefore clamp BCELoss's internal log()
function at -100 to save you from such an ignoble fate.”
Best.
K. Frank