Hello Andrei!



 tzeny:

…
Tracing it back I reached the conclusion that sometimes my model outputs very small numbers, for example -136. This leads to:
torch.sigmoid(torch.tensor([-136.])) -> tensor([0.]) which leads to -inf in torch.log().
Is it ok to use torch.clamp(torch.sigmoid(torch.tensor([-136.])), min=1e-8, max=1 - 1e-8) ?


The short (narrow) answer – use LogSigmoid.
Please see my reply to your “Weighted Binary Crossentropy”
post:

  
    
    
    Class Weighed Binary Crossentropy not working, even with equal weights autograd
  
  
    Hi Andrei! 

First, to answer the question I think you’re asking: 
You should be using (as in the comment in your code) 
BCEWithLogitsLoss. 
BCEWithLogitsLoss supports sample weights, which you 
can use for class weights. 
Let’s say you have class weight w_1 for class 1, and w_0 
for class 0.  Let w_n be the sample weight for sample n. 
Simply set w_n = w_1 if y_n = 1, and w_n = w_0 if 
y_n = 0.  (This assumes that the y_n are either 0 or 1, as 
they should be if they are binary class labels.) 
N…
  


Best.
K. Frank