Dear community,
I am trying to use the weights for the binary classification problem for CrossEntropyLoss and by now I am so lost in it….
In my network I set the output size as 1 and have sigmoid activation function at the end to ensure I get values between 0 and 1. I assume it is probability in my case. If output is set as 2 (for class 0 and 1) then for some reason the sum of the columns does not equal 1. This is why I set the output size as 1.
Loss function requires two columns in the output I assume for class 0 and class 1 but which order I am nor sure. I set weights as tensor([1., 5.]) assuming that I have five times more class 0 than class 1.
Thus in the training loop I have:
outputs = nn_model(X_batch)
I think it is probability for class0. Output of shape 1.
outputs = outputs.view(X_batch.shape[1]*X_batch.shape[0],output_size)
I reshape it to one column.
one = torch.ones(outputs.shape)
I create a tensor of ones
class1 = one-outputs
I calculate probability of  class1 by substracting probability of class 0 from 1.
outputs = torch.cat((outputs,two), 1)
I concatenate having 1st column prob class 0 2nd column prob class 1.
loss = criterion(outputs, labels)
I assume 1st col is class 0 2nd class1, so weights are tensor([1., 5.]) showing that I have five times more zeroes.
Is this right? if wrong then please help me to figure out what and why, PLEASE… Any comments are very much appreciated.