Hille:

My first task is a binary classification task. And for illustration I want to display the image with the associated probability on it. And therefore I need to apply for example F.sigmoid() to the output of the net before I can apply F.softmax() because with only softmax I get wired results for the probabilities. (Thats because of the positive and negative outputs of my net that F.softmax() can’t handle alone)


Softmax can handle logits, i.e. positive and negative values. In your current code snippet, it seems your model output is of dimension [batch_size, 2], which is a multi-class classification applied on a binary task.
In such a case, you should just apply the softmax to get the probabilities for visualization:
logits = torch.randn(4, 2) # your model output
print(logits) # positive and negative values
> tensor([[ 0.0355, -0.6179],
        [ 0.3327,  0.0276],
        [-1.3194, -0.2547],
        [-0.0072,  1.0901]])
prob = F.softmax(logits, dim=1)
print(prob)
> tensor([[ 0.6578,  0.3422],
        [ 0.5757,  0.4243],
        [ 0.2564,  0.7436],
        [ 0.2503,  0.7497]])

As you can see the probabilities sum to one over dim1.
If you would like to use sigmoid and BCELoss, your model output should just return [batch_size, 1].
Could you explain your weird results using softmax?



 Hille:

While training the net for the binary classification I apply sigmoid to the outputs first and then I send them in my loss function which is F.binary_cross_entropy . And then I propagate back.


This would most likely work, but is treating your problem as a multi-label classification task, i.e. the BCE formula is applied on each output of your model.



 Hille:

While training the net for the multiple classification. I send the outputs straight into the loss function nn.CrossEntropyLoss() because that loss function combines nn.LogSoftmax() and nn.NLLLoss() . Because of that combination I thought I don’t have to apply another function before I can send the outputs into the loss function.


That’s perfectly fine and that’s exactly what I’ve tried to explain before. Sorry, if the explanation was a bit unclear.