Okay I got your point. For the ‘binary’ task when I got you right I should use the nn.CrossEntropyLoss() as loss function too because my net outputs is of size [batch_size, 2] right ? But strange that I got really good results withF.sigmoid + BCELoss too.
Thats what I mean with weird. Thats seems not to be what I expect.
x = torch.zeros(1,2)
x[0,0] = -3
x[0,1]= 5
out = F.softmax(Variable(x), dim=1)

Variable containing:
 0.0003  0.9997
[torch.FloatTensor of size 1x2]

I guess you explained it good but my English is not that good and thats the reason I could not get your point in first place.