So I have simple test network for MNIST data as following
def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 80)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.XXXX(x)

I am using NLLLoss function to implement cross entropy loss explicitly
I wanted to understand how different act function affects the accuracy,
so I tried LogSoftmax and verify that the network trains but for some reason when I used LogSigmoid, the network fails to train.
(note that NLLLoss expects log probability)
Since softmax and sigmoid both have output value between 0 and 1.
I thought there shouldn’t be an issue.
Can anyone explain the detail I am not catching here?