Hello Doubt -
This may be a duplicate; see below*.



 DoubtWang:

I’m very confused the difference between cross-entropy loss or log likelihood loss when dealing with Multi-Class Classification (including Binary Classification) or Multi-Label Classification ?
Could you explain the difference ?
Thanks.


The cross-entropy loss and the (negative) log-likelihood are
the same in the following sense:
If you apply Pytorch’s CrossEntropyLoss to your output layer,
you get the same result as applying Pytorch’s NLLLoss to a
LogSoftmax layer added after your original output layer.
(I suspect – but don’t know for a fact – that using
CrossEntropyLoss will be more efficient because it
can collapse some calculations together, and doesn’t
introduce an additional layer.)
You are trying to maximize the “likelihood” of your model
parameters (weights) having the right values.  Maximizing
the likelihood is the same as maximizing the log-likelihood,
which is the same as minimizing the negative-log-likelihood.
For the classification problem, the cross-entropy is the
negative-log-likelihood.  (The “math” definition of cross-entropy
applies to your output layer being a (discrete) probability
distribution.  Pytorch’s CrossEntropyLoss implicitly adds
a soft-max that “normalizes” your output layer into such a
probability distribution.)
Wikipedia has some explanation of the equivalence of
negative-log-likelihood and Cross entropy 464.
*Possible duplicate:

  
    
    
    Is log_softmax + NLLLoss == CrossEntropyLoss? 
  
  
    If I’m not missing something, they should be the same. However, I tried the follow snippet, but they are not equal. 
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import numpy as np
import torch
import torch.nn.functional as F
from torch import nn
from torch.autograd import Variable


class Net(nn.Module):

    def __init__(self, n_features, n_hiddens, n_classes):
        super(Net, self).__init__()
        self.gru = torch.nn.GRU(n_features, n_hiddens)
        self.linear = torch.nn.Linear(n_…
  


Best.
K. Frank