Hi S.!



 S_M:

        P=nn.CrossEntropyLoss(softmax_out1,softmax_out2)



The short answer is that you have to write your own cross-entropy
function to do what you want – see below.
There are two things going on here:
First, as Aman noted, the input to CrossEntropyLoss (your
softmax_out1) should be raw-score logits that range from -inf to
+inf, rather than probabilities that range from 0.0 to 1.0.  So you
want to pass logits in as the input without converting them to
probabilities by running them through softmax().
Second, CrossEntropyLoss expects its target (your softmax_out2)
to be integer class labels (with shape [nBatch], rather than
[nBatch, nClass]).  So CategoricalCrossEntropyWithLogitsLoss
might be a better (if lengthier) name for this loss function.
Now, how to do what you want:
Even if you write your own cross-entropy loss function, you do not
want to pass in probabilities for your input as doing so  will be less
numerically stable than passing in logits.
It does, however, make sense to use probabilities (rather than integer
class labels) for your target.  (These are sometimes called soft labels
or soft targets.)  It’s just that pytorch doesn’t offer such a version of
cross entropy.
The following post shows how to implement such a “soft cross-entropy”
loss.  It takes logits for its input (for numerical stability) and takes
probabilities for its “soft-label” target:

  
    
    
    Soft Cross Entropy Loss (TF has it does Pytorch have it) 
  
  
    Hello Raaj! 

I do not believe that pytorch has a “soft” cross-entropy function built in. 
But you can implement it using pytorch tensor operations, so you should 
get the full benefit of autograd and gpu acceleration. 
See this (pytorch version 0.3.0) script: 
import torch
torch.__version__

# define "soft" cross-entropy with pytorch tensor operations
def softXEnt (input, target):
    logprobs = torch.nn.functional.log_softmax (input, dim = 1)
    return  -(target * logprobs).sum() / input.shap…
  


Best.
K. Frank