Body: <p>For a sequence labeling task (NER), we compute the loss by passing the softmax distribution of the classes (e.g. vocabulary) with the gold label to the loss function (<code>nn.CrossEntropyLoss(y, x)</code>):</p>
  <pre><code>y = tensor([[ 1.4775,  0.7022,  0.7499, -0.7535, -1.4983, -2.3193, -0.6166, -2.3302,
       -0.8847, -0.1915]])
  </code></pre>
  <p>label:</p>
  <pre><code>x = torch.tensor([2])
  </code></pre>
  <p>what if we don't have the softmax distribution and we have the predicted label instead, how can we compute the loss? Following the example above, 2 is given for prediction instead of <code>y</code>.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-06T22:33:30.970Z
Comments.Text: I guess you will need to assume that the predicted class has a probability $1$ and all other classes have probability $0$, unless you have more info. Can you clarify why the predicted classes/labels in your case are hard rather than soft (probabilities)?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-07T17:28:42.463Z
Comments.Text: Thanks @nbro .. this is what I actually have done, but I felt that this will confuse the model during training when the predicted label is not the correct one because all the other classes will be treated equally (zeros).
Comments.UserDisplayName: Minions
Tags: <pytorch><sequence-modeling><loss>
Title: How to compute the loss for a sequence labeling task without the Softmax distribution?
UserDisplayName: Minions
ThreadText: # How to compute the loss for a sequence labeling task without the Softmax distribution?
  
  (Asked by: username_0 on 2022-01-06)
  
  For a sequence labeling task (NER), we compute the loss by passing the softmax distribution of the classes (e.g. vocabulary) with the gold label to the loss function (`nn.CrossEntropyLoss(y, x)`):
  
  ```
  y = tensor([[ 1.4775,  0.7022,  0.7499, -0.7535, -1.4983, -2.3193, -0.6166, -2.3302,
       -0.8847, -0.1915]])
  
  ```
  
  label:
  
  ```
  x = torch.tensor([2])
  
  ```
  
  what if we don't have the softmax distribution and we have the predicted label instead, how can we compute the loss? Following the example above, 2 is given for prediction instead of `y`.
  
  **Tags:** pytorch, sequence-modeling, loss
  
  > **Comments:**
  > **username_1**: I guess you will need to assume that the predicted class has a probability $1$ and all other classes have probability $0$, unless you have more info. Can you clarify why the predicted classes/labels in your case are hard rather than soft (probabilities)?  
  > **username_0**: Thanks @username_1 .. this is what I actually have done, but I felt that this will confuse the model during training when the predicted label is not the correct one because all the other classes will be treated equally (zeros).  
  
  ---
  
  ## 0 answers
  
  