<p>For a sequence labeling task (NER), we compute the loss by passing the softmax distribution of the classes (e.g. vocabulary) with the gold label to the loss function (<code>nn.CrossEntropyLoss(y, x)</code>):</p>
<pre><code>y = tensor([[ 1.4775,  0.7022,  0.7499, -0.7535, -1.4983, -2.3193, -0.6166, -2.3302,
     -0.8847, -0.1915]])
</code></pre>
<p>label:</p>
<pre><code>x = torch.tensor([2])
</code></pre>
<p>what if we don't have the softmax distribution and we have the predicted label instead, how can we compute the loss? Following the example above, 2 is given for prediction instead of <code>y</code>.</p>

CC BY-SA 4.0
2022-01-06T22:33:30.970Z
I guess you will need to assume that the predicted class has a probability $1$ and all other classes have probability $0$, unless you have more info. Can you clarify why the predicted classes/labels in your case are hard rather than soft (probabilities)?
nbro
CC BY-SA 4.0
2022-01-07T17:28:42.463Z
Thanks @nbro .. this is what I actually have done, but I felt that this will confuse the model during training when the predicted label is not the correct one because all the other classes will be treated equally (zeros).
Minions
<pytorch><sequence-modeling><loss>
How to compute the loss for a sequence labeling task without the Softmax distribution?
Minions
# How to compute the loss for a sequence labeling task without the Softmax distribution?

(Asked by: username_0 on 2022-01-06)

For a sequence labeling task (NER), we compute the loss by passing the softmax distribution of the classes (e.g. vocabulary) with the gold label to the loss function (nn.CrossEntropyLoss(y, x)):
y = tensor([[ 1.4775,  0.7022,  0.7499, -0.7535, -1.4983, -2.3193, -0.6166, -2.3302,
     -0.8847, -0.1915]])

label:
x = torch.tensor([2])

what if we don't have the softmax distribution and we have the predicted label instead, how can we compute the loss? Following the example above, 2 is given for prediction instead of y.

**Tags:** pytorch, sequence-modeling, loss

> **Comments:**
> **username_1**: I guess you will need to assume that the predicted class has a probability $1$ and all other classes have probability $0$, unless you have more info. Can you clarify why the predicted classes/labels in your case are hard rather than soft (probabilities)?  
> **username_0**: Thanks @username_1 .. this is what I actually have done, but I felt that this will confuse the model during training when the predicted label is not the correct one because all the other classes will be treated equally (zeros).  

---

## 0 answers

