<p>This is multi-label classification, which means you have two binary classification problems, one for each of your classes. This is different than multi-class classification.</p>
<p>For this use binary cross-entropy loss, with binary encoded labels (0/1), not one-hot encoded labels, and put a sigmoid activation at the output with two neurons (one for each class).</p>

CC BY-SA 4.0
2023-03-19T12:19:59.683Z
Thank you. To make this work optimally, is there any issue that both classes occur very scarcely in the training set? I.e. very unbalanced? Would something like FocalLoss be a solution? I can create a third 'normal' class as class 1 and 2 are mutually exclusive. Trying to find what would perform best.
dorien
CC BY-SA 4.0
2023-03-19T02:59:25.817Z
2023-03-19T02:59:25.817Z
Dr. Snoopy
<p>I have a neural network that predicts 2 classes of a time series (bottom and top). Currenlty my Y labels are size 2: [1 0] for bottom and [0 1] for top. The NN has 2 output nodes.</p>
<p>Of course not every instance is a bottom or top. (Overwhelmingly) most of the time we have no bottom and no top.</p>
<p>Should I change my network (and Y labels) to have 3 outputs?</p>
<p>And what would be appropriate as a loss function. I guess with 3 outputs I could use entropy or focal loss. With 2 outputs I don't think entropy is correct.</p>
<p>Currently I use PyTorch (lightening), and I have:</p>
<pre><code>self.lstm = nn.LSTM(input_size=n_features, 
                            hidden_size=hidden_size,
                            num_layers=num_layers, 
                            dropout=dropout, 
                            batch_first=True)
self.linear = nn.Linear(hidden_size, 2)
        


def forward(self, x):
        lstm_out, _ = self.lstm(x)
        y_pred = self.linear(lstm_out[:,-1])
        return y_pred

criterion = nn.CrossEntropyLoss()
</code></pre>

<neural-networks><classification><objective-functions><pytorch>
Which loss / activation function with 2 classes that do not occur often and do not sum to one?
dorien
# Which loss / activation function with 2 classes that do not occur often and do not sum to one?

(Asked by: username_0 on 2023-03-18)

I have a neural network that predicts 2 classes of a time series (bottom and top). Currenlty my Y labels are size 2: [1 0] for bottom and [0 1] for top. The NN has 2 output nodes.
Of course not every instance is a bottom or top. (Overwhelmingly) most of the time we have no bottom and no top.
Should I change my network (and Y labels) to have 3 outputs?
And what would be appropriate as a loss function. I guess with 3 outputs I could use entropy or focal loss. With 2 outputs I don't think entropy is correct.
Currently I use PyTorch (lightening), and I have:
self.lstm = nn.LSTM(input_size=n_features, 
                            hidden_size=hidden_size,
                            num_layers=num_layers, 
                            dropout=dropout, 
                            batch_first=True)
self.linear = nn.Linear(hidden_size, 2)
        


def forward(self, x):
        lstm_out, _ = self.lstm(x)
        y_pred = self.linear(lstm_out[:,-1])
        return y_pred

criterion = nn.CrossEntropyLoss()

**Tags:** neural-networks, classification, objective-functions, pytorch


---

## 1 answers

### Answer by username_1 on 2023-03-19. Score: 1 | Accepted answer

This is multi-label classification, which means you have two binary classification problems, one for each of your classes. This is different than multi-class classification.

For this use binary cross-entropy loss, with binary encoded labels (0/1), not one-hot encoded labels, and put a sigmoid activation at the output with two neurons (one for each class).

> **Comments:**
> **username_0**: Thank you. To make this work optimally, is there any issue that both classes occur very scarcely in the training set? I.e. very unbalanced? Would something like FocalLoss be a solution? I can create a third 'normal' class as class 1 and 2 are mutually exclusive. Trying to find what would perform best.  

---

