Answers.Body: <p>First thing to notice, is that the assumptions on the target don't match the ones of multi-classifications: in particular, in multi-class classification, it's generally assumed that any other class outside the target one, is equally bad.</p>
  <p>Instead here, it's clear that this is not true:<br />
  <a href="https://i.sstatic.net/eWbBx.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/eWbBx.png" alt="enter image description here" /></a></p>
  <p>Given an input with target &quot;Healty Apple&quot;, predicting &quot;General Apple Scab&quot; is not as bad as predicting &quot;Serious Cedar Apple Rust&quot;... in other words, the class order counts.</p>
  <p>In order to capture this property, they decide to use classification regression.</p>
  <p>About the automatic threshold, they don't say anything about it on the paper, so in my opinion what they do is to adjust them to improve performance.</p>
  <p>On top of my head, one way is by predicting the regression score for the training set, and then fitting 6 Gaussian distributions (like a naive Bayes model), and adjusting the threshold by moving them so that they best fit the result.... or you can just plot them with different colors and check where the colors lies</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-25T14:52:15.180Z
Answers.Text: Oh alright, I understand now that they are using classification regression because class order is important in this case.
  Can you please also elaborate a little on using 6 Gaussian distributions for adjusting the thresholds
Answers.UserDisplayName: Dawood Ahmad
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-25T17:28:52.740Z
Answers.Text: @DawoodAhmad it's just an idea, but an even easier one might be to do a forward pass of the whole training set, for each class see where the median element is in the number line, and then adjust the threshold separating the space equally
Answers.UserDisplayName: Alberto
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-25T14:10:52.860Z
Answers.LastActivityDate: 2023-07-25T14:10:52.860Z
Answers.UserDisplayName: Alberto
Body: <p>I understand that to solve multilabel classification problems, we can use the softmax activation function in the output layer of the neural network. The softmax function outputs probabilities of each label, and the label with highest probability is then predicted as the target label.
  However, I just saw in a research paper that the authors used regression function instead of softmax function, in output layer. The paper says:</p>
  <blockquote>
  <p>Because regression classification can automatically adjust
  classification thresholds based on data distribution to maximize classification performance</p>
  </blockquote>
  <p>I do not understand how can the model learn classification thresholds by itself?
  Are these thresholds part of the neural network architecture? Are these thresholds trained like weights of layers?</p>
  <p>This is the link of the paper: <a href="https://www.sciencedirect.com/science/article/abs/pii/S016816991931556X" rel="nofollow noreferrer">https://www.sciencedirect.com/science/article/abs/pii/S016816991931556X</a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-24T21:07:05.700Z
Comments.Text: please link the paper
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-25T11:37:05.133Z
Comments.Text: I have added the link to the paper
Comments.UserDisplayName: Dawood Ahmad
Tags: <neural-networks><regression><sigmoid><multiclass-classification>
Title: How can a Regression based Neural Network learn class thresholds?
UserDisplayName: Dawood Ahmad
ThreadText: # How can a Regression based Neural Network learn class thresholds?
  
  (Asked by: username_0 on 2023-07-24)
  
  I understand that to solve multilabel classification problems, we can use the softmax activation function in the output layer of the neural network. The softmax function outputs probabilities of each label, and the label with highest probability is then predicted as the target label.
  However, I just saw in a research paper that the authors used regression function instead of softmax function, in output layer. The paper says:
  
  > Because regression classification can automatically adjust
  > classification thresholds based on data distribution to maximize classification performance
  
  I do not understand how can the model learn classification thresholds by itself?
  Are these thresholds part of the neural network architecture? Are these thresholds trained like weights of layers?
  
  This is the link of the paper: <https://www.sciencedirect.com/science/article/abs/pii/S016816991931556X>
  
  **Tags:** neural-networks, regression, sigmoid, multiclass-classification
  > **Comments:**
  > **username_1**: please link the paper  
  > **username_0**: I have added the link to the paper  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-07-25. Score: 1 | Accepted answer
  
  First thing to notice, is that the assumptions on the target don't match the ones of multi-classifications: in particular, in multi-class classification, it's generally assumed that any other class outside the target one, is equally bad.
  
  Instead here, it's clear that this is not true:  
  [![enter image description here](https://i.sstatic.net/eWbBx.png)](https://i.sstatic.net/eWbBx.png)
  
  Given an input with target "Healty Apple", predicting "General Apple Scab" is not as bad as predicting "Serious Cedar Apple Rust"... in other words, the class order counts.
  
  In order to capture this property, they decide to use classification regression.
  
  About the automatic threshold, they don't say anything about it on the paper, so in my opinion what they do is to adjust them to improve performance.
  
  On top of my head, one way is by predicting the regression score for the training set, and then fitting 6 Gaussian distributions (like a naive Bayes model), and adjusting the threshold by moving them so that they best fit the result.... or you can just plot them with different colors and check where the colors lies
  
  > **Comments:**
  > **username_0**: Oh alright, I understand now that they are using classification regression because class order is important in this case. Can you please also elaborate a little on using 6 Gaussian distributions for adjusting the thresholds  
  > **username_1**: @DawoodAhmad it's just an idea, but an even easier one might be to do a forward pass of the whole training set, for each class see where the median element is in the number line, and then adjust the threshold separating the space equally  
  
  ---
  
  