Answers.Body: <p>By making the problem binary, you are in all likelihood simplifying the decision boundary to be learned. You are not improving your 5-class classification accuracy as much as generating a good 2-class classification accuracy. You cannot really comment on if one accuracy is better than the other as they are not related.</p>
  <p>If you used the 2-class predictions and broke them into further classes until you reached the original 5 classes, you might even end up at your original accuracy.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-20T15:23:32.487Z
Answers.LastActivityDate: 2023-10-20T15:23:32.487Z
Answers.UserDisplayName: Someperson
Answers.Body: <p>Multi-class classifiers have a higher chance to misclassify a class for another that is proportional to the number of classes, say <span class="math-container">$k$</span>. So, in general, when turning a binary classification problem into a multi-class one (for example by adding an extra class), you can expect lower accuracy just because correctly classifing <span class="math-container">$k$</span> classes is harder than <span class="math-container">$2$</span>.</p>
  <p>Don't mistake me, what I mean is that not necessary the multi-class model is worse than the binary one. There are problems in which some classes are similar to others, this may be your case too in which rating 2 and 3, or 4 and 5 can be considered quite similar. In your case, you want to also look at the <em>top-k accuracy</em> (top-3, for example) to see if the correct class is among the top-3 predicted classes.</p>
  <p>You want also to consider that if you sum some of the predicted logits (or class probabilities) you can recover a binary classifier: in principle, you should expect this to be similar in performance than the binary one, although information about the <span class="math-container">$k$</span> classes introduces a different prior while learning - in your case, you should sum logits 0, 1, and 2 to obtain the negative class (rankings below or equal 3), and 3 and 4 for the positive class (or just do 1 - negative class prob.) Moreover, in case of <em>unbalanced classes</em>, fusing two classes or more in one may ease learning (because the imbalance would be mitigated), so increasing performance.</p>
  <p>The last thing you want to consider is to have a binary classifier that performs a first &quot;selection&quot; of data samples. Say you're mostly interested in classifying worse reviews. If so, you can have a binary model that &quot;filters out&quot; the good reviews (also you want to tune the classification threshold in a way that is meaningful, i.e. trading the rate of type-1 and type-2 errors) leaving only the bad ones, which are further classified by a 3-class classifier. Such an approach is not said to be better than directly training a 5-class model, but can help at enhancing the accuracy on the negative reviews, as an example.</p>
  <p>Regarding your question:</p>
  <blockquote>
  <p>Is a model considered good when it classifies data into just two distinct categories, as compared to a multi-class classification model that exhibits lower accuracy.</p>
  </blockquote>
  <p>Again, comparing binary accuracy with the categorical one is not fair: top-k is a little better, but still not the same thing. A better  approach is to consider the rate of error (I mean whatever classification metric you like) obtained by summing over the relevant class probabilities (so that to recover a binary problem) and comparing that with what a binary classifier achieved. For example, you can compare precision/recall on class 0 (provided by a binary classifier) with the same metric but computed by summing over class 0, 1, 2, of the multi-class model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-20T21:00:54.920Z
Answers.LastActivityDate: 2023-10-20T21:00:54.920Z
Answers.UserDisplayName: Luca Anzalone
Body: <p>I've achieved good accuracy in binary classification problems, but I'm facing challenges when it comes to multi-class classification tasks.</p>
  <p>While tackling a natural language processing challenge involving the classification of reviews into ratings on a scale of 1 to 5, I enhanced the performance by redefining the ratings. I assigned a value of 1 for ratings greater than 3 and 0 for ratings less than or equal to 3, resulting in a substantial improvement in accuracy.</p>
  <p>My question is, Is a model considered good when it classifies data into just two distinct categories, as compared to a multi-class classification model that exhibits lower accuracy.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-20T11:02:35.060Z
Comments.Text: I will not comment on your main question about multiclass vs binary, however you could consider continuing down the path you are on now; i.e., further refine the classification by now distinguishing between 4 and 5 or 1 and 2.  In essence, you are building a sort of decision tree of neural networks.
Comments.UserDisplayName: David Hoelzer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-20T16:05:27.203Z
Comments.Text: Would you add the type of decision making models you tried for binary and multi classes dataset? It would help us understand your situation. I think the characteristic of your dataset also import factor.
Comments.UserDisplayName: Cloud Cho
Tags: <machine-learning><data-preprocessing>
Title: Can the multiclass classification be modified to binary classification for better accuracy
UserDisplayName: Raghul Azhagaiah
ThreadText: # Can the multiclass classification be modified to binary classification for better accuracy
  
  (Asked by: username_0 on 2023-10-20)
  
  I've achieved good accuracy in binary classification problems, but I'm facing challenges when it comes to multi-class classification tasks.
  
  While tackling a natural language processing challenge involving the classification of reviews into ratings on a scale of 1 to 5, I enhanced the performance by redefining the ratings. I assigned a value of 1 for ratings greater than 3 and 0 for ratings less than or equal to 3, resulting in a substantial improvement in accuracy.
  
  My question is, Is a model considered good when it classifies data into just two distinct categories, as compared to a multi-class classification model that exhibits lower accuracy.
  
  **Tags:** machine-learning, data-preprocessing
  
  > **Comments:**
  > **username_1**: I will not comment on your main question about multiclass vs binary, however you could consider continuing down the path you are on now; i.e., further refine the classification by now distinguishing between 4 and 5 or 1 and 2.  In essence, you are building a sort of decision tree of neural networks.  
  > **username_2**: Would you add the type of decision making models you tried for binary and multi classes dataset? It would help us understand your situation. I think the characteristic of your dataset also import factor.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2023-10-20. Score: 1
  
  By making the problem binary, you are in all likelihood simplifying the decision boundary to be learned. You are not improving your 5-class classification accuracy as much as generating a good 2-class classification accuracy. You cannot really comment on if one accuracy is better than the other as they are not related.
  
  If you used the 2-class predictions and broke them into further classes until you reached the original 5 classes, you might even end up at your original accuracy.
  
  
  ---
  
  ### Answer by username_4 on 2023-10-20. Score: 1
  
  Multi-class classifiers have a higher chance to misclassify a class for another that is proportional to the number of classes, say $k$. So, in general, when turning a binary classification problem into a multi-class one (for example by adding an extra class), you can expect lower accuracy just because correctly classifing $k$ classes is harder than $2$.
  
  Don't mistake me, what I mean is that not necessary the multi-class model is worse than the binary one. There are problems in which some classes are similar to others, this may be your case too in which rating 2 and 3, or 4 and 5 can be considered quite similar. In your case, you want to also look at the *top-k accuracy* (top-3, for example) to see if the correct class is among the top-3 predicted classes.
  
  You want also to consider that if you sum some of the predicted logits (or class probabilities) you can recover a binary classifier: in principle, you should expect this to be similar in performance than the binary one, although information about the $k$ classes introduces a different prior while learning - in your case, you should sum logits 0, 1, and 2 to obtain the negative class (rankings below or equal 3), and 3 and 4 for the positive class (or just do 1 - negative class prob.) Moreover, in case of *unbalanced classes*, fusing two classes or more in one may ease learning (because the imbalance would be mitigated), so increasing performance.
  
  The last thing you want to consider is to have a binary classifier that performs a first "selection" of data samples. Say you're mostly interested in classifying worse reviews. If so, you can have a binary model that "filters out" the good reviews (also you want to tune the classification threshold in a way that is meaningful, i.e. trading the rate of type-1 and type-2 errors) leaving only the bad ones, which are further classified by a 3-class classifier. Such an approach is not said to be better than directly training a 5-class model, but can help at enhancing the accuracy on the negative reviews, as an example.
  
  Regarding your question:
  
  > Is a model considered good when it classifies data into just two distinct categories, as compared to a multi-class classification model that exhibits lower accuracy.
  
  Again, comparing binary accuracy with the categorical one is not fair: top-k is a little better, but still not the same thing. A better approach is to consider the rate of error (I mean whatever classification metric you like) obtained by summing over the relevant class probabilities (so that to recover a binary problem) and comparing that with what a binary classifier achieved. For example, you can compare precision/recall on class 0 (provided by a binary classifier) with the same metric but computed by summing over class 0, 1, 2, of the multi-class model.
  
  
  ---
  
  