Answers.Body: <p>Welcome to AI.SE @Par!</p>
  
  <p>What you have might be either a <em>multi-label</em> or a <em>multi-class</em> classification problem. If the classes are disjoint (each example belongs to just 1 of the 50 classes), it's a multi-class problem. If not (so each example can belong to several classes at once), it's a multilabel problem.</p>
  
  <p>Multi-label classification is usually handled by training a separate model for each of the labels. If you want to label a new point, you then ask each model what it thinks, and assign the union of the labels that the models suggest together. An alternative approach is to use something like a neural network, which can have many outputs. You can then have one output neuron for each possible label.</p>
  
  <p>Multi-class classification <em>can</em> be addressed by using the multilabel techniques, but this is usually not a good idea. The three main approaches that are used are "one-v-one", "one-v-all", and "many-v-many".</p>
  
  <ol>
  <li>In 1-v-1, we train one model to discriminate each <em>pair</em> of classes (n(n+1)/2 models in total for n classes). To classify a new point, we ask each model which class it belongs to and assign a "vote" to the class the model returns. The class with the most votes overall is selected as the label for the new point. In case of a tie, we can report several possible answers to the user.</li>
  <li>In 1-v-all, we train one model to discriminate each class from <em>all the other classes</em> (n models in total for n classes). To classify a new point, we ask each model whether it belongs to the model's primary class or not. Ideally, just one model claims the new point. If more than one does, we can report a tie, or use some notion of classifier confidence to select a winner.</li>
  <li>In many-v-many, we train <em>k</em> models each of which is tasked with discriminating some subset of the classes from all the rest. To classify a new point, we ask all of these models to label the point. We then pick the class that that is most consistent with the results of all the models. This can also be done by deliberately constructing the models to form an <a href="https://en.wikipedia.org/wiki/Error_correction_code" rel="nofollow noreferrer">error correcting code</a>. </li>
  </ol>
  
  <p>So which approach should you use? Well, Rifkin &amp; Klautau's 2004 <a href="http://www.jmlr.org/papers/volume5/rifkin04a/rifkin04a.pdf" rel="nofollow noreferrer">JMLR paper</a> argues convincingly that the answer is to use one-versus-all classification. This is also pretty easy to do in most packages. For example, in Python's ScikitLearn, you can do it with <a href="https://scikit-learn.org/stable/modules/generated/sklearn.multiclass.OneVsRestClassifier.html" rel="nofollow noreferrer">OneVsRest</a>. If you're not sure what to try, that's probably a safe bet.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-29T20:36:41.443Z
Answers.LastActivityDate: 2019-03-29T20:36:41.443Z
Answers.UserDisplayName: John Doucette
Body: <p>I am working on a supervised machine learning problem where I have more than 10 probably 50 or 100 predicting label categories. Which type of model can be used to work on this type of problem in anaconda python.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-28T21:48:07.623Z
Comments.Text: What do you mean by predicting categories? Do you mean that you have 50 (or more) categories, that is, that a data point in your domain can be belong to one of 50 (or more) categories? Or do you mean that you have 50 (or more) features? Anyway, I think you should describe more in detail your problem.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-29T01:54:16.127Z
Comments.Text: Please, be more specific: if you have all your data already categorized, it would fall under supervised learning; if you don't know the categories, unsuper learning
Comments.UserDisplayName: JCP
Tags: <machine-learning><python><data-science>
Title: What types of machine learning model would fit?
UserDisplayName: Par
ThreadText: # What types of machine learning model would fit?
  
  (Asked by: username_0 on 2019-03-28)
  
  I am working on a supervised machine learning problem where I have more than 10 probably 50 or 100 predicting label categories. Which type of model can be used to work on this type of problem in anaconda python.
  
  **Tags:** machine-learning, python, data-science
  
  > **Comments:**
  > **username_1**: What do you mean by predicting categories? Do you mean that you have 50 (or more) categories, that is, that a data point in your domain can be belong to one of 50 (or more) categories? Or do you mean that you have 50 (or more) features? Anyway, I think you should describe more in detail your problem.  
  > **username_2**: Please, be more specific: if you have all your data already categorized, it would fall under supervised learning; if you don't know the categories, unsuper learning  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-03-29. Score: 1
  
  Welcome to AI.SE @username_0!
  
  What you have might be either a *multi-label* or a *multi-class* classification problem. If the classes are disjoint (each example belongs to just 1 of the 50 classes), it's a multi-class problem. If not (so each example can belong to several classes at once), it's a multilabel problem.
  
  Multi-label classification is usually handled by training a separate model for each of the labels. If you want to label a new point, you then ask each model what it thinks, and assign the union of the labels that the models suggest together. An alternative approach is to use something like a neural network, which can have many outputs. You can then have one output neuron for each possible label.
  
  Multi-class classification *can* be addressed by using the multilabel techniques, but this is usually not a good idea. The three main approaches that are used are "one-v-one", "one-v-all", and "many-v-many".
  
  1. In 1-v-1, we train one model to discriminate each *pair* of classes (n(n+1)/2 models in total for n classes). To classify a new point, we ask each model which class it belongs to and assign a "vote" to the class the model returns. The class with the most votes overall is selected as the label for the new point. In case of a tie, we can report several possible answers to the user.
  2. In 1-v-all, we train one model to discriminate each class from *all the other classes* (n models in total for n classes). To classify a new point, we ask each model whether it belongs to the model's primary class or not. Ideally, just one model claims the new point. If more than one does, we can report a tie, or use some notion of classifier confidence to select a winner.
  3. In many-v-many, we train *k* models each of which is tasked with discriminating some subset of the classes from all the rest. To classify a new point, we ask all of these models to label the point. We then pick the class that that is most consistent with the results of all the models. This can also be done by deliberately constructing the models to form an [error correcting code](https://en.wikipedia.org/wiki/Error_correction_code).
  
  So which approach should you use? Well, Rifkin & Klautau's 2004 [JMLR paper](http://www.jmlr.org/papers/volume5/rifkin04a/rifkin04a.pdf) argues convincingly that the answer is to use one-versus-all classification. This is also pretty easy to do in most packages. For example, in Python's ScikitLearn, you can do it with [OneVsRest](https://scikit-learn.org/stable/modules/generated/sklearn.multiclass.OneVsRestClassifier.html). If you're not sure what to try, that's probably a safe bet.
  
  
  ---
  
  