title: Combining classifiers' output based on F1-scores
body: <p>I have 4 classifiers available (already trained), for a 4-class classification problem. For a given dataset, I have the output of each of the classifiers, as well as their Recall, Precision and F1-scores for each of the classes.</p>
  
  <p>What would be the best algorithm (or available existing algorithms) to combine the predictions of these classifiers to get 1 single final prediction, taking into consideration that some of the classifiers have higher f1-scores than others for specific classes?</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>My main problem is that some classifiers have better F1 for specific classes.</p>
  
  <p>So let's say Classifier1 (C1) predicted class A, and  has a f1 of 0.90 for class A. Then  Classifier2 (C2) predicted class B, and has a f1 of 0.80 for class B.  </p>
  
  <p>My first thought would be to choose C1 prediction based on its higher f1, but what if for example we also know that C2 has a f1 score of 0.999 for class A? If C2 is that good at predicting class A (even better than C1) but did not predict it, that should increase the probability that the real class is not A, I believe.</p>
  
  <p>On the other and, if C2 had a really low f1 for class A that should make it even more likely that the real class was A, not only because C1 predicted A and it's good at it, but also because C2 is not good in predicting that class explaining why it might have failed to detect it</p>
  
  <p>I'm not sure how to deal with these questions in practice though.</p>