Answers.Body: <p>There are two main things to consider for dealing with imbalanced data:</p>
  
  <ol>
  <li><p>During Training: Undersampling the majority class (healthy patients) so that the model is not that biased to predicting healthy</p></li>
  <li><p>During Evaluation: Using a suitable metric to try to evaluate your model and try to optimize on when you are fine-tuning your random-forest. For imbalanced data you usually use F1 score but since a high recall (predicting sick more often) is important here, F2 score (or other F-beta score where beta>1) is more suitable <a href="https://en.wikipedia.org/wiki/F1_score" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/F1_score</a> </p></li>
  </ol>
  
  <p>You can also check for example <a href="https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html" rel="nofollow noreferrer">https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html</a> for more about how to deal with imbalanced data in general</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-28T17:01:21.693Z
Answers.Text: is there a reason why undersampling would be better than oversampling?
Answers.UserDisplayName: jennifer ruurs
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-28T17:27:34.587Z
Answers.Text: not really, if you don't have memory/resource issues, you can try both and see which (if any) works better
Answers.UserDisplayName: cookiemonster
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-28T16:28:52.430Z
Answers.LastActivityDate: 2019-11-28T16:28:52.430Z
Answers.UserDisplayName: cookiemonster
Answers.Body: <p>A random forest is a collection of classification trees. If more than 50% of these trees predict class A (and not class B), the random forest will predict class A. </p>
  
  <p>What you can do is lower the percentage needed to classify it as class A (in your case, patient has the virus). This way, you can tell your random forest to predict class A if only 20% (or 10%, or 5%, ...) of the decision trees actually predicts class A. </p>
  
  <p>I don't know what code you are using for the random forest algoritm, but in most you should be able to ask the % of certainty the random forest has of each class. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-26T14:40:13.053Z
Answers.Text: Isn't this the same as what a cutoff function does?
Answers.UserDisplayName: jennifer ruurs
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-26T14:49:38.893Z
Answers.Text: yes, I must have missed the last line on your question. Did this not help your accuracy?
Answers.UserDisplayName: Lustwelpintje
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-26T15:26:08.963Z
Answers.Text: It did improve the accuracy, but i am interested in more suggestion :)
Answers.UserDisplayName: jennifer ruurs
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-26T15:37:10.153Z
Answers.Text: It is possible that the 11 features are not enough data for the random forest. 92% accuracy is actually pretty low considering you are right 98% of the time if you just say nobody has the virus.
Answers.UserDisplayName: Lustwelpintje
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-26T15:44:12.187Z
Answers.Text: Have you tried instead of increasing the virus examples with SMOTE decreasing the non virus examples? Only use about 4% of your database (all the virus examples (2%), and a low amount of the non virus examples(2%))?
Answers.UserDisplayName: Lustwelpintje
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-27T06:30:51.540Z
Answers.Text: do you have a python example (sklearn) or r where I can: "ask the % of certainty the random forest has of each class" (I btw also tried indeed undersampling)
Answers.UserDisplayName: jennifer ruurs
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-27T09:59:47.570Z
Answers.Text: in scikit learn this is predict_proba(). following link has a example of this: https://stackoverflow.com/questions/30814231/using-the-predict-proba-function-of-randomforestclassifier-in-the-safe-and-rig
Answers.UserDisplayName: Lustwelpintje
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-26T13:22:57.310Z
Answers.LastActivityDate: 2019-11-26T13:22:57.310Z
Answers.UserDisplayName: Lustwelpintje
Body: <p>I have a big dataset (28354359 rows) that has some blood values as features (11 features) and the label or outcome variable that tells whether a patient has a virus caused by a Neoplasm or not.</p>
  
  <p>The problem with my dataset is that 2% of the patients that are in my dataset have the virus and 98% does not have the virus. </p>
  
  <p>I am mandatory to use the random forest algorithm. While my random forest model has a high accuracy scores 92%, the problem is that more than 90% of the patients that have the virus are predicted that they don’t have the virus.</p>
  
  <p>I want the opposite effect, I want that my random forest is likely to predict more often that a patient has the virus (even if the patient does not have the virus (ideally I don’t want this side effect , but rather this than the opposite)).</p>
  
  <p>The idea behind this is that performing an extra test (via an echo) could not harm the patient that has not the virus, but not testing a patient will have result terrible for the patient.</p>
  
  <p><strong>Does somebody have advice how I could tweak my random forest model for this task?</strong></p>
  
  <p>I my self experimented with the SMOTE transformation and other sampling techniques but maybe you guys have other suggestion.</p>
  
  <p>I also have tried to apply a cutoff function.</p>
  
Tags: <machine-learning><random-forests>
Title: Oposite type of predictions for unbalanced dataset
UserDisplayName: jennifer ruurs
ThreadText: # Oposite type of predictions for unbalanced dataset
  
  (Asked by: username_0 on 2019-11-25)
  
  I have a big dataset (28354359 rows) that has some blood values as features (11 features) and the label or outcome variable that tells whether a patient has a virus caused by a Neoplasm or not.
  
  The problem with my dataset is that 2% of the patients that are in my dataset have the virus and 98% does not have the virus.
  
  I am mandatory to use the random forest algorithm. While my random forest model has a high accuracy scores 92%, the problem is that more than 90% of the patients that have the virus are predicted that they don’t have the virus.
  
  I want the opposite effect, I want that my random forest is likely to predict more often that a patient has the virus (even if the patient does not have the virus (ideally I don’t want this side effect , but rather this than the opposite)).
  
  The idea behind this is that performing an extra test (via an echo) could not harm the patient that has not the virus, but not testing a patient will have result terrible for the patient.
  
  **Does somebody have advice how I could tweak my random forest model for this task?**
  
  I my self experimented with the SMOTE transformation and other sampling techniques but maybe you guys have other suggestion.
  
  I also have tried to apply a cutoff function.
  
  **Tags:** machine-learning, random-forests
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2019-11-28. Score: 2
  
  There are two main things to consider for dealing with imbalanced data:
  
  1. During Training: Undersampling the majority class (healthy patients) so that the model is not that biased to predicting healthy
  2. During Evaluation: Using a suitable metric to try to evaluate your model and try to optimize on when you are fine-tuning your random-forest. For imbalanced data you usually use F1 score but since a high recall (predicting sick more often) is important here, F2 score (or other F-beta score where beta>1) is more suitable <https://en.wikipedia.org/wiki/F1_score>
  
  You can also check for example <https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html> for more about how to deal with imbalanced data in general
  
  > **Comments:**
  > **username_0**: is there a reason why undersampling would be better than oversampling?  
  > **username_1**: not really, if you don't have memory/resource issues, you can try both and see which (if any) works better  
  
  ---
  
  ### Answer by username_2 on 2019-11-26. Score: 2
  
  A random forest is a collection of classification trees. If more than 50% of these trees predict class A (and not class B), the random forest will predict class A.
  
  What you can do is lower the percentage needed to classify it as class A (in your case, patient has the virus). This way, you can tell your random forest to predict class A if only 20% (or 10%, or 5%, ...) of the decision trees actually predicts class A.
  
  I don't know what code you are using for the random forest algoritm, but in most you should be able to ask the % of certainty the random forest has of each class.
  
  > **Comments:**
  > **username_0**: Isn't this the same as what a cutoff function does?  
  > **username_2**: yes, I must have missed the last line on your question. Did this not help your accuracy?  
  > **username_0**: It did improve the accuracy, but i am interested in more suggestion :)  
  > **username_2**: It is possible that the 11 features are not enough data for the random forest. 92% accuracy is actually pretty low considering you are right 98% of the time if you just say nobody has the virus.  
  > **username_2**: Have you tried instead of increasing the virus examples with SMOTE decreasing the non virus examples? Only use about 4% of your database (all the virus examples (2%), and a low amount of the non virus examples(2%))?  
  > **username_0**: do you have a python example (sklearn) or r where I can: "ask the % of certainty the random forest has of each class" (I btw also tried indeed undersampling)  
  > **username_2**: in scikit learn this is predict_proba(). following link has a example of this: https://stackoverflow.com/questions/30814231/using-the-predict-proba-function-of-randomforestclassifier-in-the-safe-and-rig  
  
  ---
  
  