Answers.Body: <h1>You can balance your data-set.</h1>
  <p>Many models work with batches of samples. If you have a very unbalanced dataset, you can simply split it and ensure your batches are balanced (for example, for a Neural Network, using minibatches of 32 samples, you could draw 16 from your fraud users, and 16 from non-fraud users).</p>
  <p>During the learning phase, this ensures the model doesn't just output the most common class, but instead tries to learn to distinguish both.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-19T23:21:48.000Z
Answers.Text: Ive used it before for a skewed movie ratings dataset without issues. Keep in mind ypu still use the same amount of samples. You just change their relative frequency, such that both classes are equally likely to appear
Answers.UserDisplayName: BlueMoon93
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-19T17:53:44.260Z
Answers.LastActivityDate: 2018-04-19T17:53:44.260Z
Answers.LastEditDate: 2020-06-17T09:57:20.710Z
Answers.UserDisplayName: BlueMoon93
Answers.Body: <p>Heavily imbalanced classification tasks do not need a certain type of model, you can get different ones to work. </p>
  
  <p>You have two options: either use class weights (for example setting them to 'balanced' in the <a href="http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html" rel="nofollow noreferrer">ScikitLearn SVM</a>) in order to indicate that samples from a class are more important (the underrepresented one) or rebalance your dataset. For rebalancing purposes, and assuming you are using Python, I recommend <a href="https://github.com/scikit-learn-contrib/imbalanced-learn" rel="nofollow noreferrer">Imbalanced Learn</a>. There you have algorithms for over-sampling, under-sampling, over-sampling followed by under-sampling or ensemble sampling. If you use them, please check the plausibility of the synthetic samples you created by reducing dimensionality first and then plotting them for example in two dimensions. Are the synthetic samples similar to the true class?</p>
  
  <p>I would also recommend you to think about relevant metrics for (heavily) imbalanced problems and consider the no-information rate. That is another question though.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-24T09:45:57.077Z
Answers.Text: I tried under-sampling. It performs quite unstable between each run, some times better and most of the time it performs worse.
Answers.UserDisplayName: BerSerK
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-24T11:54:13.203Z
Answers.Text: What I would do is not try one of the techniques randomly, but reduce your data to 2D and plot it before and after the transformation. Then check for plausibility. Try also for example the Smote-Tomek technique, which combines Over- and Undersampling. If both methods do not work to your satisfaction simply use class weights.
Answers.UserDisplayName: AlexGuevara
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-20T05:44:58.960Z
Answers.LastActivityDate: 2018-04-20T05:44:58.960Z
Answers.UserDisplayName: AlexGuevara
Body: <p>I am working on an anti-fraud project. In the project, we are trying to predict the fraud user in the out time data set. But the fraud user has a very low ratio, only 3%. We expect a model with a precision more than 15%.</p>
  
  <p>I tried Logistic Regression, GBDT+LR, xgboost. All models are not good enough. Step wise Logistic Regression performs best, which has a precision of 9% with recall rate 6%.</p>
  
  <p>Is there any other models that I can use for this problem or any other advise ?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-22T08:32:57.863Z
Comments.Text: @DuttaA This should probably be upgraded to an answer, since the OP's problem is exactly what Anomaly Detection is for.
Comments.UserDisplayName: DrMcCleod
Tags: <ai-design>
Title: What model to use for fully unbalanced data?
UserDisplayName: BerSerK
ThreadText: # What model to use for fully unbalanced data?
  
  (Asked by: username_0 on 2018-04-19)
  
  I am working on an anti-fraud project. In the project, we are trying to predict the fraud user in the out time data set. But the fraud user has a very low ratio, only 3%. We expect a model with a precision more than 15%.
  
  I tried Logistic Regression, GBDT+LR, xgboost. All models are not good enough. Step wise Logistic Regression performs best, which has a precision of 9% with recall rate 6%.
  
  Is there any other models that I can use for this problem or any other advise ?
  
  **Tags:** ai-design
  
  > **Comments:**
  > **username_1**: @DuttaA This should probably be upgraded to an answer, since the OP's problem is exactly what Anomaly Detection is for.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2018-04-19. Score: 1
  
  # You can balance your data-set.
  
  Many models work with batches of samples. If you have a very unbalanced dataset, you can simply split it and ensure your batches are balanced (for example, for a Neural Network, using minibatches of 32 samples, you could draw 16 from your fraud users, and 16 from non-fraud users).
  
  During the learning phase, this ensures the model doesn't just output the most common class, but instead tries to learn to distinguish both.
  
  > **Comments:**
  > **username_2**: Ive used it before for a skewed movie ratings dataset without issues. Keep in mind ypu still use the same amount of samples. You just change their relative frequency, such that both classes are equally likely to appear  
  
  ---
  
  ### Answer by username_3 on 2018-04-20. Score: 0
  
  Heavily imbalanced classification tasks do not need a certain type of model, you can get different ones to work.
  
  You have two options: either use class weights (for example setting them to 'balanced' in the [ScikitLearn SVM](http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)) in order to indicate that samples from a class are more important (the underrepresented one) or rebalance your dataset. For rebalancing purposes, and assuming you are using Python, I recommend [Imbalanced Learn](https://github.com/scikit-learn-contrib/imbalanced-learn). There you have algorithms for over-sampling, under-sampling, over-sampling followed by under-sampling or ensemble sampling. If you use them, please check the plausibility of the synthetic samples you created by reducing dimensionality first and then plotting them for example in two dimensions. Are the synthetic samples similar to the true class?
  
  I would also recommend you to think about relevant metrics for (heavily) imbalanced problems and consider the no-information rate. That is another question though.
  
  > **Comments:**
  > **username_0**: I tried under-sampling. It performs quite unstable between each run, some times better and most of the time it performs worse.  
  > **username_3**: What I would do is not try one of the techniques randomly, but reduce your data to 2D and plot it before and after the transformation. Then check for plausibility. Try also for example the Smote-Tomek technique, which combines Over- and Undersampling. If both methods do not work to your satisfaction simply use class weights.  
  
  ---
  
  