Answers.Body: <p>Random over sampling creates duplicates of existing examples, so applying this to your training data would be the same as increasing the weight of the oversampled examples. If it's done to all of the examples uniformly then the effects will probably cancel out. SMOTE, on the other hand, creates synthetic examples that are linear combinations of existing examples. Thus it can be thought of as a type of data augmentation, and in some situations this might improve your model's predictions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-26T22:30:12.697Z
Answers.LastActivityDate: 2021-04-26T22:30:12.697Z
Answers.UserDisplayName: Mike NZ
Answers.Body: <p>If your dataset is almost balanced, I don't think any sampling method would provide a significant improvement in accuracy. You can also experiment with the <a href="https://arxiv.org/abs/1708.02002?fbclid" rel="nofollow noreferrer">focal loss</a> function designed to deal with minor class-imbalance, but I'm not sure how much that would help.</p>
  <p>I don't think you can artificially increase the dataset size besides doing data augmentation. For example, for computer vision tasks (like classification and object detection) dealing with images, you can randomly translate and rotate the images, so that the neural network is effectively seeing different &quot;representations&quot; of the same images to alleviate overfitting. This is especially important if your dataset size is small, but does not pertain to solving the class imbalance issue.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-30T23:28:22.857Z
Answers.LastActivityDate: 2022-08-30T23:28:22.857Z
Answers.UserDisplayName: PeaBrane
Answers.Body: <p>I presume you are attempting to solve a classification problem.</p>
  <p>IMO,  there's no decision-making template you could follow to know whether to use over sampling or not.I would typically compare results (ROC AUC, PRC curves) across datasets (Original vs Undersampled vs Oversampled) to decide.</p>
  <p>You can consider some additional variants of SMOTE like SMOTE NC (SMOTE does not work if any of your predictors is categorical , SMOTE NC does), Borderline SMOTE, K-means SMOTE, as well as ADASYN.
  Alternately, you can also choose to undersample your majority class using techniques such as ENN, Tomek Links, Instance Hardness, CNN, One sided Selection etc.
  They usually generate better results than random over/under sampling.</p>
  <p>Do note that over/under sampling methods are generally used for imbalanced datasets.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-27T11:38:16.010Z
Answers.Text: Hey, yeah I know they(all the aforementioned methods) are generally used for imbalanced datasets but I'm trying to add more data points in my balanced dataset.
Answers.UserDisplayName: Sourabh Yadav
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-27T11:34:29.640Z
Answers.LastActivityDate: 2021-04-27T11:34:29.640Z
Answers.UserDisplayName: Nibbles
Answers.Body: <p>Does your question pertain to general data augmentation? That is already in heavy use- using transformations while training is very common, and over several epochs the network benefits from learning the new representations. The transformations are applied to all classes, with a probability of transformation ( horizontal flip, for example) specified by the user.
  If you want to make your almost balanced dataset a balanced one, you can look into specific augmentations that you perform to the (almost) minority class before feeding it to the model. You could look into preprocessing methods that libraries like Keras have made open-source.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-29T02:44:32.953Z
Answers.LastActivityDate: 2021-06-29T02:44:32.953Z
Answers.UserDisplayName: anon
Answers.Body: <p>If your data is well balanced but small, I would recommend using a simpler algorithm to classify your data.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-29T08:44:06.677Z
Answers.LastActivityDate: 2021-04-29T08:44:06.677Z
Answers.UserDisplayName: Marcus
Answers.Body: <p><a href="https://console.gretel.cloud/projects/_new/" rel="nofollow noreferrer">Gretel</a> is a good tool for processing data. <a href="https://github.com/PAIR-code/facets" rel="nofollow noreferrer">Facets</a> is good for the visualizations.</p>
  <p>Is it worth it? <a href="https://journalofbigdata.springeropen.com/articles/10.1186/s40537-019-0192-5" rel="nofollow noreferrer">most learners will exhibit bias towards the majority class</a>, and in extreme cases, may ignore the minority class altogether.</p>
  <p>It really depends on the goal and requirements of your project. Not because it's desirable it's better for your particular project, if your dataset is <strong>almost</strong> balanced probably you should continue with something else and consider balancing your data later in the project.</p>
  <p><strong>Edit</strong>: What's new to learn from the same example?</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-27T11:34:35.933Z
Answers.Text: I meant is it worth adding more data points in a balanced dataset.
Answers.UserDisplayName: Sourabh Yadav
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-27T03:15:20.107Z
Answers.LastActivityDate: 2021-08-05T20:07:40.320Z
Answers.LastEditDate: 2021-08-05T20:07:40.320Z
Answers.UserDisplayName: Echo9k
Body: <p>I am trying to add more data points in my (almost) balanced dataset for training my neural network. I have come across techniques such as SMOTE or Random Over Sampling, but they work best for imbalanced data (as they balance the dataset). How can I do this and is it even worth it?</p>
  <p>P.S. I know copying the same data points and appending them at the end doesn't add much value, but can we do it, and can it help to increase the prediction accuracy?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-25T04:07:45.140Z
Comments.Text: There is nothing stopping you from doing SMOTE-style synthesis of majority-category points, but [SMOTE doesn’t seem to be particularly good at synthesizing realistic points](https://stats.stackexchange.com/a/588248/247274).
Comments.UserDisplayName: Dave
Tags: <deep-learning><datasets><data-preprocessing><data-augmentation>
Title: How do I increase the size of an (almost) balanced dataset?
UserDisplayName: Sourabh Yadav
ThreadText: # How do I increase the size of an (almost) balanced dataset?
  
  (Asked by: username_0 on 2021-04-26)
  
  I am trying to add more data points in my (almost) balanced dataset for training my neural network. I have come across techniques such as SMOTE or Random Over Sampling, but they work best for imbalanced data (as they balance the dataset). How can I do this and is it even worth it?
  
  P.S. I know copying the same data points and appending them at the end doesn't add much value, but can we do it, and can it help to increase the prediction accuracy?
  
  **Tags:** deep-learning, datasets, data-preprocessing, data-augmentation
  > **Comments:**
  > **username_1**: There is nothing stopping you from doing SMOTE-style synthesis of majority-category points, but [SMOTE doesn’t seem to be particularly good at synthesizing realistic points](https://stats.stackexchange.com/a/588248/247274).  
  
  ---
  
  ## 6 answers
  
  ### Answer by username_2 on 2021-04-26. Score: 1
  
  Random over sampling creates duplicates of existing examples, so applying this to your training data would be the same as increasing the weight of the oversampled examples. If it's done to all of the examples uniformly then the effects will probably cancel out. SMOTE, on the other hand, creates synthetic examples that are linear combinations of existing examples. Thus it can be thought of as a type of data augmentation, and in some situations this might improve your model's predictions.
  
  
  ---
  
  ### Answer by username_3 on 2022-08-30. Score: 1
  
  If your dataset is almost balanced, I don't think any sampling method would provide a significant improvement in accuracy. You can also experiment with the [focal loss](https://arxiv.org/abs/1708.02002?fbclid) function designed to deal with minor class-imbalance, but I'm not sure how much that would help.
  
  I don't think you can artificially increase the dataset size besides doing data augmentation. For example, for computer vision tasks (like classification and object detection) dealing with images, you can randomly translate and rotate the images, so that the neural network is effectively seeing different "representations" of the same images to alleviate overfitting. This is especially important if your dataset size is small, but does not pertain to solving the class imbalance issue.
  
  
  ---
  
  ### Answer by username_4 on 2021-04-27. Score: 1
  
  I presume you are attempting to solve a classification problem.
  
  IMO, there's no decision-making template you could follow to know whether to use over sampling or not.I would typically compare results (ROC AUC, PRC curves) across datasets (Original vs Undersampled vs Oversampled) to decide.
  
  You can consider some additional variants of SMOTE like SMOTE NC (SMOTE does not work if any of your predictors is categorical , SMOTE NC does), Borderline SMOTE, K-means SMOTE, as well as ADASYN.
  Alternately, you can also choose to undersample your majority class using techniques such as ENN, Tomek Links, Instance Hardness, CNN, One sided Selection etc.
  They usually generate better results than random over/under sampling.
  
  Do note that over/under sampling methods are generally used for imbalanced datasets.
  
  > **Comments:**
  > **username_0**: Hey, yeah I know they(all the aforementioned methods) are generally used for imbalanced datasets but I'm trying to add more data points in my balanced dataset.  
  
  ---
  
  ### Answer by username_5 on 2021-06-29. Score: 0
  
  Does your question pertain to general data augmentation? That is already in heavy use- using transformations while training is very common, and over several epochs the network benefits from learning the new representations. The transformations are applied to all classes, with a probability of transformation ( horizontal flip, for example) specified by the user.
  If you want to make your almost balanced dataset a balanced one, you can look into specific augmentations that you perform to the (almost) minority class before feeding it to the model. You could look into preprocessing methods that libraries like Keras have made open-source.
  
  
  ---
  
  ### Answer by username_6 on 2021-04-29. Score: 0
  
  If your data is well balanced but small, I would recommend using a simpler algorithm to classify your data.
  
  
  ---
  
  ### Answer by username_7 on 2021-04-27. Score: -1
  
  [Gretel](https://console.gretel.cloud/projects/_new/) is a good tool for processing data. [Facets](https://github.com/PAIR-code/facets) is good for the visualizations.
  
  Is it worth it? [most learners will exhibit bias towards the majority class](https://journalofbigdata.springeropen.com/articles/10.1186/s40537-019-0192-5), and in extreme cases, may ignore the minority class altogether.
  
  It really depends on the goal and requirements of your project. Not because it's desirable it's better for your particular project, if your dataset is **almost** balanced probably you should continue with something else and consider balancing your data later in the project.
  
  **Edit**: What's new to learn from the same example?
  
  > **Comments:**
  > **username_0**: I meant is it worth adding more data points in a balanced dataset.  
  
  ---
  
  