Answers.Body: <p>Definitely. NNs can learn the data that you teach to them. If you teach them biased, the network will be biased. As you mention, one solution is to reduce the data that you have. However, it is not the best approach as you will be losing the precious data. I would suggest to try data augmentation for remaining dataset to increase missing data type samples and have similar numbers for best, evenly distributed accuracy. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-11-01T08:58:00.937Z
Answers.LastActivityDate: 2017-11-01T08:58:00.937Z
Answers.UserDisplayName: Deniz Beker
Answers.Body: <p>Yes. Skewed data is one of the biggest problems in AI applications. As you rightly identified, the real world distribution is skewed. Doing a random sampling results in one major issue of an uneven sampling (like in your case). Even worse could happen, all of the samples may fall into a single class and other classes may not even be recognized by your classifier. This is called as the <strong>class imbalance problem</strong>.</p>
  
  <p>There are many ways to mitigate this problem. Few of them have been mentioned <a href="http://www.chioka.in/class-imbalance-problem/" rel="nofollow noreferrer">here</a>. I'll summarize them for you:</p>
  
  <ol>
  <li>Sampling based mitigation (when you try to deal with skewed data)</li>
  <li>Cost function based mitigation (when you try to improve the classifier, in case skewed data cannot be avoided)</li>
  </ol>
  
  <p>Sampling based mitigation can be done by oversampling from the minority class(es), or undersampling from the majority class(es) or using a combination of both. </p>
  
  <p>Another fairer method to do this is by doing a stratified sampling. If class A has 1/6th, class B has 2/6th and class C has 3/6th of the total population, then, you should take 1/6th of the samples by random sampling from objects in class A, 2/6th from class B and 3/6th from class C. This way all of the sample may represent the population in the right proportion, and none of the classes will be missed.</p>
  
  <p>In case this doesn't help (say, you dataset is small and sampling doesn't make sense) and the data is skewed too much), the cost function can be modified to make a more sensible classifier. Keep different costs for misclassification and/or correct classification. Misclassification could be given a higher penalty. This could help in reaching to a better classifier with a skewed data.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-24T17:32:11.583Z
Answers.LastActivityDate: 2017-12-24T17:32:11.583Z
Answers.UserDisplayName: Naveen Venkat
Body: <p>I'm working on an image classification problem using a neural network. In the training data set, 90% of the samples fall into 10% of all categories, while 10% of the sample fall into the other 90% categories. So an example is not evenly distributed among all categories. If we assume this distribution reflects the real-world distribution, do I need to filter my dataset before training so that each category has a similar number of samples?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-10-17T05:59:49.640Z
Comments.Text: [8 Tactics to Combat Imbalanced Classes in Your Machine Learning Dataset](https://machinelearningmastery.com/tactics-to-combat-imbalanced-classes-in-your-machine-learning-dataset/)
Comments.UserDisplayName: SmallChess
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-24T19:47:37.030Z
Comments.Text: [Likely of interest](https://stats.meta.stackexchange.com/q/6349/247274), especially [one particular link contained within](https://stats.stackexchange.com/q/357466/247274)
Comments.UserDisplayName: Dave
Tags: <neural-networks><image-recognition><data-preprocessing><imbalanced-datasets>
Title: Does data skew matter in classification problem?
UserDisplayName: James Gan
ThreadText: # Does data skew matter in classification problem?
  
  (Asked by: username_0 on 2017-10-16)
  
  I'm working on an image classification problem using a neural network. In the training data set, 90% of the samples fall into 10% of all categories, while 10% of the sample fall into the other 90% categories. So an example is not evenly distributed among all categories. If we assume this distribution reflects the real-world distribution, do I need to filter my dataset before training so that each category has a similar number of samples?
  
  **Tags:** neural-networks, image-recognition, data-preprocessing, imbalanced-datasets
  > **Comments:**
  > **username_1**: [8 Tactics to Combat Imbalanced Classes in Your Machine Learning Dataset](https://machinelearningmastery.com/tactics-to-combat-imbalanced-classes-in-your-machine-learning-dataset/)  
  > **username_2**: [Likely of interest](https://stats.meta.stackexchange.com/q/6349/247274), especially [one particular link contained within](https://stats.stackexchange.com/q/357466/247274)  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2017-12-24. Score: 3 | Accepted answer
  
  Yes. Skewed data is one of the biggest problems in AI applications. As you rightly identified, the real world distribution is skewed. Doing a random sampling results in one major issue of an uneven sampling (like in your case). Even worse could happen, all of the samples may fall into a single class and other classes may not even be recognized by your classifier. This is called as the **class imbalance problem**.
  
  There are many ways to mitigate this problem. Few of them have been mentioned [here](http://www.chioka.in/class-imbalance-problem/). I'll summarize them for you:
  
  1. Sampling based mitigation (when you try to deal with skewed data)
  2. Cost function based mitigation (when you try to improve the classifier, in case skewed data cannot be avoided)
  
  Sampling based mitigation can be done by oversampling from the minority class(es), or undersampling from the majority class(es) or using a combination of both.
  
  Another fairer method to do this is by doing a stratified sampling. If class A has 1/6th, class B has 2/6th and class C has 3/6th of the total population, then, you should take 1/6th of the samples by random sampling from objects in class A, 2/6th from class B and 3/6th from class C. This way all of the sample may represent the population in the right proportion, and none of the classes will be missed.
  
  In case this doesn't help (say, you dataset is small and sampling doesn't make sense) and the data is skewed too much), the cost function can be modified to make a more sensible classifier. Keep different costs for misclassification and/or correct classification. Misclassification could be given a higher penalty. This could help in reaching to a better classifier with a skewed data.
  
  
  ---
  
  ### Answer by username_3 on 2017-11-01. Score: 1
  
  Definitely. NNs can learn the data that you teach to them. If you teach them biased, the network will be biased. As you mention, one solution is to reduce the data that you have. However, it is not the best approach as you will be losing the precious data. I would suggest to try data augmentation for remaining dataset to increase missing data type samples and have similar numbers for best, evenly distributed accuracy.
  
  
  ---
  
  