Answers.Body: <p>This would be a pretty fun opportunity to experiment a bit. I know there's been breakthroughs in OCR technology by training the network on images that have been altered.</p>
  
  <p>Long story short, there was a pre-process that took the source images and did minor manipulations to them. This involved skewing, rotating, changing resolution &amp; adding artifacts into the image to introduce "Noise". This allowed the network to learn to recognize characters more accurately. It may be worthwhile to create an image pre-processing pipeline that will do a variation of modifications to the source images. When training you may want to lower your learning rate &amp; split up your data into 70% train &amp; 30% test. Then you can loop through all the images repeatedly until you hit a maximum for the learning rate.</p>
  
  <p>Some examples
  <a href="https://matthewearl.github.io/2016/05/06/cnn-anpr/" rel="nofollow noreferrer">https://matthewearl.github.io/2016/05/06/cnn-anpr/</a>
  <a href="https://medium.com/@shreyas.s/image-data-generation-for-optical-character-recognition-ocr-9b19300649c8" rel="nofollow noreferrer">https://medium.com/@shreyas.s/image-data-generation-for-optical-character-recognition-ocr-9b19300649c8</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-01T06:23:59.307Z
Answers.Text: So if I do an augmentation of the minority class, of 2000 images to bring it to 19000 images. bringing the data from 10% to 90% using augmentation,don't  you think it will skew my data, create a bias issue. Basically I am flipping rotating , grey scaling,other than that I am not bringing in any new feature information necessary for the classifier to learn.As practically a flipped image or grey scale image would be a rare thing in my case since i will standardise my testing  data.What is your take on this ?
Answers.UserDisplayName: user10573543
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-01T20:19:24.853Z
Answers.Text: None of these algorithms mean anything without data. The model may not reach the full potential without more source images. It can become biased so that it over samples the class with 2000 images and it's over-represented in your results. The way you train it may also affect how much of an impact that will be.
Answers.UserDisplayName: Zakk Diaz
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-29T16:31:26.773Z
Answers.LastActivityDate: 2018-11-29T16:57:37.127Z
Answers.LastEditDate: 2018-11-29T16:57:37.127Z
Answers.UserDisplayName: Zakk Diaz
Body: <p>I am working on an image data-set. As you may have guessed it is imbalanced data. I have 'Class A, 19,000 images' and 'Class B, 2,876 images'.</p>
  
  <p>So I did an undersampling by removing randomly from the majority class till it becomes equal to the minority class.</p>
  
  <p>On doing this I am loosing lot of information from those 19000 images which I could get. 
  So I do an oversampling of minority class, by simply copying the 2,876 images again and again.</p>
  
  <p>Is this undersampling method correct, will it effect my accuracy? I trained an Inceptionv4 model using this oversampled data and it is not at all stable and I am getting poor accuracy.</p>
  
  <p><strong><em>What should be my strategy ?</em></strong></p>
  
Tags: <convolutional-neural-networks><ai-design>
Title: Can I do oversampling by copying the same image multiple times? Will it effect my neural network accuracy?
UserDisplayName: user10573543
ThreadText: # Can I do oversampling by copying the same image multiple times? Will it effect my neural network accuracy?
  
  (Asked by: username_0 on 2018-11-26)
  
  I am working on an image data-set. As you may have guessed it is imbalanced data. I have 'Class A, 19,000 images' and 'Class B, 2,876 images'.
  
  So I did an undersampling by removing randomly from the majority class till it becomes equal to the minority class.
  
  On doing this I am loosing lot of information from those 19000 images which I could get.
  So I do an oversampling of minority class, by simply copying the 2,876 images again and again.
  
  Is this undersampling method correct, will it effect my accuracy? I trained an Inceptionv4 model using this oversampled data and it is not at all stable and I am getting poor accuracy.
  
  ***What should be my strategy ?***
  
  **Tags:** convolutional-neural-networks, ai-design
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-11-29. Score: 2
  
  This would be a pretty fun opportunity to experiment a bit. I know there's been breakthroughs in OCR technology by training the network on images that have been altered.
  
  Long story short, there was a pre-process that took the source images and did minor manipulations to them. This involved skewing, rotating, changing resolution & adding artifacts into the image to introduce "Noise". This allowed the network to learn to recognize characters more accurately. It may be worthwhile to create an image pre-processing pipeline that will do a variation of modifications to the source images. When training you may want to lower your learning rate & split up your data into 70% train & 30% test. Then you can loop through all the images repeatedly until you hit a maximum for the learning rate.
  
  Some examples
  <https://matthewearl.github.io/2016/05/06/cnn-anpr/>
  <https://medium.com/@shreyas.s/image-data-generation-for-optical-character-recognition-ocr-9b19300649c8>
  
  > **Comments:**
  > **username_0**: So if I do an augmentation of the minority class, of 2000 images to bring it to 19000 images. bringing the data from 10% to 90% using augmentation,don't  you think it will skew my data, create a bias issue. Basically I am flipping rotating , grey scaling,other than that I am not bringing in any new feature information necessary for the classifier to learn.As practically a flipped image or grey scale image would be a rare thing in my case since i will standardise my testing  data.What is your take on this ?  
  > **username_1**: None of these algorithms mean anything without data. The model may not reach the full potential without more source images. It can become biased so that it over samples the class with 2000 images and it's over-represented in your results. The way you train it may also affect how much of an impact that will be.  
  
  ---
  
  