Answers.Body: <p>The problem of overfitting is given that your model is being too flexible with the training data. The approaches you could take will be:</p>
  
  <ol>
  <li>Download pretrained models (VGG-16) and use transfer learning.</li>
  <li>Increase the value of your dropout (e.g. from 0.25 to 0.50).</li>
  <li>Use data augmentation for your images.</li>
  <li>Reduce the number of fully connected layers.</li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-13T21:09:28.773Z
Answers.LastActivityDate: 2019-03-13T21:09:28.773Z
Answers.UserDisplayName: Bitzel
Body: <p>Currently I'm feeding spectrogram of audio to the CNN with 3 convolution.</p>
  
  <p>Each convolution is followed by a max pool of filter size 2.</p>
  
  <p>First -> 5x5x4</p>
  
  <p>Second - > 5x5x8</p>
  
  <p>Third - > 5x5x16</p>
  
  <p>and final layer is a fully connected with 512 unit.</p>
  
  <p>But while training with dropout of 0.25, getting train accuracy of 0.97 with 150 iterations.
  and on test data accuracy is just 0.60.</p>
  
  <p>Tell me how to improve the results.</p>
  
  <p>Yes both train and test data come from same distribution.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-13T19:50:05.490Z
Comments.Text: What is your dataset size, and how easy is it to get more data? Would your target output remain the same if you altered the audio source - e.g. adding noise, changing speed (or just pitch), etc?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-13T20:10:58.210Z
Comments.Text: Dataset contains 1k images. because this is a people voice classification problem more data is not available but haven't tried data augmentation. will try and update here.
Comments.UserDisplayName: Ajmal Rasi
Tags: <machine-learning><deep-learning><convolutional-neural-networks>
Title: How to reduce over-fitting on training set?
UserDisplayName: Ajmal Rasi
ThreadText: # How to reduce over-fitting on training set?
  
  (Asked by: username_0 on 2019-03-13)
  
  Currently I'm feeding spectrogram of audio to the CNN with 3 convolution.
  
  Each convolution is followed by a max pool of filter size 2.
  
  First -> 5x5x4
  
  Second - > 5x5x8
  
  Third - > 5x5x16
  
  and final layer is a fully connected with 512 unit.
  
  But while training with dropout of 0.25, getting train accuracy of 0.97 with 150 iterations.
  and on test data accuracy is just 0.60.
  
  Tell me how to improve the results.
  
  Yes both train and test data come from same distribution.
  
  **Tags:** machine-learning, deep-learning, convolutional-neural-networks
  > **Comments:**
  > **username_1**: What is your dataset size, and how easy is it to get more data? Would your target output remain the same if you altered the audio source - e.g. adding noise, changing speed (or just pitch), etc?  
  > **username_0**: Dataset contains 1k images. because this is a people voice classification problem more data is not available but haven't tried data augmentation. will try and update here.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-03-13. Score: 2
  
  The problem of overfitting is given that your model is being too flexible with the training data. The approaches you could take will be:
  
  1. Download pretrained models (VGG-16) and use transfer learning.
  2. Increase the value of your dropout (e.g. from 0.25 to 0.50).
  3. Use data augmentation for your images.
  4. Reduce the number of fully connected layers.
  
  
  ---
  
  