Answers.Body: <p>Your zero-classes just do not exist for the model. There is no information about them in training and test sets. I think the reason for overfitting is that your have very small size of the training set comparing to the number of parameters in your model. If it is easier for a model to remember all training entries - it will just do that. You need to prune your model to force it to make some generalizations about the data.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-18T08:40:18.507Z
Answers.Text: Those images are from the original dataset. Before feeding it to the model , I reduce the class by 1 ( class 1 becomes class 0 and so on). Yes, I know in case of overfitting I have to reduce the model complexity. But I don't think it is the case in this problem,
Answers.UserDisplayName: Swakshar Deb
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-18T09:24:20.570Z
Answers.Text: You have about 20k entries in the training data and 500k parameters to learn on them. The proportion is reversed, you'd better have 5-10m entries in the training data to properly train such a model.
Answers.UserDisplayName: oleg.mosalov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-18T08:23:16.440Z
Answers.LastActivityDate: 2020-06-18T08:23:16.440Z
Answers.UserDisplayName: oleg.mosalov
Body: <p>I am doing human activity recognition project. I have total of 12 classes. The class distribution look like this:</p>
  <p><a href="https://i.sstatic.net/z7iTi.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/z7iTi.png" alt="enter image description here" /></a></p>
  <p><span class="math-container">$\color{red}{If \ you \ watch \ carefully, you \ can \ see \ that \ I \ have \ no \ data \ points \ for \ class \ 11 \ and \ class \ 8.}$</span> Also, the dataset is highly imbalanced. So, I took minimum data points (in this case 2028) for all of the classes. Now my balanced data look like this:</p>
  <p><a href="https://i.sstatic.net/2Iakj.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/2Iakj.png" alt="enter image description here" /></a></p>
  <p>After doing this it looks like a balance data. <span class="math-container">$\color{red}{But \ still, \ I \ think \ it \ not, \ because \ I \ have \ zero \ datapoints \ for \ class \ 11 \ and \ class \ 8}$</span>. In my opinion the classes are still imbalance.</p>
  <p>I am using CNN model to solve this activity project. My model summary is following:</p>
  <p><a href="https://i.sstatic.net/abLdb.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/abLdb.png" alt="enter image description here" /></a></p>
  <p>The main problem is, my model starts overfitting heavily when I train it.</p>
  <p><a href="https://i.sstatic.net/GPX9n.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/GPX9n.png" alt="enter image description here" /></a></p>
  <p>Is it due to my imbalance data( class 8 and 11 has zero data points) or something else?</p>
  <p><span class="math-container">$\textbf{Hyperperameter:}$</span></p>
  <p><span class="math-container">$\textbf{features:}$</span> X, Y, Z of mobile accelerometer</p>
  <p><span class="math-container">$\textbf{frame size:}$</span> 80</p>
  <p><span class="math-container">$\textbf{optimizer:}$</span> Adam, <span class="math-container">$\textbf{Learning rate:}$</span> 0.001</p>
  <p><span class="math-container">$\textbf{Loss:}$</span> Sparse categorical cross-entropy</p>
  
Tags: <machine-learning><deep-learning><computer-vision>
Title: Can imbalance data create overfitting?
UserDisplayName: Swakshar Deb
ThreadText: # Can imbalance data create overfitting?
  
  (Asked by: username_0 on 2020-06-18)
  
  I am doing human activity recognition project. I have total of 12 classes. The class distribution look like this:
  
  [![enter image description here](https://i.sstatic.net/z7iTi.png)](https://i.sstatic.net/z7iTi.png)
  
  $\color{red}{If \ you \ watch \ carefully, you \ can \ see \ that \ I \ have \ no \ data \ points \ for \ class \ 11 \ and \ class \ 8.}$ Also, the dataset is highly imbalanced. So, I took minimum data points (in this case 2028) for all of the classes. Now my balanced data look like this:
  
  [![enter image description here](https://i.sstatic.net/2Iakj.png)](https://i.sstatic.net/2Iakj.png)
  
  After doing this it looks like a balance data. $\color{red}{But \ still, \ I \ think \ it \ not, \ because \ I \ have \ zero \ datapoints \ for \ class \ 11 \ and \ class \ 8}$. In my opinion the classes are still imbalance.
  
  I am using CNN model to solve this activity project. My model summary is following:
  
  [![enter image description here](https://i.sstatic.net/abLdb.png)](https://i.sstatic.net/abLdb.png)
  
  The main problem is, my model starts overfitting heavily when I train it.
  
  [![enter image description here](https://i.sstatic.net/GPX9n.png)](https://i.sstatic.net/GPX9n.png)
  
  Is it due to my imbalance data( class 8 and 11 has zero data points) or something else?
  
  $\textbf{Hyperperameter:}$
  
  $\textbf{features:}$ X, Y, Z of mobile accelerometer
  
  $\textbf{frame size:}$ 80
  
  $\textbf{optimizer:}$ Adam, $\textbf{Learning rate:}$ 0.001
  
  $\textbf{Loss:}$ Sparse categorical cross-entropy
  
  **Tags:** machine-learning, deep-learning, computer-vision
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-06-18. Score: 1
  
  Your zero-classes just do not exist for the model. There is no information about them in training and test sets. I think the reason for overfitting is that your have very small size of the training set comparing to the number of parameters in your model. If it is easier for a model to remember all training entries - it will just do that. You need to prune your model to force it to make some generalizations about the data.
  
  > **Comments:**
  > **username_0**: Those images are from the original dataset. Before feeding it to the model , I reduce the class by 1 ( class 1 becomes class 0 and so on). Yes, I know in case of overfitting I have to reduce the model complexity. But I don't think it is the case in this problem,  
  > **username_1**: You have about 20k entries in the training data and 500k parameters to learn on them. The proportion is reversed, you'd better have 5-10m entries in the training data to properly train such a model.  
  
  ---
  
  