Answers.Body: <p>The first thing which you first have to understand is that does your trained model is working efficiently with both the training and testing data. If yes then its not overfitting. There is only one case where over-fitting doesn't matter and that is when your testing data is same as training data and dealing with is very important or it could be disastrous. Therefore, to avoid overfitting you can try several methods and they are followings: </p>
  
  <ol>
  <li>Optimization Algorithms like sgd, Adam, RMSprop, Adagard, are few gradient descents.</li>
  <li>Loss function like Hinge loss could be useful.</li>
  <li>Parameter initialization greatly influence the optimization process.</li>
  </ol>
  
  <p>Overall, Large datasets are better for building ML models but there are some methods there is an interesting article that i would recommend you to read: <a href="https://towardsdatascience.com/breaking-the-curse-of-small-datasets-in-machine-learning-part-1-36f28b0c044d" rel="nofollow noreferrer">Article Link</a></p>
  
  <p>I Hope this Helps!!!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-30T16:58:40.467Z
Answers.Text: Thx for your answer, but I think that you didn't understand the question. My problem here is not to avoid overfitting, I'm searching for a way to make my model train faster.
Answers.UserDisplayName: Sebastien
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-30T17:01:11.437Z
Answers.Text: @Sebastien If you are really looking for a faster training model you have to include the coding, datasets, and modules. Looking forward to help!
Answers.UserDisplayName: techPirate99
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-30T17:06:29.140Z
Answers.Text: I'm not willing to include everything (this may become more complicated than my simple question. I just want to know if sometime, doing *1) train model on small data* until overfit  then *2) train model on all data* can be faster than just *2)*
Answers.UserDisplayName: Sebastien
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-30T17:29:46.613Z
Answers.Text: @Sebastien Droping small pieces of data until overfitting will converge faster in training data. And you really focusing on convergence then check article too [link](https://towardsdatascience.com/https-medium-com-super-convergence-very-fast-training-of-neural-networks-using-large-learning-rates-decb689b9eb0)
Answers.UserDisplayName: techPirate99
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-30T15:56:40.033Z
Answers.LastActivityDate: 2020-04-30T16:56:57.503Z
Answers.LastEditDate: 2020-04-30T16:56:57.503Z
Answers.UserDisplayName: techPirate99
Body: <p>I working on a classification problem that needs to detect patterns on a time serie. Basically, there's a catch-all class that means "no pattern detected", the other are for the specific patterns. The data is imbalanced (ratio 1/10 at least), but I adapted the class weights.</p>
  
  <p>I'm able to overfit successfully on a few days of data, but when I train on 2 years of data, the model seems stuck on class1 "no pattern detected" for a veeeery long time. I've tried several  learning rates, but it doesn't make the convergence happen significatively faster.</p>
  
  <p>Is it a better starting point for my training to use the overfitting model's weight as a starting point? Could this allow the model to converge faster?</p>
  
Tags: <convolutional-neural-networks><classification><overfitting><convergence><learning-rate>
Title: Is it a good idea to overfit on a small part of your data for faster model convergence?
UserDisplayName: Sebastien
ThreadText: # Is it a good idea to overfit on a small part of your data for faster model convergence?
  
  (Asked by: username_0 on 2020-04-30)
  
  I working on a classification problem that needs to detect patterns on a time serie. Basically, there's a catch-all class that means "no pattern detected", the other are for the specific patterns. The data is imbalanced (ratio 1/10 at least), but I adapted the class weights.
  
  I'm able to overfit successfully on a few days of data, but when I train on 2 years of data, the model seems stuck on class1 "no pattern detected" for a veeeery long time. I've tried several learning rates, but it doesn't make the convergence happen significatively faster.
  
  Is it a better starting point for my training to use the overfitting model's weight as a starting point? Could this allow the model to converge faster?
  
  **Tags:** convolutional-neural-networks, classification, overfitting, convergence, learning-rate
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-04-30. Score: 1
  
  The first thing which you first have to understand is that does your trained model is working efficiently with both the training and testing data. If yes then its not overfitting. There is only one case where over-fitting doesn't matter and that is when your testing data is same as training data and dealing with is very important or it could be disastrous. Therefore, to avoid overfitting you can try several methods and they are followings:
  
  1. Optimization Algorithms like sgd, Adam, RMSprop, Adagard, are few gradient descents.
  2. Loss function like Hinge loss could be useful.
  3. Parameter initialization greatly influence the optimization process.
  
  Overall, Large datasets are better for building ML models but there are some methods there is an interesting article that i would recommend you to read: [Article Link](https://towardsdatascience.com/breaking-the-curse-of-small-datasets-in-machine-learning-part-1-36f28b0c044d)
  
  I Hope this Helps!!!
  
  > **Comments:**
  > **username_0**: Thx for your answer, but I think that you didn't understand the question. My problem here is not to avoid overfitting, I'm searching for a way to make my model train faster.  
  > **username_1**: @username_0 If you are really looking for a faster training model you have to include the coding, datasets, and modules. Looking forward to help!  
  > **username_0**: I'm not willing to include everything (this may become more complicated than my simple question. I just want to know if sometime, doing *1) train model on small data* until overfit  then *2) train model on all data* can be faster than just *2)*  
  > **username_1**: @username_0 Droping small pieces of data until overfitting will converge faster in training data. And you really focusing on convergence then check article too [link](https://towardsdatascience.com/https-medium-com-super-convergence-very-fast-training-of-neural-networks-using-large-learning-rates-decb689b9eb0)  
  
  ---
  
  