Answers.Body: <p>As you have trained your model in batch_size of 500. Weights has been updated for each batch therefore 600 times(300000/500) by the end of one epoch.<br>
  So, Your model generalized well. Check the predictions. If Predictions are well. Your model is ready.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-04T09:42:30.630Z
Answers.LastActivityDate: 2019-09-04T09:42:30.630Z
Answers.UserDisplayName: anil kumar
Answers.Body: <p>It can be normal and there might be nothing wrong with your model. If there is a very strong and clear correlation in your data(good separability) then a network can achive very high accuracy very fast. After reaching some value learning gets harder.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-09T20:08:15.433Z
Answers.LastActivityDate: 2019-09-09T20:08:15.433Z
Answers.UserDisplayName: GKozinski
Answers.Body: <p>It actually depends on a couple of things here -</p>
  
  <ol>
  <li>How many output classes do you have? If you have only 2 or 3 classes, it is a very easy task for the classifier that you have built. So, it is highly possible that convergence has occurred.</li>
  <li>As @Djib2011 mentioned already, if your input training set is not balanced and is heavier with one of the output classes (95%), then this accuracy you see makes sense but note that your model won't do well in production.</li>
  <li>Do not try to evaluate your model on the basis of your training accuracy. Test it on data your model has never seen before and then evaluate the classification accuracy making sure that your training/testing data is not heavy with one of the classes. </li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-09T10:38:12.637Z
Answers.LastActivityDate: 2019-09-09T10:38:12.637Z
Answers.UserDisplayName: Dishant Sheth
Body: <p>I have a simple text classifier, with the following structure:</p>
  
  <pre><code>    input = keras.layers.Input(shape=(len(train_x[0]),))
  
      x=keras.layers.Dense(500, activation='relu')(input)
      x=keras.layers.Dropout(0.5)(x)
      x=keras.layers.Dense(250, activation='relu')(x)
      x=keras.layers.Dropout(0.5)(x)
      preds = keras.layers.Dense(len(train_y[0]), activation="sigmoid")(x)
  
      model = keras.Model(input, preds)
  </code></pre>
  
  <p>When training it with 300,000 samples, with a batch size of 500, I get an accuracy value of .95 and loss of .22 in the first iteration, and the subsequent iterations are .96 and .11.</p>
  
  <p>Why does the accuracy grow so quickly, and then just stop growing?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-02T20:44:36.553Z
Comments.Text: By any chance you are running it on a Jupyter notebook, can you share more of your code, especially about the training part ?
Comments.UserDisplayName: Semih Korkmaz
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-02T21:03:26.113Z
Comments.Text: I'm not. I'm new to this. I don't even know what a jupyter notebook is :P
Comments.UserDisplayName: hjf
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-02T21:21:07.353Z
Comments.Text: Without more information I don't think we can help you. A few things I can thinkg of: First of all, are you sure you mean iteration and not epoch? Secondly, are your data balanced (i.e. same number of samples in each class)? If not it is possible, if let's say 95% of your samples belong to a single class that your model is predicting this class only and achieving 0.95 accuracy. Finally, it is possible that it is just a very simple task and your model actually scores so high.
Comments.UserDisplayName: Djib2011
Tags: <training><tensorflow><keras>
Title: Accuracy too high too fast?
UserDisplayName: hjf
ThreadText: # Accuracy too high too fast?
  
  (Asked by: username_0 on 2019-09-02)
  
  I have a simple text classifier, with the following structure:
      input = keras.layers.Input(shape=(len(train_x[0]),))
  
      x=keras.layers.Dense(500, activation='relu')(input)
      x=keras.layers.Dropout(0.5)(x)
      x=keras.layers.Dense(250, activation='relu')(x)
      x=keras.layers.Dropout(0.5)(x)
      preds = keras.layers.Dense(len(train_y[0]), activation="sigmoid")(x)
  
      model = keras.Model(input, preds)
  
  When training it with 300,000 samples, with a batch size of 500, I get an accuracy value of .95 and loss of .22 in the first iteration, and the subsequent iterations are .96 and .11.
  Why does the accuracy grow so quickly, and then just stop growing?
  
  **Tags:** training, tensorflow, keras
  > **Comments:**
  > **username_1**: By any chance you are running it on a Jupyter notebook, can you share more of your code, especially about the training part ?  
  > **username_0**: I'm not. I'm new to this. I don't even know what a jupyter notebook is :P  
  > **username_2**: Without more information I don't think we can help you. A few things I can thinkg of: First of all, are you sure you mean iteration and not epoch? Secondly, are your data balanced (i.e. same number of samples in each class)? If not it is possible, if let's say 95% of your samples belong to a single class that your model is predicting this class only and achieving 0.95 accuracy. Finally, it is possible that it is just a very simple task and your model actually scores so high.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_5 on 2019-09-09. Score: 4 | Accepted answer
  
  It actually depends on a couple of things here -
  
  1. How many output classes do you have? If you have only 2 or 3 classes, it is a very easy task for the classifier that you have built. So, it is highly possible that convergence has occurred.
  2. As @Djib2011 mentioned already, if your input training set is not balanced and is heavier with one of the output classes (95%), then this accuracy you see makes sense but note that your model won't do well in production.
  3. Do not try to evaluate your model on the basis of your training accuracy. Test it on data your model has never seen before and then evaluate the classification accuracy making sure that your training/testing data is not heavy with one of the classes.
  
  
  ---
  
  ### Answer by username_3 on 2019-09-04. Score: 2
  
  As you have trained your model in batch\_size of 500. Weights has been updated for each batch therefore 600 times(300000/500) by the end of one epoch.  
  So, Your model generalized well. Check the predictions. If Predictions are well. Your model is ready.
  
  
  ---
  
  ### Answer by username_4 on 2019-09-09. Score: 1
  
  It can be normal and there might be nothing wrong with your model. If there is a very strong and clear correlation in your data(good separability) then a network can achive very high accuracy very fast. After reaching some value learning gets harder.
  
  
  ---
  
  