Answers.Body: <blockquote>
    <p>Why would giving my AI more data make it perform worse?</p>
  </blockquote>
  
  <p>A lot of possible reasons:</p>
  
  <ol>
  <li>In forecasting, you could have a seasonality. If you have it exactly 3 times, then it is good. If you have it 3.5 times, it becomes worse because it overfits on the months which occur more often than the others.</li>
  <li>Data quality: In practice, improving the quality of data often yields better results (see <a href="https://arxiv.org/pdf/1707.09725.pdf" rel="nofollow noreferrer">Analysis and Optimization of Convolutional Neural Network Architectures, page 15 (Analysis Techniques)</a> for 7 approaches to improve your results)</li>
  <li>Instable training / bad luck: some architectures / problems are super instable. You can execute the randomly initialized training 5 times and get vastly different results.</li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-23T19:32:52.043Z
Answers.LastActivityDate: 2018-09-25T06:31:49.260Z
Answers.LastEditDate: 2018-09-25T06:31:49.260Z
Answers.UserDisplayName: Martin Thoma
Body: <p>So I trained an AI to generate shakespeare, which it did somewhat well. I used <a href="https://cs.stanford.edu/people/karpathy/char-rnn/shakespear.txt" rel="nofollow noreferrer">this 10,000 character sample</a>.</p>
  
  <p>Next I tried to get it to generate limericks using these <a href="http://oedilf.com/" rel="nofollow noreferrer">100,000 limericks</a>. It generated garbage output.</p>
  
  <p>When I limited it to 10,000 characters, it then started giving reasonable limerick output.</p>
  
  <p>How could this happen? I thought more data was always better.</p>
  
  <p>The AI was a neural network with some LSTM layers, implemented in keras.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-23T06:18:36.673Z
Comments.Text: Maybe give the architecture of your LSTM, plus short samples of what you consider "garbage" and "reasonable limerick output". Also the losses or learning curves or other training logs might be useful to help with analysis.
Comments.UserDisplayName: Neil Slater
Tags: <python><recurrent-neural-networks><datasets><keras><long-short-term-memory>
Title: Why would giving my AI more data make it perform worse?
UserDisplayName: Christopher King
ThreadText: # Why would giving my AI more data make it perform worse?
  
  (Asked by: username_0 on 2018-09-23)
  
  So I trained an AI to generate shakespeare, which it did somewhat well. I used [this 10,000 character sample](https://cs.stanford.edu/people/karpathy/char-rnn/shakespear.txt).
  
  Next I tried to get it to generate limericks using these [100,000 limericks](http://oedilf.com/). It generated garbage output.
  
  When I limited it to 10,000 characters, it then started giving reasonable limerick output.
  
  How could this happen? I thought more data was always better.
  
  The AI was a neural network with some LSTM layers, implemented in keras.
  
  **Tags:** python, recurrent-neural-networks, datasets, keras, long-short-term-memory
  > **Comments:**
  > **username_1**: Maybe give the architecture of your LSTM, plus short samples of what you consider "garbage" and "reasonable limerick output". Also the losses or learning curves or other training logs might be useful to help with analysis.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-09-23. Score: 2
  
  > Why would giving my AI more data make it perform worse?
  
  A lot of possible reasons:
  
  1. In forecasting, you could have a seasonality. If you have it exactly 3 times, then it is good. If you have it 3.5 times, it becomes worse because it overfits on the months which occur more often than the others.
  2. Data quality: In practice, improving the quality of data often yields better results (see [Analysis and Optimization of Convolutional Neural Network Architectures, page 15 (Analysis Techniques)](https://arxiv.org/pdf/1707.09725.pdf) for 7 approaches to improve your results)
  3. Instable training / bad luck: some architectures / problems are super instable. You can execute the randomly initialized training 5 times and get vastly different results.
  
  
  ---
  
  