Body: <p>I am trying to use a Keras LSTM neural network for character level language modelling. As the input, I give it the last 50 characters and it has to output the next one. It has 3 layers of 400 neurons each. For the training data, I am using 'War of The Worlds' by H.G. Wells which adds up to 269639 training samples and 67410 validation samples.</p>
  
  <p>After 7 epochs the validation accuracy has reached 35.1% and the validation loss has reached 2.31. However, after being fed the first sentence of war of the worlds to start it outputs:</p>
  
  <blockquote>
    <p>the the the the the the the the the the the the the the the the...</p>
  </blockquote>
  
  <p>I'm not sure where I'm going wrong; I don't want it to overfit and output passages straight from the training data but I also don't want it to just output 'the' repeatedly. I'm really at a loss as to what I should do to improve it.</p>
  
  <p>Any help would be greatly appreciated. Thanks!</p>
  
Tags: <neural-networks><natural-language-processing><recurrent-neural-networks><keras><long-short-term-memory>
Title: LSTM language model not working
UserDisplayName: user117279
ThreadText: # LSTM language model not working
  
  (Asked by: username_0 on 2018-12-30)
  
  I am trying to use a Keras LSTM neural network for character level language modelling. As the input, I give it the last 50 characters and it has to output the next one. It has 3 layers of 400 neurons each. For the training data, I am using 'War of The Worlds' by H.G. Wells which adds up to 269639 training samples and 67410 validation samples.
  
  After 7 epochs the validation accuracy has reached 35.1% and the validation loss has reached 2.31. However, after being fed the first sentence of war of the worlds to start it outputs:
  
  > the the the the the the the the the the the the the the the the...
  
  I'm not sure where I'm going wrong; I don't want it to overfit and output passages straight from the training data but I also don't want it to just output 'the' repeatedly. I'm really at a loss as to what I should do to improve it.
  
  Any help would be greatly appreciated. Thanks!
  
  **Tags:** neural-networks, natural-language-processing, recurrent-neural-networks, keras, long-short-term-memory
  
  ---
  
  ## 0 answers
  
  