<p>I am trying to use a Keras LSTM neural network for character level language modelling. As the input, I give it the last 50 characters and it has to output the next one. It has 3 layers of 400 neurons each. For the training data, I am using 'War of The Worlds' by H.G. Wells which adds up to 269639 training samples and 67410 validation samples.</p>

<p>After 7 epochs the validation accuracy has reached 35.1% and the validation loss has reached 2.31. However, after being fed the first sentence of war of the worlds to start it outputs:</p>

<blockquote>
  <p>the the the the the the the the the the the the the the the the...</p>
</blockquote>

<p>I'm not sure where I'm going wrong; I don't want it to overfit and output passages straight from the training data but I also don't want it to just output 'the' repeatedly. I'm really at a loss as to what I should do to improve it.</p>

<p>Any help would be greatly appreciated. Thanks!</p>

<neural-networks><natural-language-processing><recurrent-neural-networks><keras><long-short-term-memory>
LSTM language model not working
user117279
# LSTM language model not working

(Asked by: username_0 on 2018-12-30)

I am trying to use a Keras LSTM neural network for character level language modelling. As the input, I give it the last 50 characters and it has to output the next one. It has 3 layers of 400 neurons each. For the training data, I am using 'War of The Worlds' by H.G. Wells which adds up to 269639 training samples and 67410 validation samples.

After 7 epochs the validation accuracy has reached 35.1% and the validation loss has reached 2.31. However, after being fed the first sentence of war of the worlds to start it outputs:

> the the the the the the the the the the the the the the the the...

I'm not sure where I'm going wrong; I don't want it to overfit and output passages straight from the training data but I also don't want it to just output 'the' repeatedly. I'm really at a loss as to what I should do to improve it.

Any help would be greatly appreciated. Thanks!

**Tags:** neural-networks, natural-language-processing, recurrent-neural-networks, keras, long-short-term-memory


---

## 0 answers

