Fun hack. If anything, it highlights how compelling deep learning and RNNs are: no messing with NLP, no messing with building other features or adding up classifiers, etc.  The manual feature engineering means it might work better on a smaller dataset, but even then probably not.<p>For comparison with Andrej Karpathy&#x27;s RNN code (<a href="http:&#x2F;&#x2F;karpathy.github.io&#x2F;2015&#x2F;05&#x2F;21&#x2F;rnn-effectiveness&#x2F;" rel="nofollow">http:&#x2F;&#x2F;karpathy.github.io&#x2F;2015&#x2F;05&#x2F;21&#x2F;rnn-effectiveness&#x2F;</a>) training on the &quot;HarryPotter(xxlarge).txt&quot; (76K) file using the default hyperparameters and a batch size of 25 gets me:<p><pre><code>  &gt; But Atfa the loom proset! No contarin — mibll,’s just pucking to live
  &gt; note left them hard and fitther, clooked of course little happered to
  &gt; trige on the fistpened. Their knew Harry mear from the shind-beas
  &gt; eveided, at Uncle Vernon’s thepped to spept were pelled and beadn
  &gt; Harry, distine dy use. Harry had in a amalout, into the fish sfary door.
</code></pre>
The difference here is tokenizing on words vs letters: the RNN code is trying to learn the structure of English from completely zero whereas the code here gets to work with well-formed words from the beginning. But otherwise, the results in the linked post are about as silly semantically:<p><pre><code>  &gt; Input: &quot;Harry don&#x27;t look&quot;
  &gt; Output: &quot;Harry don&#x27;t look , incredibly that a year for been parents in .
  &gt;   followers , Harry , and Potter was been curse . Harry was up a year ,
  &gt;   Harry was been curse &quot;
</code></pre>
EDIT: Updated the RNN output text. Was sampling from a checkpoint file for a different input corpus.  Got confused by the long similar-looking filenames. Doesn&#x27;t change the overall point though.