For anyone getting up to date with language AI, taliesinb recommended this paper as &quot;a real eye-opener&quot; in the discussion about Google&#x27;s Go AI <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=10984973" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=10984973</a> :<p>&quot;A Cognitive Neural Architecture Able to Learn and Communicate through Natural Language&quot;<p><a href="http:&#x2F;&#x2F;journals.plos.org&#x2F;plosone&#x2F;article?id=10.1371%2Fjournal.pone.0140866" rel="nofollow">http:&#x2F;&#x2F;journals.plos.org&#x2F;plosone&#x2F;article?id=10.1371%2Fjourna...</a><p>It uses a more complex (I daresay more advanced) model than the LSTM paper you linked, a model that can be trained to learn and use language constructs (in his words, &quot;mimic certain kinds of child-level cognition with TINY amounts of training data&quot;). Definitely worth a read.<p>This ANNABELL model doesn&#x27;t require much training input to go from blank slate to conversation-capable (1587 input sentences -&gt; 521 output sentences; see Appendix S1 for examples). This high learning efficiency might imply that it indeed resembles language-processing architecture in humans (which it was modelled after). At the low level it&#x27;s also neurons, but organised and connected into specific, planned subsystems.<p>A key point is that the central executive (i.e. the core, which controls the flow of data between slave systems) is a trainable neural network itself, which learns to generate the &quot;mental actions&quot; that control the flow of data between slave systems (like short-term and long-term memory), rather than rely on fixed rules to control the flow. This apparently allows the system to generalise.