Recent Advances in Natural Language Processing
Separate comment because separate point.<p>To my mind, all this benchmark-beating is just pointless faffing about and not a little frustrating to see that this is where the current state of the art is at. NLP systems keep getting better and better at beating benchmarks and yet we still don&#x27;t have artificial agents that can hold an intelligent conversation, even in a restricted context.<p>Or rather, we had such programs since the late 1960&#x27;s, with no one else&#x27;s than Terry Winograd&#x27;s own doctoral thesis system, SHRDLU [1], but it seems that in the race to build powerful classifiers, we forgot how to build systems that can interact intelligently with a human.<p>Here&#x27;s a small excerpt from SHRDLU (taken from the wikipedia page on SHRDLU, originally in an appendix in Terry Winograd&#x27;s thesis):<p><pre><code>    Person: Is there a large block behind a pyramid?
    Computer: YES, THREE OF THEM: A LARGE RED ONE, A LARGE GREEN CUBE, AND THE BLUE ONE.
    Person: Put a small one onto the green cube which supports a pyramid.
    Computer: OK.
    Person: Put the littlest pyramid on top of it.
    Computer: OK.
    Person: Does the shortest thing the tallest pyramid&#x27;s support supports support anything green?
    Computer: YES, THE GREEN PYRAMID.
    Person: What color is it?
    Computer: BY &quot;IT&quot;, I ASSUME YOU MEAN THE SHORTEST THING THE TALLEST PYRAMID&#x27;S SUPPORT SUPPORTS.
    Computer: RED
</code></pre>
Note that this is not just mindless text generation. The conversation is held in the context of a &quot;blocks world&quot; where a robot with a single hand and eye (&quot;Computer&quot; in the excerpt above) moves blocks of various shapes and colours around, as directed by a human user in free-form natural language. When the Computer says &quot;OK&quot; after it&#x27;s directed to &quot;put the littlest pyramid on top of it&quot; it&#x27;s because it really has grabbed the smallest pyramid in the blocks world and placed it on top of the small block in an earlier sentence, as the Person asked. The program has a memory module to keep track of what ellipses like &quot;it&quot;, &quot;one&quot; etc refer to throughout the conversation.<p>SHRDLU was a traditional program hand-crafted by a single PhD student- no machine learning, no statistical techniques. It included, among other things, a context-free grammar (!) of natural English and a planner (to control the robot&#x27;s hand) all written in Lisp and PLANNER. In its limited domain, it was smarter than anything ever created with statisical NLP methods.<p>______________________<p>[1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;SHRDLU" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;SHRDLU</a>