A skeptical take on ChatGPT: Ezra Klein interviews Gary Marcus
I&#x27;ve seen threads involving Gary Marcus on twitter, and when people provide concrete evidence that his point of view is wrong, he just stops replying, then proceeds to continue to spout the disproven claim in other places.<p>Just the other day, when he claimed that GPT was literally just doing memorization&#x2F;word statistics&#x2F;syntax and has no grasp of semantics, some folks demonstrated that GPT can literally act as an interpreter for arbitrary code.<p>(there are some tricks involved here, you have to get it to interpret the program &quot;with pen and paper&quot; by getting it to record all the state updates &#x2F; variable mutations that happen in the code, this can be done by inserting copious print statements)<p>He then claimed that it was only able to interpret this program because it must have seen it before in its vast training data. He accused a commenter of not understanding how big the training data was.<p>It was then shown that GPT can interpret a python program that is operating on two randomly chosen large integers, the combination of which is <i>certainly</i> not in its training data. This shows that it must be &quot;understanding&quot; (for lack of a better word) the semantics of the program. Gary then stopped responding.<p>I don&#x27;t think GPT <i>on its own</i> will lead anywhere close to AGI -- and I don&#x27;t think anyone serious thinks this. But GPT combined with a sophisticated pipeline of wrapper scripts to feed its own output back into itself in clever ways and give it access to external data sources and tools? Possibly!