Ask HN: How do we know ChatGPT is not a scam?
&gt; Where it somehow never has any trouble with is correctly interpreting the task that a user is asking for. It somehow gets this done, even if the task is complex, requires deep language understanding and meta knowledge about the current situation ... This alone has been the holy grail of language processing for the last 40 years. It&#x27;s something no other network so far has even come close to. ... All of this makes me believe we&#x27;re either truly on the verge of AGI - or the network is in fact a human and the whole thing is fake.<p>This is exactly what struck me first when I used the model. Let me share with you two &quot;conversations&quot; that I had this model.<p>So I picked up a C programming textbook problem involving the itoa() function. The problem gives you an implementation of the itoa() function and tells you to verify that it does not work with the largest negative number, and modify the implementation so that it works.
I ask ChatGPT whether it knows about the itoa() function, it says yes and gives me its own implementation. I give it the implementation in the textbook and tell it to identify problems in this implementation.
I prompt it to identify problems four times. Out of these four times, only once does it identify a valid problem (buffer overflow), while the other three times it identifies nonsensical problems. Anyway, each of these times, it provides a valid code snippet along with a detailed explanation.
The fifth time, I give it a hint. I ask it, &quot;Here&#x27;s a hint. What is the largest negative number?&quot; And immediately, it realizes that the problem is that this implementation doesn&#x27;t work with the largest negative number, explains why the code doesn&#x27;t work with that number, and provides a valid code snippet. Now, the amazing thing is that I never told it that the problem is that it doesn&#x27;t work with the largest negative number - I just asked it what is that number.
Here&#x27;s something even more baffling. I next asked it which book this problem comes from. It didn&#x27;t have an answer and returned an &quot;I&#x27;m sorry ...&quot; message. But then I gave it a hint: &quot;The problem is Exercise 3-4 in this book.&quot; And from <i>that</i> - it guessed the book, Kernighan and Ritchie. It quoted the problem from that book, and added a note telling how nice that book is and how good it is for beginners.<p>As you&#x27;re saying, this model seems have to in a single shot <i>solved</i> the holy grail of natural language: comprehension, fluency, clear understanding of intent, maintaining context in dialogue, world knowledge, information retrieval. Till now, we had seen separate models for each of these problems, and not with so much success.<p>What surprises me the most is the clear understanding of intent. Let me come to the second conversation. Inspired by some posts I saw in the internet to make it act like a virtual machine, I ask it to download my personal blog post repository, start a Jekyll server and render the result in the Lynx browser. Giving it only the name of the GitHub repository, it correctly guesses the URL and uses the right command to start the server. I ask it how many files are there in the _drafts folder of the repository. It says there are none. I say that my repository has 4 Markdown files there. Then it correctly comes with 4 Markdown files, though their names are hypothetical. I tell it that the Lynx browser that it has rendered inside a code block has the wrong menu bar; specifically the menu bar doesn&#x27;t have an entry called &quot;Blog&quot;. It returns me a menu bar with only that entry removed. I tell it that the menu bar is still wrong; specifically it should start with the entries H = Help, K = Keystrokes and O = Options. It returns me a menu bar that looks like this: H = Help, K = Keystrokes, O = Options, P = Preferences, X = Exit.<p>Every single time, it understands extremely precisely what I am saying. All the machine-learning models we have seen till now always have had some amount of natural error when it comes to understanding intent: they don&#x27;t get exactly what you are saying, but they pick up something similar that they have seen in their training data. And many people have argued over the years that such precise understanding cannot come from the pattern-matching algorithms that we have in machine learning today: they can only come from an agent having symbolic knowledge, both knowledge of its own environment and that of the world.<p>Are these humans in a call center? Even if you forget about the superhuman response time, which human can have so much world knowledge?<p>A human plus a model? Even then, such coordination would have to be extremely well orchestrated.