What ChatGPT can't do
I&#x27;m not sure I buy the author&#x27;s strict implied definition of &quot;understanding&quot;. Yes, ChatGPT sometimes makes mistakes, contradicts itself, and loses track of its own reasoning. Those are all things that humans do frequently, although the frequency of those mistakes and in what contexts they&#x27;re made is different. I have on many occasions gotten minutes into a discussion before realizing that I was simultaneously arguing for and against my own position, and having to step back and reexamine. Yet I&#x27;m pretty sure that I still understand things.<p>Certainly, there are certain topics where ChatGPT can be said to utterly and consistently fail to understand, but again, there are many topics where a human will consistently fail to understand as well. I once sat in the middle of an intro philosophy class during which a room full of adults with highschool diplomas failed for thirty minutes straight to understand that &quot;if X then Y&quot; is not the same as &quot;if Y then X&quot; despite the TA&#x27;s efforts using multiple real-world examples. I would not infer from that that those humans were incapable of understanding in general, but rather that they had not yet developed the cognitive tools to deal with logic abstractly.<p>I&#x27;ve also seen plenty of examples of humans following the format of an explanation or argument while filling in the details with vague jargon and inapplicable nonsense, making their words sound official and superficially convincing, without actually doing the cognitive work to say anything useful. In fact, I&#x27;ve seen that in documents written by people in their capacity as professional engineers and managers. If you&#x27;ve worked in tech for any significant time, you&#x27;ve probably seen it too. But it would be wild for me to claim that the authors of those documents were generally incapable of understanding.<p>Does that mean that ChatGPT definitely understands things? No. I think it means that we don&#x27;t really know what we mean by words like &quot;understand&quot; yet, because up until now, we&#x27;ve only had to apply words like that to human brains, and we&#x27;re generally happy to operate in a framework where we assume that human brains mostly work more or less the same way.<p>But I have noticed a trend of people dramatically underestimating the actual practical things that GPT-3 can do based on their assumptions about how it works and what its limitations are.