ChatGPT has no inner monologue or meta-cognition
I noticed a weird phenomenon at work, which is surprising considering I work with software engineers: some people just accept ChatGPT as an oracle (especially version 4).<p>Suddenly, in discussions and code reviews I see weird statements and explanations being presented with utter confidence, only to be gravely mistaken, and then I ask if this explanation came from ChatGPT, and you can see the instant embarrassment (I stopped asking this for this reason, but they know I know).<p>More people need to understand this. ChatGPT and LLMs are not AGI, they have no consciousness, they are not self aware. They don&#x27;t &quot;know&quot; what they are doing, who you are or what they are. They are unable to &quot;know&quot; anything the way people do.<p>Chatbots based on LLMs are simple machines, while the LLMs can definitely output text that has value and is impressive, it does not come from a sentient intelligence, and it can produce so much garbage. It is not an oracle.<p>Most importantly, LLMs see diminishing returns after a certain number of parameters and they are still unable to perform well in real problem solving such as advanced programming tests.<p>There is something yet missing for true AGI capable of problem solving. A LLM is just not enough.