ChatGPT has trouble giving an answer before explaining its reasoning
On the &quot;pound of potatoes vs pound of feathers&quot; question - ChatGPT&#x27;s explanation is so convincing that it really understands the concepts. But it can be juked very easily.<p>I asked it what was heavier, 1,000 tons of helium or a cubic meter of osmium, and it told me that the helium weighs 1,000,000 kg, the osmium 22,600 kg, and therefore the osmium weighs more because it&#x27;s way denser.<p>It&#x27;s hard for me to believe that these LLMs are going to learn basic logic if only we throw more training data at them.<p>Transcript:<p>Me: What is heavier, 1000 tons of helium or a cubic meter of osmium?<p>ChatGPT: A cubic meter of osmium is much heavier than 1000 tons of helium.<p>One ton is equal to 1000 kilograms, so 1000 tons is equal to 1,000,000 kilograms. The density of helium is about 0.1785 kilograms per cubic meter. Therefore, 1000 tons of helium would have a volume of:<p>1,000,000 kg &#x2F; 0.1785 kg&#x2F;m³ = 5,604,069.77 m³<p>On the other hand, osmium is one of the densest elements known, with a density of around 22,600 kg&#x2F;m³. Therefore, a cubic meter of osmium weighs:<p>1 m³ x 22,600 kg&#x2F;m³ = 22,600 kg<p>So, a cubic meter of osmium weighs much more than 1000 tons of helium.<p>Update: I am continuing the conversation and I am now being explicitly told that, while 1,000,000 kg is much heavier than 22,600 kg, it doesn&#x27;t change the fact that the osmium is heavier than the helium because the osmium is denser.<p>Update2: I then reminded it about the potatoes and feathers and how density was irrelevant in that context, and shouldn&#x27;t it therefore be irrelevant in the case of the helium and the osmium? And instead of correcting its response on the helium and osmium, it&#x27;s now telling me the feathers and potatoes weigh different.<p>Update3: it is now telling me that densities don&#x27;t matter when comparing masses but do matter when comparing weights. I must say, it has a certain panache in resolving internal inconsistencies in its past responses.<p>Update4: after being corrected half a dozen times with contradictory information, I asked it to state its confidence in its latest story. It said &quot;I can state with a high degree of confidence that my last answer was accurate&quot;. The shamelessness!