Why are LLMs general learners?
I&#x27;m not sure I&#x27;m personally convinced LLMs are bad at arithmetic, I think they might just approach it differently to us.<p>Something you&#x27;ll find if you ever train a neural network to learn a mathematical function is that it will only ever approximate that function. It won&#x27;t try to guess what the function is exactly like a human might do.<p>For example consider, f(1) = 2, f(2) = 4, f(3) = 6, f(4) = 8, f(5) = 10.<p>As a human you know how important precision is in maths and you know generally humans like round numbers so you naturally assume that, f(x) = x<i>2<p>Neural networks don&#x27;t have these biases by default. They&#x27;ll look for a function that gets close enough maybe something like, f(x) = x</i>1.993929910302942223<p>From a neural network&#x27;s perspective the loss between this answer and the actual answer is almost so trivial that it&#x27;s basically irrelevant.<p>Then a human who likes round numbers comes along and asks the network, what&#x27;s f(1,000)? To which the neural network replies, 19939.3<p>Then the human then goes away convinced the AI doesn&#x27;t know maths, when in reality the AI basically does know maths, it just doesn&#x27;t care as much about aromatic precession as the human does. Because again, to the AI 19939.3 is a perfectly acceptable answer.<p>So now for fun let me ask ChatGPT some arithmetic questions...<p>&gt; ME<p>&gt; what&#x27;s 2343423 + 9988733?<p>&gt; ChatGPT<p>&gt; The sum of 2343423 and 9988733 is 12392156.<p>WRONG! It&#x27;s actually 12332156. That&#x27;s an entire digit out and almost 0.5% larger than the actual answer!<p>&gt; ME<p>&gt; what is 8379270 + 387299177?<p>&gt; ChatGPT<p>&gt; The sum of 8379270 and 387299177 is 395678447.<p>Er, okay, that was right. Bad example, let me try again.<p>&gt; ME<p>&gt; what is 2233322223333 + 387299177?<p>&gt; ChatGPT<p>&gt; The sum of 2233322223333 and 387299177 is 2233322610510.<p>WRONG! It&#x27;s actually 2233709522510. That&#x27;s 6 digits out and almost 0.02% smaller than the actual answer!<p>If you take a more open minded view I think it&#x27;s fair to say ChatGPT basically does know arithmetic, but its reward function probably didn&#x27;t prioritise arithmetic precision in the same way a decade of schooling does for us humans. For ChatGPT having a few digits wrong in an arithmetic problem is probably less important that its reply containing that sum being slightly improperly worded.<p>I guess what I&#x27;m saying is that I&#x27;m not sure I quite agree with the author that LLMs don&#x27;t do arithmetic at all. It&#x27;s not that they&#x27;re trying to guess the next word without arithmetic, but more that they&#x27;re not doing arithmetic the same as we humans do it. Which is may have been the point the author was making... I&#x27;m not really sure.