I asked GPT-NeoX-20B a hundred arithmetic questions
What? You think this is poor performance?<p>This totally blows my mind. I would never have guessed that GPT could get ANY of these right.<p>I mean, is there a data point in the dataset used to train where you can read 2241 + 19873 = 22114? Quite unlikely...<p>And those multiplications. It's consistently getting the number of digits right and the first two numbers correct. How the hell does this happen?<p>Sure, it's sometimes way off. But generally it is in the right ballpark.<p>I certainly think people should look into what's happening inside the model.