Cerebras-GPT vs. LLaMA AI Model Performance Comparison
>>Is 10000 bigger than 10050?<p>>>Yes, 10000 is bigger than 10050.<p>>But even the mighty ChatGPT often can't do simple math<p>GPT is bad at math because BPE input compression obfuscates individual digits. <a href="https://bbot.org/etc/gpt-math.png" rel="nofollow">https://bbot.org/etc/gpt-math.png</a> You'd be bad at math too if every number was scrambled.<p>The graph is from page 22 of the GPT-3 paper from 2020. <a href="https://arxiv.org/abs/2005.14165" rel="nofollow">https://arxiv.org/abs/2005.14165</a> Even with 175 billion parameters it can't reliably do four digit addition.<p>An example from 4 days ago of ChatGPT being as bad as you'd expect at string reversal: <a href="https://news.ycombinator.com/item?id=35297183" rel="nofollow">https://news.ycombinator.com/item?id=35297183</a><p>(Although, I just tested ChatGPT Mar 14 Version against the above question after doing a bunch of math prompting and it got it right...)