GPT-4 can't reason
Arithmetic problems have nothing to do with reasoning, and are a known weakness due to the tokenization schemes in these models.<p>Secondly, finding failure modes doesn&#x27;t mean that the model doesn&#x27;t have any reasoning ability. Humans can reason, despite the fact that high school students are pretty bad at formal logic.<p>So, the conclusion is over broad, and the paper fails to incorporate existing knowledge about these models. Kinda crap.