GPT-4 can't reason
There are some serious problems with this paper, namely that I just tried to reproduce it and it failed every test: I tested out several of the problems presented in the paper which it was claimed that GPT-4 failed on and it passed every one every time.<p>I used the standard chat.openai.com web interface with no special or additional prompting.<p>It seems like there are others that have the same issues in this thread, which raises the question - what went wrong here? I can&#x27;t see which version of GPT-4 the paper uses mentioned anywhere, did anyone else spot this?<p>I&#x27;ve contacted the author and included this thread, so hopefully we get some insight into what&#x27;s happening here. To clarify, I am not accusing the author of anything and on the contrary I recognize that OpenAI is rather opaque about the models and changes them frequently. That said, the responses from GPT-4 in the paper do not match my personal experience using GPT-4 with reasoning tasks at any point during the last several months, which is why I am curious if the author may have accidentally used GPT-3.5.