ChatGPT's Chess Elo is 1400
This is so easy to disprove it makes it look like the author didn't even try.Here is the convo I just had:me: You are a chess grandmaster playing as black and your goal is to win in as few moves as possible. I will give you the move sequence, and you will return your next move. No explanation neededChatGPT: Sure, I'd be happy to help! Please provide the move sequence and I'll give you my response.me: 1. e3ChatGPT: 1... e5me: 2. ne2ChatGPT: 2... d6me: 3. g4ChatGPT: 3... h5me: 4. g5ChatGPT: 4... Ne4Ne4 is illegal. Also you'd be hard pressed to find any human player that would miss the hanging pawns much less a 1400 elo player. GPT has been trained on chess games and is predicting patterns, this has been known since the GPT-2 days. The daily hype articles where someone coerces a statistical model into "demonstrating" an emergent capability sure are getting old. Claiming emergent capabilities (like having learned chess) are an extraordinary claim (LLMs most likely cant learn things) and I really wish people would put more effort when they make these claims. But the AI hype problem is as old as AI research itself...