ChatGPT's Chess Elo is 1400
This is so easy to disprove it makes it look like the author didn't even try.<p>Here is the convo I just had:<p>me: You are a chess grandmaster playing as black and your goal is to win in as few moves as possible. I will give you the move sequence, and you will return your next move. No explanation needed<p>ChatGPT: Sure, I'd be happy to help! Please provide the move sequence and I'll give you my response.<p>me: 1. e3<p>ChatGPT: 1... e5<p>me: 2. ne2<p>ChatGPT: 2... d6<p>me: 3. g4<p>ChatGPT: 3... h5<p>me: 4. g5<p>ChatGPT: 4... Ne4<p>Ne4 is illegal. Also you'd be hard pressed to find any human player that would miss the hanging pawns much less a 1400 elo player. GPT has been trained on chess games and is predicting patterns, this has been known since the GPT-2 days. The daily hype articles where someone coerces a statistical model into "demonstrating" an emergent capability sure are getting old. Claiming emergent capabilities (like having learned chess) are an extraordinary claim (LLMs most likely cant learn things) and I really wish people would put more effort when they make these claims. But the AI hype problem is as old as AI research itself...