TTT-Bench: A Benchmark for Evaluating Reasoning Ability with Simple and
  Novel Tic-Tac-Toe-style Games
Large reasoning models (LRMs) have demonstrated impressive reasoning
capabilities across a broad range of tasks including Olympiad-level
mathematical problems, indicating evidence of their complex reasoning
abilities. While many reasoning benchmarks focus on the STEM domain, the
ability of LRMs to reason correctly in broader task domains remains
underexplored. In this work, we introduce TTT-Bench, a new benchmark
that is designed to evaluate basic strategic, spatial, and logical reasoning
abilities in LRMs through a suite of four two-player Tic-Tac-Toe-style games
that humans can effortlessly solve from a young age. We propose a simple yet
scalable programmatic approach for generating verifiable two-player game
problems for TTT-Bench. Although these games are trivial for humans, they
require reasoning about the intentions of the opponent, as well as the game
board's spatial configurations, to ensure a win. We evaluate a diverse set of
state-of-the-art LRMs, and discover that the models that excel at hard
math problems frequently fail at these simple reasoning games. Further testing
reveals that our evaluated reasoning models score on average downarrow 41\%
\& downarrow 5\% lower on TTT-Bench compared to MATH 500 \& AIME 2024
respectively, with larger models achieving higher performance using shorter
reasoning traces, where most of the models struggle on long-term strategic
reasoning situations on simple and new TTT-Bench tasks.
684b73173b733ba333686f75
Prakamya Mishra
extracted_confirmed
2025-06-13T00:53:00.119Z
65bb2a8400e143d672b170c9
/avatars/e1d745d12f0a4c8dae95d2f6983d5201.svg
Prakamya Mishra
user
Prakamya
684b73173b733ba333686f76
Jiang Liu
684b73173b733ba333686f77
Jialian Wu
684b73173b733ba333686f78
Xiaodong Yu
684b73173b733ba333686f79
Zicheng Liu
684b73173b733ba333686f7a
Emad Barsoum
TTT-Bench: A Benchmark for Evaluating Reasoning Ability with Simple and
  Novel Tic-Tac-Toe-style Games
Large reasoning models (LRMs) have demonstrated impressive reasoning
capabilities across a broad range of tasks including Olympiad-level
mathematical problems, indicating evidence of their complex reasoning
abilities. While many reasoning benchmarks focus on the STEM domain, the
ability of LRMs to reason correctly in broader task domains remains
underexplored. In this work, we introduce TTT-Bench, a new benchmark
that is designed to evaluate basic strategic, spatial, and logical reasoning
abilities in LRMs through a suite of four two-player Tic-Tac-Toe-style games
that humans can effortlessly solve from a young age. We propose a simple yet
scalable programmatic approach for generating verifiable two-player game
problems for TTT-Bench. Although these games are trivial for humans, they
require reasoning about the intentions of the opponent, as well as the game
board's spatial configurations, to ensure a win. We evaluate a diverse set of
state-of-the-art LRMs, and discover that the models that excel at hard
math problems frequently fail at these simple reasoning games. Further testing
reveals that our evaluated reasoning models score on average downarrow 41\%
\& downarrow 5\% lower on TTT-Bench compared to MATH 500 \& AIME 2024
respectively, with larger models achieving higher performance using shorter
reasoning traces, where most of the models struggle on long-term strategic
reasoning situations on simple and new TTT-Bench tasks.
TTT-Bench evaluates strategic, spatial, and logical reasoning in large reasoning models through Tic-Tac-Toe-style games, revealing that models proficient in advanced math often struggle with these basic tasks.
large reasoning models reasoning benchmarks TTT-Bench two-player Tic-Tac-Toe-style games strategic reasoning spatial reasoning logical reasoning reasoning traces