title: Enigmata: Scaling Logical Reasoning in Large Language Models with
    Synthetic Verifiable Puzzles
summary: Large Language Models (LLMs), such as OpenAI's o1 and DeepSeek's R1, excel at
  advanced reasoning tasks like math and coding via Reinforcement Learning with
  Verifiable Rewards (RLVR), but still struggle with puzzles solvable by humans
  without domain knowledge. We introduce Enigmata, the first comprehensive suite
  tailored for improving LLMs with puzzle reasoning skills. It includes 36 tasks
  across seven categories, each with 1) a generator that produces unlimited
  examples with controllable difficulty and 2) a rule-based verifier for
  automatic evaluation. This generator-verifier design supports scalable,
  multi-task RL training, fine-grained analysis, and seamless RLVR integration.
  We further propose Enigmata-Eval, a rigorous benchmark, and develop optimized
  multi-task RLVR strategies. Our trained model, Qwen2.5-32B-Enigmata,
  consistently surpasses o3-mini-high and o1 on the puzzle reasoning benchmarks
  like Enigmata-Eval, ARC-AGI (32.8%), and ARC-AGI 2 (0.6%). It also generalizes
  well to out-of-domain puzzle benchmarks and mathematical reasoning, with little
  multi-tasking trade-off. When trained on larger models like Seed1.5-Thinking
  (20B activated parameters and 200B total parameters), puzzle data from Enigmata
  further boosts SoTA performance on advanced math and STEM reasoning tasks such
  as AIME (2024-2025), BeyondAIME and GPQA (Diamond), showing nice generalization
  benefits of Enigmata. This work offers a unified, controllable framework for
  advancing logical reasoning in LLMs. Resources of this work can be found at
  https://seed-enigmata.github.io.
paper_authors._id: 68353e41f995630ab88c198b
paper_authors.name: Jiangjie Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T07:49:21.006Z
paper_authors._id: 606ed1884ffe81d1e03e81e5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1639375346654-606ed1884ffe81d1e03e81e5.png
paper_authors.fullname: Jiangjie Chen
paper_authors.type: user
paper_authors.user: jiangjiechen
paper_authors._id: 68353e41f995630ab88c198c
paper_authors.name: Qianyu He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-27T07:49:23.290Z
paper_authors._id: 636b36351340f879a2ec2bb1
paper_authors.avatarUrl: /avatars/260a1c15f9c14c967125469072020946.svg
paper_authors.fullname: QianyuHe
paper_authors.type: user
paper_authors.user: Abbey4799
paper_authors._id: 68353e41f995630ab88c198d
paper_authors.name: Siyu Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-27T10:14:02.514Z
paper_authors._id: 62d62b333bf5e059f7d2b286
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1668513815771-62d62b333bf5e059f7d2b286.jpeg
paper_authors.fullname: Siyu Yuan
paper_authors.type: user
paper_authors.user: siyuyuan
paper_authors._id: 68353e41f995630ab88c198e
paper_authors.name: Aili Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T20:18:46.580Z
paper_authors._id: 63f86b099f87cc3e645b51d9
paper_authors.avatarUrl: /avatars/27ca5ba425640bf67474cee871e8e53a.svg
paper_authors.fullname: Ellie Chen
paper_authors.type: user
paper_authors.user: sheep33333
paper_authors._id: 68353e41f995630ab88c198f
paper_authors.name: Zhicheng Cai
paper_authors._id: 68353e41f995630ab88c1990
paper_authors.name: Weinan Dai
paper_authors._id: 68353e41f995630ab88c1991
paper_authors.name: Hongli Yu
paper_authors._id: 68353e41f995630ab88c1992
paper_authors.name: Qiying Yu
paper_authors._id: 68353e41f995630ab88c1993
paper_authors.name: Xuefeng Li
paper_authors._id: 68353e41f995630ab88c1994
paper_authors.name: Jiaze Chen
paper_authors._id: 68353e41f995630ab88c1995
paper_authors.name: Hao Zhou
paper_authors._id: 68353e41f995630ab88c1996
paper_authors.name: Mingxuan Wang
paper_title: Enigmata: Scaling Logical Reasoning in Large Language Models with
    Synthetic Verifiable Puzzles
paper_summary: Large Language Models (LLMs), such as OpenAI's o1 and DeepSeek's R1, excel at
  advanced reasoning tasks like math and coding via Reinforcement Learning with
  Verifiable Rewards (RLVR), but still struggle with puzzles solvable by humans
  without domain knowledge. We introduce Enigmata, the first comprehensive suite
  tailored for improving LLMs with puzzle reasoning skills. It includes 36 tasks
  across seven categories, each with 1) a generator that produces unlimited
  examples with controllable difficulty and 2) a rule-based verifier for
  automatic evaluation. This generator-verifier design supports scalable,
  multi-task RL training, fine-grained analysis, and seamless RLVR integration.
  We further propose Enigmata-Eval, a rigorous benchmark, and develop optimized
  multi-task RLVR strategies. Our trained model, Qwen2.5-32B-Enigmata,
  consistently surpasses o3-mini-high and o1 on the puzzle reasoning benchmarks
  like Enigmata-Eval, ARC-AGI (32.8%), and ARC-AGI 2 (0.6%). It also generalizes
  well to out-of-domain puzzle benchmarks and mathematical reasoning, with little
  multi-tasking trade-off. When trained on larger models like Seed1.5-Thinking
  (20B activated parameters and 200B total parameters), puzzle data from Enigmata
  further boosts SoTA performance on advanced math and STEM reasoning tasks such
  as AIME (2024-2025), BeyondAIME and GPQA (Diamond), showing nice generalization
  benefits of Enigmata. This work offers a unified, controllable framework for
  advancing logical reasoning in LLMs. Resources of this work can be found at
  https://seed-enigmata.github.io.
paper_ai_summary: Enigmata is a comprehensive suite for improving LLMs in puzzle reasoning through scalable multi-task RL training, leading to better performance on benchmarks and advanced math tasks.
paper_ai_keywords: Large Language Models OpenAI's o1 DeepSeek's R1 Reinforcement Learning with Verifiable Rewards Enigmata Enigmata-Eval multi-task RL training puzzle reasoning rule-based verifier ARC-AGI Qwen2.5-32B-Enigmata Seed1.5-Thinking AIME BeyondAIME GPQA