title: M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
summary: Effective reasoning is crucial to solving complex mathematical problems.
  Recent large language models (LLMs) have boosted performance by scaling
  test-time computation through long chain-of-thought reasoning. However,
  transformer-based models are inherently limited in extending context length due
  to their quadratic computational complexity and linear memory requirements. In
  this paper, we introduce a novel hybrid linear RNN reasoning model, M1, built
  on the Mamba architecture, which allows memory-efficient inference. Our
  approach leverages a distillation process from existing reasoning models and is
  further enhanced through RL training. Experimental results on the AIME and MATH
  benchmarks show that M1 not only outperforms previous linear RNN models but
  also matches the performance of state-of-the-art Deepseek R1 distilled
  reasoning models at a similar scale. We also compare our generation speed with
  a highly performant general purpose inference engine, vLLM, and observe more
  than a 3x speedup compared to a same size transformer. With throughput speedup,
  we are able to achieve higher accuracy compared to DeepSeek R1 distilled
  transformer reasoning models under a fixed generation time budget using
  self-consistency voting. Overall, we introduce a hybrid Mamba reasoning model
  and provide a more effective approach to scaling test-time generation using
  self-consistency or long chain of thought reasoning.
paper_authors._id: 67fe15543aa5a5684ca7229e
paper_authors.name: Junxiong Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:35:36.587Z
paper_authors._id: 63dc68bea99b2c8a7c20f1d0
paper_authors.avatarUrl: /avatars/cf8ddc91415ef1f895803f4390ff1f6f.svg
paper_authors.fullname: Junxiong Wang
paper_authors.type: user
paper_authors.user: JunxiongWang
paper_authors._id: 67fe15543aa5a5684ca7229f
paper_authors.name: Wen-Ding Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:35:44.671Z
paper_authors._id: 62e221dfcb1f164f2cb8a66b
paper_authors.avatarUrl: /avatars/06f05622e232304d3f0b8c291f3263be.svg
paper_authors.fullname: Wen-Ding Li
paper_authors.type: user
paper_authors.user: xu3kev
paper_authors._id: 67fe15543aa5a5684ca722a0
paper_authors.name: Daniele Paliotta
paper_authors._id: 67fe15543aa5a5684ca722a1
paper_authors.name: Daniel Ritter
paper_authors._id: 67fe15543aa5a5684ca722a2
paper_authors.name: Alexander M. Rush
paper_authors._id: 67fe15543aa5a5684ca722a3
paper_authors.name: Tri Dao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-15T08:36:25.942Z
paper_authors._id: 64b8a6b5cf14c2fabe98159b
paper_authors.avatarUrl: /avatars/dbc009451865435bf290791beadc4723.svg
paper_authors.fullname: Tri Dao
paper_authors.type: user
paper_authors.user: tridao
paper_title: M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
paper_summary: Effective reasoning is crucial to solving complex mathematical problems.
  Recent large language models (LLMs) have boosted performance by scaling
  test-time computation through long chain-of-thought reasoning. However,
  transformer-based models are inherently limited in extending context length due
  to their quadratic computational complexity and linear memory requirements. In
  this paper, we introduce a novel hybrid linear RNN reasoning model, M1, built
  on the Mamba architecture, which allows memory-efficient inference. Our
  approach leverages a distillation process from existing reasoning models and is
  further enhanced through RL training. Experimental results on the AIME and MATH
  benchmarks show that M1 not only outperforms previous linear RNN models but
  also matches the performance of state-of-the-art Deepseek R1 distilled
  reasoning models at a similar scale. We also compare our generation speed with
  a highly performant general purpose inference engine, vLLM, and observe more
  than a 3x speedup compared to a same size transformer. With throughput speedup,
  we are able to achieve higher accuracy compared to DeepSeek R1 distilled
  transformer reasoning models under a fixed generation time budget using
  self-consistency voting. Overall, we introduce a hybrid Mamba reasoning model
  and provide a more effective approach to scaling test-time generation using
  self-consistency or long chain of thought reasoning.
paper_ai_summary: A hybrid linear RNN reasoning model, M1, based on the Mamba architecture, achieves memory-efficient inference and outperforms other models, including DeepSeek R1, with faster generation speed and higher accuracy.
paper_ai_keywords: LLMs chain-of-thought reasoning transformer-based models computational complexity memory requirements hybrid linear RNN Mamba architecture distillation process RL training AIME MATH benchmarks Deepseek R1 vLLM self-consistency voting