title: Crosslingual Reasoning through Test-Time Scaling
summary: Reasoning capabilities of large language models are primarily studied for
  English, even when pretrained models are multilingual. In this work, we
  investigate to what extent English reasoning finetuning with long
  chain-of-thoughts (CoTs) can generalize across languages. First, we find that
  scaling up inference compute for English-centric reasoning language models
  (RLMs) improves multilingual mathematical reasoning across many languages
  including low-resource languages, to an extent where they outperform models
  twice their size. Second, we reveal that while English-centric RLM's CoTs are
  naturally predominantly English, they consistently follow a quote-and-think
  pattern to reason about quoted non-English inputs. Third, we discover an
  effective strategy to control the language of long CoT reasoning, and we
  observe that models reason better and more efficiently in high-resource
  languages. Finally, we observe poor out-of-domain reasoning generalization, in
  particular from STEM to cultural commonsense knowledge, even for English.
  Overall, we demonstrate the potentials, study the mechanisms and outline the
  limitations of crosslingual generalization of English reasoning test-time
  scaling. We conclude that practitioners should let English-centric RLMs reason
  in high-resource languages, while further work is needed to improve reasoning
  in low-resource languages and out-of-domain contexts.
paper_authors._id: 681daba2e3775056736651ce
paper_authors.name: Zheng-Xin Yong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-09T07:20:45.630Z
paper_authors._id: 61424bf4f0d914a5f606a823
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61424bf4f0d914a5f606a823/0td8lR4elBaVvJUD9Pojh.png
paper_authors.fullname: Yong Zheng-Xin
paper_authors.type: user
paper_authors.user: yongzx
paper_authors._id: 681daba2e3775056736651cf
paper_authors.name: M. Farid Adilazuarda
paper_authors._id: 681daba2e3775056736651d0
paper_authors.name: Jonibek Mansurov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T09:03:39.843Z
paper_authors._id: 6509feb92257a3afbaeecfea
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6509feb92257a3afbaeecfea/a_UbA-2WtZeLTf0ugVzSh.jpeg
paper_authors.fullname: Jonibek Mansurov
paper_authors.type: user
paper_authors.user: MJonibek
paper_authors._id: 681daba2e3775056736651d1
paper_authors.name: Ruochen Zhang
paper_authors._id: 681daba2e3775056736651d2
paper_authors.name: Niklas Muennighoff
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T09:04:00.900Z
paper_authors._id: 5f1eb362eec0ad2a071ad6e2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5f1eb362eec0ad2a071ad6e2/IXMYkYKuTwn6kBdWnQeeY.png
paper_authors.fullname: Niklas Muennighoff
paper_authors.type: user
paper_authors.user: Muennighoff
paper_authors._id: 681daba2e3775056736651d3
paper_authors.name: Carsten Eickhoff
paper_authors._id: 681daba2e3775056736651d4
paper_authors.name: Genta Indra Winata
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T09:04:13.878Z
paper_authors._id: 5f5c4b20e56d546cd6233098
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1637813888895-5f5c4b20e56d546cd6233098.jpeg
paper_authors.fullname: Genta Indra Winata
paper_authors.type: user
paper_authors.user: gentaiscool
paper_authors._id: 681daba2e3775056736651d5
paper_authors.name: Julia Kreutzer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-09T09:04:29.257Z
paper_authors._id: 6544e43b12da508864c38f96
paper_authors.avatarUrl: /avatars/76f0cd55b4bf9c03d2686e146c6f795f.svg
paper_authors.fullname: Julia Kreutzer
paper_authors.type: user
paper_authors.user: JuliaKreutzerCohere
paper_authors._id: 681daba2e3775056736651d6
paper_authors.name: Stephen H. Bach
paper_authors._id: 681daba2e3775056736651d7
paper_authors.name: Alham Fikri Aji
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-12T06:50:58.421Z
paper_authors._id: 61a4dc053205e107691e0d82
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61a4dc053205e107691e0d82/BESoEHlHYXstXudh6dOdT.jpeg
paper_authors.fullname: Alham Fikri Aji
paper_authors.type: user
paper_authors.user: afaji
paper_title: Crosslingual Reasoning through Test-Time Scaling
paper_summary: Reasoning capabilities of large language models are primarily studied for
  English, even when pretrained models are multilingual. In this work, we
  investigate to what extent English reasoning finetuning with long
  chain-of-thoughts (CoTs) can generalize across languages. First, we find that
  scaling up inference compute for English-centric reasoning language models
  (RLMs) improves multilingual mathematical reasoning across many languages
  including low-resource languages, to an extent where they outperform models
  twice their size. Second, we reveal that while English-centric RLM's CoTs are
  naturally predominantly English, they consistently follow a quote-and-think
  pattern to reason about quoted non-English inputs. Third, we discover an
  effective strategy to control the language of long CoT reasoning, and we
  observe that models reason better and more efficiently in high-resource
  languages. Finally, we observe poor out-of-domain reasoning generalization, in
  particular from STEM to cultural commonsense knowledge, even for English.
  Overall, we demonstrate the potentials, study the mechanisms and outline the
  limitations of crosslingual generalization of English reasoning test-time
  scaling. We conclude that practitioners should let English-centric RLMs reason
  in high-resource languages, while further work is needed to improve reasoning
  in low-resource languages and out-of-domain contexts.
paper_ai_summary: Scaling English-centric reasoning models and controlling chain-of-thought language improves multilingual mathematical reasoning but has limitations in low-resource languages and out-of-domain contexts.
paper_ai_keywords: reasoning language models chain-of-thoughts English-centric multilingual long chain-of-thoughts low-resource languages high-resource languages out-of-domain reasoning cultural commonsense knowledge STEM