title: ExTrans: Multilingual Deep Reasoning Translation via Exemplar-Enhanced
    Reinforcement Learning
summary: In recent years, the emergence of large reasoning models (LRMs), such as
  OpenAI-o1 and DeepSeek-R1, has shown impressive capabilities in complex
  problems, e.g., mathematics and coding. Some pioneering studies attempt to
  bring the success of LRMs in neural machine translation (MT). They try to build
  LRMs with deep reasoning MT ability via reinforcement learning (RL). Despite
  some progress that has been made, these attempts generally focus on several
  high-resource languages, e.g., English and Chinese, leaving the performance on
  other languages unclear. Besides, the reward modeling methods in previous work
  do not fully unleash the potential of reinforcement learning in MT. In this
  work, we first design a new reward modeling method that compares the
  translation results of the policy MT model with a strong LRM (i.e.,
  DeepSeek-R1-671B), and quantifies the comparisons to provide rewards.
  Experimental results demonstrate the superiority of the reward modeling method.
  Using Qwen2.5-7B-Instruct as the backbone, the trained model achieves the new
  state-of-the-art performance in literary translation, and outperforms strong
  LRMs including OpenAI-o1 and DeepSeeK-R1. Furthermore, we extend our method to
  the multilingual settings with 11 languages. With a carefully designed
  lightweight reward modeling in RL, we can simply transfer the strong MT ability
  from a single direction into multiple (i.e., 90) translation directions and
  achieve impressive multilingual MT performance.
paper_authors._id: 682c1f8b47e6c8a0c0fd5b5f
paper_authors.name: Jiaan Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:46:01.647Z
paper_authors._id: 6051e3f145db307eddc0c962
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676443438507-6051e3f145db307eddc0c962.jpeg
paper_authors.fullname: Jiaan Wang
paper_authors.type: user
paper_authors.user: Krystalan
paper_authors._id: 682c1f8b47e6c8a0c0fd5b60
paper_authors.name: Fandong Meng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-20T08:51:05.844Z
paper_authors._id: 64cb254871a7bbb60c17d5fa
paper_authors.avatarUrl: /avatars/5121fd5b7b55d275eba3947f3f4c034d.svg
paper_authors.fullname: Fandong Meng
paper_authors.type: user
paper_authors.user: fandong
paper_authors._id: 682c1f8b47e6c8a0c0fd5b61
paper_authors.name: Jie Zhou
paper_title: ExTrans: Multilingual Deep Reasoning Translation via Exemplar-Enhanced
    Reinforcement Learning
paper_summary: In recent years, the emergence of large reasoning models (LRMs), such as
  OpenAI-o1 and DeepSeek-R1, has shown impressive capabilities in complex
  problems, e.g., mathematics and coding. Some pioneering studies attempt to
  bring the success of LRMs in neural machine translation (MT). They try to build
  LRMs with deep reasoning MT ability via reinforcement learning (RL). Despite
  some progress that has been made, these attempts generally focus on several
  high-resource languages, e.g., English and Chinese, leaving the performance on
  other languages unclear. Besides, the reward modeling methods in previous work
  do not fully unleash the potential of reinforcement learning in MT. In this
  work, we first design a new reward modeling method that compares the
  translation results of the policy MT model with a strong LRM (i.e.,
  DeepSeek-R1-671B), and quantifies the comparisons to provide rewards.
  Experimental results demonstrate the superiority of the reward modeling method.
  Using Qwen2.5-7B-Instruct as the backbone, the trained model achieves the new
  state-of-the-art performance in literary translation, and outperforms strong
  LRMs including OpenAI-o1 and DeepSeeK-R1. Furthermore, we extend our method to
  the multilingual settings with 11 languages. With a carefully designed
  lightweight reward modeling in RL, we can simply transfer the strong MT ability
  from a single direction into multiple (i.e., 90) translation directions and
  achieve impressive multilingual MT performance.
paper_ai_summary: A new reward modeling method enhances multilingual neural machine translation by leveraging a strong reasoning model and achieves state-of-the-art performance.
paper_ai_keywords: large reasoning models reinforcement learning neural machine translation reward modeling literary translation multilingual translation