An Open Recipe: Adapting Language-Specific LLMs to a Reasoning Model in
  One Day via Model Merging
This paper investigates data selection and model merging methodologies aimed
at incorporating advanced reasoning capabilities such as those of DeepSeek R1
into language-specific large language models (LLMs), with a particular focus on
the Thai LLM. Our goal is to enhance the reasoning capabilities of
language-specific LLMs while maintaining their target language abilities.
DeepSeek R1 excels in reasoning but primarily benefits high-resource languages
such as English and Chinese. However, low-resource languages remain underserved
due to the dominance of English-centric training data and model optimizations,
which limit performance in these languages. This limitation results in
unreliable code-switching and diminished effectiveness on tasks in low-resource
languages. Meanwhile, local and regional LLM initiatives have attempted to
bridge this gap by developing language-specific LLMs that focus on improving
local linguistic fidelity. We demonstrate that, with only publicly available
datasets and a computational budget of $120, it is possible to enhance the
reasoning capabilities of language-specific LLMs to match the level of DeepSeek
R1, without compromising their performance on target language tasks.
67aea8d7926b659c7e959bbc
Kunat Pipatanakul
admin_assigned
2025-02-14T12:04:29.437Z
62d192c2d50433c35eb1b48e
https://cdn-avatars.huggingface.co/v1/production/uploads/62d192c2d50433c35eb1b48e/VjmDu8GOIuLuQNBQdQLLS.png
Kunat Pipatanakul
user
kunato
67aea8d7926b659c7e959bbd
Pittawat Taveekitworachai
claimed_verified
2025-02-14T08:01:21.838Z
615313b0793ef66b3324da1f
https://cdn-avatars.huggingface.co/v1/production/uploads/615313b0793ef66b3324da1f/VyJniD3dxbV5a2CMgVVQ2.jpeg
Pittawat Taveekitworachai
user
pittawat
67aea8d7926b659c7e959bbe
Potsawee Manakul
admin_assigned
2025-02-14T12:04:35.434Z
63f6a050b4c9a104f4b95755
https://cdn-avatars.huggingface.co/v1/production/uploads/63f6a050b4c9a104f4b95755/eJQyJkenSz536j-EGcpkH.jpeg
Potsawee Manakul
user
potsawee
67aea8d7926b659c7e959bbf
Kasima Tharnpipitchai
An Open Recipe: Adapting Language-Specific LLMs to a Reasoning Model in
  One Day via Model Merging
This paper investigates data selection and model merging methodologies aimed
at incorporating advanced reasoning capabilities such as those of DeepSeek R1
into language-specific large language models (LLMs), with a particular focus on
the Thai LLM. Our goal is to enhance the reasoning capabilities of
language-specific LLMs while maintaining their target language abilities.
DeepSeek R1 excels in reasoning but primarily benefits high-resource languages
such as English and Chinese. However, low-resource languages remain underserved
due to the dominance of English-centric training data and model optimizations,
which limit performance in these languages. This limitation results in
unreliable code-switching and diminished effectiveness on tasks in low-resource
languages. Meanwhile, local and regional LLM initiatives have attempted to
bridge this gap by developing language-specific LLMs that focus on improving
local linguistic fidelity. We demonstrate that, with only publicly available
datasets and a computational budget of $120, it is possible to enhance the
reasoning capabilities of language-specific LLMs to match the level of DeepSeek
R1, without compromising their performance on target language tasks.
The paper enhances the reasoning capabilities of language-specific LLMs, particularly for low-resource languages like Thai, by merging data selection methods and integrating advanced reasoning models.
DeepSeek R1 language-specific large language models data selection model merging reasoning capabilities low-resource languages English-centric training data computational budget local linguistic fidelity target language tasks