title: Pensez: Less Data, Better Reasoning -- Rethinking French LLM
summary: Large language models (LLMs) have demonstrated remarkable capabilities in
  various natural language processing tasks. However, achieving strong
  performance in specialized domains like mathematical reasoning and non-English
  languages often requires extensive training on massive datasets. This paper
  investigates a contrasting approach: strategic fine-tuning on a small,
  high-quality, bilingual (English-French) dataset to enhance both the reasoning
  capabilities and French language proficiency of a large language model. Rather
  than relying on scale, we explore the hypothesis that targeted data curation
  and optimized training can achieve competitive, or even superior, performance.
  We demonstrate, through targeted supervised fine-tuning (SFT) on only 2,000
  carefully selected samples, significant improvements in mathematical reasoning.
  Specifically, Pensez 7B exhibits an increase in accuracy of the base model up
  to 20% on the AIME25 and a 12% increase on a French MATH level 5 benchmark.
  These results challenge the prevailing assumption that massive datasets are
  aprerequisite for strong reasoning performance in LLMs, highlighting the
  potential of strategic data curation and optimized fine-tuning for enhancing
  both specialized skills and multilingual capabilities. Our findings have
  implications for the efficient development of high-performing, multilingual
  LLMs, especially in resource-constrained scenarios.
paper_authors._id: 67da8a9fdab8cc723c349fb0
paper_authors.name: Huy Hoang Ha
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-19T09:43:52.541Z
paper_authors._id: 630a5ef0e81e1dea2cedcec0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/630a5ef0e81e1dea2cedcec0/ATtyCvYoX4z7uxsm2sJU2.png
paper_authors.fullname: Hà Huy Hoàng
paper_authors.type: user
paper_authors.user: HoangHa
paper_title: Pensez: Less Data, Better Reasoning -- Rethinking French LLM
paper_summary: Large language models (LLMs) have demonstrated remarkable capabilities in
  various natural language processing tasks. However, achieving strong
  performance in specialized domains like mathematical reasoning and non-English
  languages often requires extensive training on massive datasets. This paper
  investigates a contrasting approach: strategic fine-tuning on a small,
  high-quality, bilingual (English-French) dataset to enhance both the reasoning
  capabilities and French language proficiency of a large language model. Rather
  than relying on scale, we explore the hypothesis that targeted data curation
  and optimized training can achieve competitive, or even superior, performance.
  We demonstrate, through targeted supervised fine-tuning (SFT) on only 2,000
  carefully selected samples, significant improvements in mathematical reasoning.
  Specifically, Pensez 7B exhibits an increase in accuracy of the base model up
  to 20% on the AIME25 and a 12% increase on a French MATH level 5 benchmark.
  These results challenge the prevailing assumption that massive datasets are
  aprerequisite for strong reasoning performance in LLMs, highlighting the
  potential of strategic data curation and optimized fine-tuning for enhancing
  both specialized skills and multilingual capabilities. Our findings have
  implications for the efficient development of high-performing, multilingual
  LLMs, especially in resource-constrained scenarios.
paper_ai_summary: Strategic fine-tuning on a small, high-quality bilingual dataset significantly enhances the mathematical reasoning and French language capabilities of a large language model.
paper_ai_keywords: large language models specialized domains mathematical reasoning non-English languages strategic fine-tuning bilingual (English-French) dataset targeted data curation optimized training supervised fine-tuning (SFT) AIME25 French MATH level 5 benchmark multilingual LLMs resource-constrained scenarios