title: Entropy-Based Adaptive Weighting for Self-Training
summary: The mathematical problem-solving capabilities of large language models have
  become a focal point of research, with growing interests in leveraging
  self-generated reasoning paths as a promising way to refine and enhance these
  models. These paths capture step-by-step logical processes while requiring only
  the correct answer for supervision. The self-training method has been shown to
  be effective in reasoning tasks while eliminating the need for external models
  and manual annotations. However, optimizing the use of self-generated data for
  model training remains an open challenge. In this work, we propose
  Entropy-Based Adaptive Weighting for Self-Training (EAST), an adaptive
  weighting strategy designed to prioritize uncertain data during self-training.
  Specifically, EAST employs a mapping function with a tunable parameter that
  controls the sharpness of the weighting, assigning higher weights to data where
  the model exhibits greater uncertainty. This approach guides the model to focus
  on more informative and challenging examples, thereby enhancing its reasoning
  ability. We evaluate our approach on GSM8K and MATH benchmarks. Empirical
  results show that, while the vanilla method yields virtually no improvement
  (0%) on MATH, EAST achieves around a 1% gain over backbone model. On GSM8K,
  EAST attains a further 1-2% performance boost compared to the vanilla method.
paper_authors._id: 67ebaea15baac6e5085afcb9
paper_authors.name: Xiaoxuan Wang
paper_authors._id: 67ebaea15baac6e5085afcba
paper_authors.name: Yihe Deng
paper_authors._id: 67ebaea15baac6e5085afcbb
paper_authors.name: Mingyu Derek Ma
paper_authors._id: 67ebaea15baac6e5085afcbc
paper_authors.name: Wei Wang
paper_title: Entropy-Based Adaptive Weighting for Self-Training
paper_summary: The mathematical problem-solving capabilities of large language models have
  become a focal point of research, with growing interests in leveraging
  self-generated reasoning paths as a promising way to refine and enhance these
  models. These paths capture step-by-step logical processes while requiring only
  the correct answer for supervision. The self-training method has been shown to
  be effective in reasoning tasks while eliminating the need for external models
  and manual annotations. However, optimizing the use of self-generated data for
  model training remains an open challenge. In this work, we propose
  Entropy-Based Adaptive Weighting for Self-Training (EAST), an adaptive
  weighting strategy designed to prioritize uncertain data during self-training.
  Specifically, EAST employs a mapping function with a tunable parameter that
  controls the sharpness of the weighting, assigning higher weights to data where
  the model exhibits greater uncertainty. This approach guides the model to focus
  on more informative and challenging examples, thereby enhancing its reasoning
  ability. We evaluate our approach on GSM8K and MATH benchmarks. Empirical
  results show that, while the vanilla method yields virtually no improvement
  (0%) on MATH, EAST achieves around a 1% gain over backbone model. On GSM8K,
  EAST attains a further 1-2% performance boost compared to the vanilla method.
paper_ai_summary: Entropy-Based Adaptive Weighting for Self-Training (EAST) enhances large language models' reasoning capabilities by prioritizing uncertain data, resulting in improved performance on benchmarks like GSM8K and MATH.
paper_ai_keywords: Entropy-Based Adaptive Weighting EAST adaptive weighting strategy self-training model uncertainty self-generated data GSM8K MATH