EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling
<|reference_start|>EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling: Recently, Large Language Models (LLMs) have demonstrated outstanding performance across a wide range of downstream language tasks. Temperature sampling is a commonly used decoding strategy for LLMs' generation process. However, a fixed temperature parameter is used in most cases, which may not always be an optimal choice for balancing generation quality and diversity. In this paper, we propose an effective Entropy-based Dynamic Temperature (EDT) Sampling method, to achieve a more balanced performance in terms of both generation quality and diversity by dynamically selecting the temperature parameter. Additionally, we also show model performance and comprehensive analyses for 4 different generation benchmarks. Our experiments show that EDT significantly outperforms the existing strategies across different tasks.<|reference_end|>
@article{zhang2024edt:,
    title={EDT: Improving Large Language Models' Generation by Entropy-based
  Dynamic Temperature Sampling},
    author={Shimao Zhang, Yu Bao, Shujian Huang},
    journal={arXiv preprint arXiv:2403.14541},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.14541},
    primaryClass={cs.CL}
}