AdaptEval: Evaluating Large Language Models on Domain Adaptation for Text Summarization
<|reference_start|>AdaptEval: Evaluating Large Language Models on Domain Adaptation for Text Summarization: Despite the advances in the abstractive summarization task using Large Language Models (LLM), there is a lack of research that asses their abilities to easily adapt to different domains. We evaluate the domain adaptation abilities of a wide range of LLMs on the summarization task across various domains in both fine-tuning and in-context learning settings. We also present AdaptEval, the first domain adaptation evaluation suite. AdaptEval includes a domain benchmark and a set of metrics to facilitate the analysis of domain adaptation. Our results demonstrate that LLMs exhibit comparable performance in the in-context learning setting, regardless of their parameter scale.<|reference_end|>
@article{afzal2024adapteval:,
    title={AdaptEval: Evaluating Large Language Models on Domain Adaptation for
  Text Summarization},
    author={Anum Afzal, Ribin Chalumattu, Florian Matthes, Laura Mascarell},
    journal={arXiv preprint arXiv:2407.11591},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.11591},
    primaryClass={cs.CL}
}