SEGMENT+: Long Text Processing with Short-Context Language Models
<|reference_start|>SEGMENT+: Long Text Processing with Short-Context Language Models: There is a growing interest in expanding the input capacity of language models (LMs) across various domains. However, simply increasing the context window does not guarantee robust performance across diverse long-input processing tasks, such as understanding extensive documents and extracting detailed information from lengthy and noisy data. In response, we introduce SEGMENT+, a general framework that enables LMs to handle extended inputs within limited context windows efficiently. SEGMENT+ utilizes structured notes and a filtering module to manage information flow, resulting in a system that is both controllable and interpretable. Our extensive experiments across various model sizes, focusing on long-document question-answering and Needle-in-a-Haystack tasks, demonstrate the effectiveness of SEGMENT+ in improving performance.<|reference_end|>
@article{shi2024segment+:,
    title={SEGMENT+: Long Text Processing with Short-Context Language Models},
    author={Wei Shi, Shuang Li, Kerun Yu, Jinglei Chen, Zujie Liang, Xinhui Wu,
  Yuxi Qian, Feng Wei, Bo Zheng, Jiaqing Liang, Jiangjie Chen, Yanghua Xiao},
    journal={arXiv preprint arXiv:2410.06519},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.06519},
    primaryClass={cs.CL}
}