Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell
<|reference_start|>Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell: Large Language Models (LLMs) exhibit positional bias, struggling to utilize information from the middle or end of long contexts. Our study explores LLMs' long-context reasoning by probing their hidden representations. We find that while LLMs encode the position of target information, they often fail to leverage this in generating accurate responses. This reveals a disconnect between information retrieval and utilization, a "know but don't tell" phenomenon. We further analyze the relationship between extraction time and final accuracy, offering insights into the underlying mechanics of transformer models.<|reference_end|>
@article{lu2024insights,
    title={Insights into LLM Long-Context Failures: When Transformers Know but
  Don't Tell},
    author={Taiming Lu, Muhan Gao, Kuai Yu, Adam Byerly, Daniel Khashabi},
    journal={arXiv preprint arXiv:2406.14673},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.14673},
    primaryClass={cs.CL}
}