Atomic Self-Consistency for Better Long Form Generations
<|reference_start|>Atomic Self-Consistency for Better Long Form Generations: Recent work has aimed to improve LLM generations by filtering out hallucinations, thereby improving the precision of the information in responses. Correctness of a long-form response, however, also depends on the recall of multiple pieces of information relevant to the question. In this paper, we introduce Atomic Self-Consistency (ASC), a technique for improving the recall of relevant information in an LLM response. ASC follows recent work, Universal Self-Consistency (USC) in using multiple stochastic samples from an LLM to improve the long-form response. Unlike USC which only focuses on selecting the best single generation, ASC picks authentic subparts from the samples and merges them into a superior composite answer. Through extensive experiments and ablations, we show that merging relevant subparts of multiple samples performs significantly better than picking a single sample. ASC demonstrates significant gains over USC on multiple factoids and open-ended QA datasets - ASQA, QAMPARI, QUEST, ELI5 with ChatGPT and Llama2. Our analysis also reveals untapped potential for enhancing long-form generations using approach of merging multiple samples.<|reference_end|>
@article{thirukovalluru2024atomic,
    title={Atomic Self-Consistency for Better Long Form Generations},
    author={Raghuveer Thirukovalluru, Yukun Huang, Bhuwan Dhingra},
    journal={arXiv preprint arXiv:2405.13131},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.13131},
    primaryClass={cs.CL}
}