LLMs are Superior Feedback Providers: Bootstrapping Reasoning for Lie Detection with Self-Generated Feedback
<|reference_start|>LLMs are Superior Feedback Providers: Bootstrapping Reasoning for Lie Detection with Self-Generated Feedback: Large Language Models (LLMs) excel at generating human-like dialogues and comprehending text. However, understanding the subtleties of complex exchanges in language remains a challenge. We propose a bootstrapping framework that leverages self-generated feedback to enhance LLM reasoning capabilities for lie detection. The framework consists of three stages: suggestion, feedback collection, and modification. In the suggestion stage, a cost-effective language model generates initial predictions based on game state and dialogue. The feedback-collection stage involves a language model providing feedback on these predictions. In the modification stage, a more advanced language model refines the initial predictions using the auto-generated feedback. We investigate the application of the proposed framework for detecting betrayal and deception in Diplomacy games, and compare it with feedback from professional human players. The LLM-generated feedback exhibits superior quality and significantly enhances the performance of the model. Our approach achieves a 39% improvement over the zero-shot baseline in lying-F1 without the need for any training data, rivaling state-of-the-art supervised learning results.<|reference_end|>
@article{banerjee2024llms,
    title={LLMs are Superior Feedback Providers: Bootstrapping Reasoning for Lie
  Detection with Self-Generated Feedback},
    author={Tanushree Banerjee, Richard Zhu, Runzhe Yang, Karthik Narasimhan},
    journal={arXiv preprint arXiv:2408.13915},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.13915},
    primaryClass={cs.CL cs.AI}
}