Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigm
<|reference_start|>Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigm: In-context learning of large-language models (LLMs) has achieved remarkable success in the field of natural language processing, while extensive case studies reveal that the single-step chain-of-thought prompting approach faces challenges such as attention diffusion and inadequate performance in complex tasks like text-to-SQL. To improve the contextual learning capabilities of LLMs in text-to-SQL, a workflow paradigm method is proposed, aiming to enhance the attention and problem-solving scope of LLMs through decomposition. Specifically, the information determination module for eliminating redundant information and the brand-new prompt structure based on problem classification greatly enhance the model's attention. Additionally, the inclusion of self-correction and active learning modules greatly expands the problem-solving scope of LLMs, hence improving the upper limit of LLM-based approaches. Extensive experiments conducted on three datasets demonstrate that our approach outperforms other methods by a significant margin. About 2-3 percentage point improvements compared to the existing baseline on the Spider Dev, Spider-Realistic, and Bird Dev datasets and new SOTA results on the Spider Test dataset are achieved. Our code is available on GitHub: \url{https://github.com/FlyingFeather/DEA-SQL}.<|reference_end|>
@article{xie2024decomposition,
    title={Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL
  through Workflow Paradigm},
    author={Yuanzhen Xie, Xinzhou Jin, Tao Xie, MingXiong Lin, Liang Chen, Chenyun
  Yu, Lei Cheng, ChengXiang Zhuo, Bo Hu, Zang Li},
    journal={arXiv preprint arXiv:2402.10671},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.10671},
    primaryClass={cs.CL}
}