title: SQL-PaLM: Improved Large Language ModelAdaptation for Text-to-SQL
summary: One impressive emergent capability of large language models (LLMs) is
  generation of code, including Structured Query Language (SQL) for databases.
  For the task of converting natural language text to SQL queries, Text-to-SQL,
  adaptation of LLMs is of paramount importance, both in in-context learning and
  fine-tuning settings, depending on the amount of adaptation data used. In this
  paper, we propose an LLM-based Text-to-SQL model SQL-PaLM, leveraging on
  PaLM-2, that pushes the state-of-the-art in both settings. Few-shot SQL-PaLM is
  based on an execution-based self-consistency prompting approach designed for
  Text-to-SQL, and achieves 77.3% in test-suite accuracy on Spider, which to our
  best knowledge is the first to outperform previous state-of-the-art with
  fine-tuning by a significant margin, 4%. Furthermore, we demonstrate that the
  fine-tuned SQL-PALM outperforms it further by another 1%. Towards applying
  SQL-PaLM to real-world scenarios we further evaluate its robustness on other
  challenging variants of Spider and demonstrate the superior generalization
  capability of SQL-PaLM. In addition, via extensive case studies, we demonstrate
  the impressive intelligent capabilities and various success enablers of
  LLM-based Text-to-SQL.
paper_authors._id: 64795f1da684545663563580
paper_authors.name: Ruoxi Sun
paper_authors._id: 64795f1da684545663563581
paper_authors.name: Sercan O Arik
paper_authors._id: 64795f1da684545663563582
paper_authors.name: Hootan Nakhost
paper_authors._id: 64795f1da684545663563583
paper_authors.name: Hanjun Dai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-05T10:59:04.742Z
paper_authors._id: 642bd29877967049cf4a7877
paper_authors.avatarUrl: /avatars/41ecd2074b152891073a4af1f5cc220a.svg
paper_authors.fullname: Hanjun Dai
paper_authors.type: user
paper_authors.user: kangaroo0123
paper_authors._id: 64795f1da684545663563584
paper_authors.name: Rajarishi Sinha
paper_authors._id: 64795f1da684545663563585
paper_authors.name: Pengcheng Yin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-05T11:00:11.053Z
paper_authors._id: 5e6147daf0c9e93097d0d4a5
paper_authors.avatarUrl: /avatars/7cf2bad604d44b1efa4730a629506d8a.svg
paper_authors.fullname: Pengcheng Yin
paper_authors.type: user
paper_authors.user: pcyin
paper_authors._id: 64795f1da684545663563586
paper_authors.name: Tomas Pfister
paper_title: SQL-PaLM: Improved Large Language ModelAdaptation for Text-to-SQL
paper_summary: One impressive emergent capability of large language models (LLMs) is
  generation of code, including Structured Query Language (SQL) for databases.
  For the task of converting natural language text to SQL queries, Text-to-SQL,
  adaptation of LLMs is of paramount importance, both in in-context learning and
  fine-tuning settings, depending on the amount of adaptation data used. In this
  paper, we propose an LLM-based Text-to-SQL model SQL-PaLM, leveraging on
  PaLM-2, that pushes the state-of-the-art in both settings. Few-shot SQL-PaLM is
  based on an execution-based self-consistency prompting approach designed for
  Text-to-SQL, and achieves 77.3% in test-suite accuracy on Spider, which to our
  best knowledge is the first to outperform previous state-of-the-art with
  fine-tuning by a significant margin, 4%. Furthermore, we demonstrate that the
  fine-tuned SQL-PALM outperforms it further by another 1%. Towards applying
  SQL-PaLM to real-world scenarios we further evaluate its robustness on other
  challenging variants of Spider and demonstrate the superior generalization
  capability of SQL-PaLM. In addition, via extensive case studies, we demonstrate
  the impressive intelligent capabilities and various success enablers of
  LLM-based Text-to-SQL.
paper_ai_summary: SQL-PaLM, an LLM-based Text-to-SQL model, outperforms existing systems in few-shot learning and fine-tuning settings and demonstrates superior generalization on diverse variants of the Spider dataset.
paper_ai_keywords: LLMs Text-to-SQL SQL-PaLM PaLM-2 execution-based self-consistency Spider dataset test-suite accuracy few-shot learning fine-tuning robustness generalization capability