SQL-PaLM: Improved Large Language ModelAdaptation for Text-to-SQL
One impressive emergent capability of large language models (LLMs) is
generation of code, including Structured Query Language (SQL) for databases.
For the task of converting natural language text to SQL queries, Text-to-SQL,
adaptation of LLMs is of paramount importance, both in in-context learning and
fine-tuning settings, depending on the amount of adaptation data used. In this
paper, we propose an LLM-based Text-to-SQL model SQL-PaLM, leveraging on
PaLM-2, that pushes the state-of-the-art in both settings. Few-shot SQL-PaLM is
based on an execution-based self-consistency prompting approach designed for
Text-to-SQL, and achieves 77.3% in test-suite accuracy on Spider, which to our
best knowledge is the first to outperform previous state-of-the-art with
fine-tuning by a significant margin, 4%. Furthermore, we demonstrate that the
fine-tuned SQL-PALM outperforms it further by another 1%. Towards applying
SQL-PaLM to real-world scenarios we further evaluate its robustness on other
challenging variants of Spider and demonstrate the superior generalization
capability of SQL-PaLM. In addition, via extensive case studies, we demonstrate
the impressive intelligent capabilities and various success enablers of
LLM-based Text-to-SQL.
64795f1da684545663563580
Ruoxi Sun
64795f1da684545663563581
Sercan O Arik
64795f1da684545663563582
Hootan Nakhost
64795f1da684545663563583
Hanjun Dai
admin_assigned
2023-06-05T10:59:04.742Z
642bd29877967049cf4a7877
/avatars/41ecd2074b152891073a4af1f5cc220a.svg
Hanjun Dai
user
kangaroo0123
64795f1da684545663563584
Rajarishi Sinha
64795f1da684545663563585
Pengcheng Yin
admin_assigned
2023-06-05T11:00:11.053Z
5e6147daf0c9e93097d0d4a5
/avatars/7cf2bad604d44b1efa4730a629506d8a.svg
Pengcheng Yin
user
pcyin
64795f1da684545663563586
Tomas Pfister
SQL-PaLM: Improved Large Language ModelAdaptation for Text-to-SQL
One impressive emergent capability of large language models (LLMs) is
generation of code, including Structured Query Language (SQL) for databases.
For the task of converting natural language text to SQL queries, Text-to-SQL,
adaptation of LLMs is of paramount importance, both in in-context learning and
fine-tuning settings, depending on the amount of adaptation data used. In this
paper, we propose an LLM-based Text-to-SQL model SQL-PaLM, leveraging on
PaLM-2, that pushes the state-of-the-art in both settings. Few-shot SQL-PaLM is
based on an execution-based self-consistency prompting approach designed for
Text-to-SQL, and achieves 77.3% in test-suite accuracy on Spider, which to our
best knowledge is the first to outperform previous state-of-the-art with
fine-tuning by a significant margin, 4%. Furthermore, we demonstrate that the
fine-tuned SQL-PALM outperforms it further by another 1%. Towards applying
SQL-PaLM to real-world scenarios we further evaluate its robustness on other
challenging variants of Spider and demonstrate the superior generalization
capability of SQL-PaLM. In addition, via extensive case studies, we demonstrate
the impressive intelligent capabilities and various success enablers of
LLM-based Text-to-SQL.
SQL-PaLM, an LLM-based Text-to-SQL model, outperforms existing systems in few-shot learning and fine-tuning settings and demonstrates superior generalization on diverse variants of the Spider dataset.
LLMs Text-to-SQL SQL-PaLM PaLM-2 execution-based self-consistency Spider dataset test-suite accuracy few-shot learning fine-tuning robustness generalization capability