Text2SQL is Not Enough: Unifying AI and Databases with TAG
AI systems that serve natural language questions over databases promise to
unlock tremendous value. Such systems would allow users to leverage the
powerful reasoning and knowledge capabilities of language models (LMs)
alongside the scalable computational power of data management systems. These
combined capabilities would empower users to ask arbitrary natural language
questions over custom data sources. However, existing methods and benchmarks
insufficiently explore this setting. Text2SQL methods focus solely on natural
language questions that can be expressed in relational algebra, representing a
small subset of the questions real users wish to ask. Likewise,
Retrieval-Augmented Generation (RAG) considers the limited subset of queries
that can be answered with point lookups to one or a few data records within the
database. We propose Table-Augmented Generation (TAG), a unified and
general-purpose paradigm for answering natural language questions over
databases. The TAG model represents a wide range of interactions between the LM
and database that have been previously unexplored and creates exciting research
opportunities for leveraging the world knowledge and reasoning capabilities of
LMs over data. We systematically develop benchmarks to study the TAG problem
and find that standard methods answer no more than 20% of queries correctly,
confirming the need for further research in this area. We release code for the
benchmark at https://github.com/TAG-Research/TAG-Bench.
66ce9234a3ce189b050205d0
Asim Biswal
admin_assigned
2024-08-28T09:42:57.964Z
653c19f303ff30a76c238d1e
/avatars/52bb64f21a988b574fb55c4bf27ad529.svg
Asim Biswal
user
abiswal
66ce9234a3ce189b050205d1
Liana Patel
66ce9234a3ce189b050205d2
Siddarth Jha
admin_assigned
2024-08-28T09:48:03.368Z
6672a2f8b3c23292e5e45e6a
/avatars/fc5617dde79d3f57c56e463c11c29ebe.svg
Siddarth Jha
user
jhasi1
66ce9234a3ce189b050205d3
Amog Kamsetty
admin_assigned
2024-08-28T09:48:16.464Z
6010826305d1c56b5b370bb9
https://cdn-avatars.huggingface.co/v1/production/uploads/1611694686233-noauth.jpeg
Amog Kamsetty
user
amogkam
66ce9234a3ce189b050205d4
Shu Liu
claimed_verified
2024-08-29T07:23:35.490Z
65401959cdc151e46270f6fb
/avatars/b7e71a9c68cdabd432a7ae9e8e640c4f.svg
Shu Liu
user
lynnliu030
66ce9234a3ce189b050205d5
Joseph E. Gonzalez
admin_assigned
2024-08-28T09:48:41.422Z
645d2e8401f4eaab2a0878ce
/avatars/1273c5fb607b4b622a746a42692fa632.svg
Joseph E. Gonzalez
user
ProfJoeyG
66ce9234a3ce189b050205d6
Carlos Guestrin
66ce9234a3ce189b050205d7
Matei Zaharia
admin_assigned
2024-08-28T09:48:58.766Z
642509c6d476e4ad5568dd31
https://cdn-avatars.huggingface.co/v1/production/uploads/642509c6d476e4ad5568dd31/dPiEV70hMeuY-cveLNPQg.jpeg
Matei Zaharia
user
mateiz
Text2SQL is Not Enough: Unifying AI and Databases with TAG
AI systems that serve natural language questions over databases promise to
unlock tremendous value. Such systems would allow users to leverage the
powerful reasoning and knowledge capabilities of language models (LMs)
alongside the scalable computational power of data management systems. These
combined capabilities would empower users to ask arbitrary natural language
questions over custom data sources. However, existing methods and benchmarks
insufficiently explore this setting. Text2SQL methods focus solely on natural
language questions that can be expressed in relational algebra, representing a
small subset of the questions real users wish to ask. Likewise,
Retrieval-Augmented Generation (RAG) considers the limited subset of queries
that can be answered with point lookups to one or a few data records within the
database. We propose Table-Augmented Generation (TAG), a unified and
general-purpose paradigm for answering natural language questions over
databases. The TAG model represents a wide range of interactions between the LM
and database that have been previously unexplored and creates exciting research
opportunities for leveraging the world knowledge and reasoning capabilities of
LMs over data. We systematically develop benchmarks to study the TAG problem
and find that standard methods answer no more than 20% of queries correctly,
confirming the need for further research in this area. We release code for the
benchmark at https://github.com/TAG-Research/TAG-Bench.
A unified Table-Augmented Generation (TAG) paradigm addresses limitations of existing Text2SQL and retrieval-based methods, enabling a broader range of natural language queries over databases with improved accuracy and flexibility.
Text2SQL Retrieval-Augmented Generation RAG Table-Augmented Generation TAG natural language questions relational algebra point lookups data records database interactions world knowledge reasoning capabilities benchmarks