Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards
  for Reasoning-Enhanced Text-to-SQL
Text-to-SQL is a challenging task involving multiple reasoning-intensive
subtasks, including natural language understanding, database schema
comprehension, and precise SQL query formulation. Existing approaches often
rely on handcrafted reasoning paths with inductive biases that can limit their
overall effectiveness. Motivated by the recent success of reasoning-enhanced
models such as DeepSeek R1 and OpenAI o1, which effectively leverage
reward-driven self-exploration to enhance reasoning capabilities and
generalization, we propose a novel set of partial rewards tailored specifically
for the Text-to-SQL task. Our reward set includes schema-linking, AI feedback,
n-gram similarity, and syntax check, explicitly designed to address the reward
sparsity issue prevalent in reinforcement learning (RL). Leveraging group
relative policy optimization (GRPO), our approach explicitly encourages large
language models (LLMs) to develop intrinsic reasoning skills necessary for
accurate SQL query generation. With models of different sizes, we demonstrate
that RL-only training with our proposed rewards consistently achieves higher
accuracy and superior generalization compared to supervised fine-tuning (SFT).
Remarkably, our RL-trained 14B-parameter model significantly outperforms larger
proprietary models, e.g. o3-mini by 4% and Gemini-1.5-Pro-002 by 3% on the BIRD
benchmark. These highlight the efficacy of our proposed RL-training framework
with partial rewards for enhancing both accuracy and reasoning capabilities in
Text-to-SQL tasks.
67ed3cd4ec3d31f7c7591f56
Mohammadreza Pourreza
admin_assigned
2025-04-02T13:55:55.210Z
61c9d9574534e41f9b58b5c8
/avatars/dd7dd0e57782217c726348abbd4fa9c6.svg
Mohammadreza Pourreza
user
MrezaPRZ
67ed3cd4ec3d31f7c7591f57
Shayan Talaei
admin_assigned
2025-04-02T13:56:45.205Z
65cd45d1b4b92671346f2d3d
/avatars/2814478bbd2e4ac4c19d437f265b247d.svg
Shayan Talaei
user
stalaei
67ed3cd4ec3d31f7c7591f58
Ruoxi Sun
admin_assigned
2025-04-02T13:57:00.441Z
653093ad471261a41241e048
/avatars/54263ce75b102b81211d27b696920870.svg
Ruoxi Sun
user
brycesun
67ed3cd4ec3d31f7c7591f59
Xingchen Wan
extracted_confirmed
2025-04-11T22:28:58.447Z
65bf213f8467e2a3d6374d4b
/avatars/0194cdba95d7a4c01fbbdd505e384a3d.svg
X Wan
user
masonxw
67ed3cd4ec3d31f7c7591f5a
Hailong Li
67ed3cd4ec3d31f7c7591f5b
Azalia Mirhoseini
extracted_pending
2025-04-02T13:34:14.113Z
66ba4e72c9b2ab14b3707be0
/avatars/97ef14d683ed2d0115a9c4694b9763dc.svg
Azalia Mirhoseini
user
am34
67ed3cd4ec3d31f7c7591f5c
Amin Saberi
67ed3cd4ec3d31f7c7591f5d
Sercan "O. Arik
extracted_pending
2025-05-08T21:22:51.624Z
680579efa4dd84280a275767
/avatars/f1905e850ae200105accba5aa220c049.svg
Sercan Arik
user
sercanarik
Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards
  for Reasoning-Enhanced Text-to-SQL
Text-to-SQL is a challenging task involving multiple reasoning-intensive
subtasks, including natural language understanding, database schema
comprehension, and precise SQL query formulation. Existing approaches often
rely on handcrafted reasoning paths with inductive biases that can limit their
overall effectiveness. Motivated by the recent success of reasoning-enhanced
models such as DeepSeek R1 and OpenAI o1, which effectively leverage
reward-driven self-exploration to enhance reasoning capabilities and
generalization, we propose a novel set of partial rewards tailored specifically
for the Text-to-SQL task. Our reward set includes schema-linking, AI feedback,
n-gram similarity, and syntax check, explicitly designed to address the reward
sparsity issue prevalent in reinforcement learning (RL). Leveraging group
relative policy optimization (GRPO), our approach explicitly encourages large
language models (LLMs) to develop intrinsic reasoning skills necessary for
accurate SQL query generation. With models of different sizes, we demonstrate
that RL-only training with our proposed rewards consistently achieves higher
accuracy and superior generalization compared to supervised fine-tuning (SFT).
Remarkably, our RL-trained 14B-parameter model significantly outperforms larger
proprietary models, e.g. o3-mini by 4% and Gemini-1.5-Pro-002 by 3% on the BIRD
benchmark. These highlight the efficacy of our proposed RL-training framework
with partial rewards for enhancing both accuracy and reasoning capabilities in
Text-to-SQL tasks.
A novel set of partial rewards for reinforcement learning improves the accuracy and generalization of large language models in the Text-to-SQL task compared to supervised fine-tuning.
Text-to-SQL natural language understanding database schema comprehension SQL query formulation reward-driven self-exploration reinforcement learning reward sparsity group relative policy optimization large language models intrinsic reasoning skills BIRD benchmark