title: Table-GPT: Table-tuned GPT for Diverse Table Tasks
summary: Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable
  abilities to follow diverse human instructions and perform a wide range of
  tasks. However, when probing language models using a range of basic
  table-understanding tasks, we observe that today's language models are still
  sub-optimal in many table-related tasks, likely because they are pre-trained
  predominantly on one-dimensional natural-language texts, whereas
  relational tables are two-dimensional objects.
    In this work, we propose a new "table-tuning" paradigm, where we
  continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using
  diverse table-tasks synthesized from real tables as training data, with the
  goal of enhancing language models' ability to understand tables and perform
  table tasks. We show that our resulting Table-GPT models demonstrate (1) better
  table-understanding capabilities, by consistently outperforming the
  vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout
  unseen tasks, and (2) strong generalizability, in its ability to respond
  to diverse human instructions to perform new table-tasks, in a manner similar
  to GPT-3.5 and ChatGPT.
paper_authors._id: 652c9b55a2d97e682b62b270
paper_authors.name: Peng Li
paper_authors._id: 652c9b55a2d97e682b62b271
paper_authors.name: Yeye He
paper_authors._id: 652c9b55a2d97e682b62b272
paper_authors.name: Dror Yashar
paper_authors._id: 652c9b55a2d97e682b62b273
paper_authors.name: Weiwei Cui
paper_authors._id: 652c9b55a2d97e682b62b274
paper_authors.name: Song Ge
paper_authors._id: 652c9b55a2d97e682b62b275
paper_authors.name: Haidong Zhang
paper_authors._id: 652c9b55a2d97e682b62b276
paper_authors.name: Danielle Rifinski Fainman
paper_authors._id: 652c9b55a2d97e682b62b277
paper_authors.name: Dongmei Zhang
paper_authors._id: 652c9b55a2d97e682b62b278
paper_authors.name: Surajit Chaudhuri
paper_title: Table-GPT: Table-tuned GPT for Diverse Table Tasks
paper_summary: Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable
  abilities to follow diverse human instructions and perform a wide range of
  tasks. However, when probing language models using a range of basic
  table-understanding tasks, we observe that today's language models are still
  sub-optimal in many table-related tasks, likely because they are pre-trained
  predominantly on one-dimensional natural-language texts, whereas
  relational tables are two-dimensional objects.
    In this work, we propose a new "table-tuning" paradigm, where we
  continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using
  diverse table-tasks synthesized from real tables as training data, with the
  goal of enhancing language models' ability to understand tables and perform
  table tasks. We show that our resulting Table-GPT models demonstrate (1) better
  table-understanding capabilities, by consistently outperforming the
  vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout
  unseen tasks, and (2) strong generalizability, in its ability to respond
  to diverse human instructions to perform new table-tasks, in a manner similar
  to GPT-3.5 and ChatGPT.
paper_ai_keywords: table-tuning table-understanding generalizability Table-GPT models