title: TableBench: A Comprehensive and Complex Benchmark for Table Question
    Answering
summary: Recent advancements in Large Language Models (LLMs) have markedly enhanced
  the interpretation and processing of tabular data, introducing previously
  unimaginable capabilities. Despite these achievements, LLMs still encounter
  significant challenges when applied in industrial scenarios, particularly due
  to the increased complexity of reasoning required with real-world tabular data,
  underscoring a notable disparity between academic benchmarks and practical
  applications. To address this discrepancy, we conduct a detailed investigation
  into the application of tabular data in industrial scenarios and propose a
  comprehensive and complex benchmark TableBench, including 18 fields within four
  major categories of table question answering (TableQA) capabilities.
  Furthermore, we introduce TableLLM, trained on our meticulously constructed
  training set TableInstruct, achieving comparable performance with GPT-3.5.
  Massive experiments conducted on TableBench indicate that both open-source and
  proprietary LLMs still have significant room for improvement to meet real-world
  demands, where the most advanced model, GPT-4, achieves only a modest score
  compared to humans.
paper_authors._id: 66c5940c83672a810a1d9861
paper_authors.name: Xianjie Wu
paper_authors._id: 66c5940c83672a810a1d9862
paper_authors.name: Jian Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T19:04:48.724Z
paper_authors._id: 64ccb9bfead94891d12aef42
paper_authors.avatarUrl: /avatars/c54809d43d93d3f0766bd2555cecc4e3.svg
paper_authors.fullname: Yang Jian
paper_authors.type: user
paper_authors.user: CSJianYang
paper_authors._id: 66c5940c83672a810a1d9863
paper_authors.name: Linzheng Chai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-22T19:50:49.335Z
paper_authors._id: 64ba096e760936217a3ad2e2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64ba096e760936217a3ad2e2/aNQK83Jg5PsBkY0UDg-RA.jpeg
paper_authors.fullname: Linzheng Chai
paper_authors.type: user
paper_authors.user: Challenging666
paper_authors._id: 66c5940c83672a810a1d9864
paper_authors.name: Ge Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-22T19:50:52.645Z
paper_authors._id: 638efcf4c67af472d316d424
paper_authors.avatarUrl: /avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
paper_authors.fullname: Ge Zhang
paper_authors.type: user
paper_authors.user: zhangysk
paper_authors._id: 66c5940c83672a810a1d9865
paper_authors.name: Jiaheng Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T16:31:42.952Z
paper_authors._id: 65377c30e48353201e6fdda0
paper_authors.avatarUrl: /avatars/a8f803b6f2e598eaee9c52c0d2ddfc16.svg
paper_authors.fullname: Jiaheng Liu
paper_authors.type: user
paper_authors.user: CheeryLJH
paper_authors._id: 66c5940c83672a810a1d9866
paper_authors.name: Xinrun Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-03T14:24:51.742Z
paper_authors._id: 654907a4a1faff97850c4eff
paper_authors.avatarUrl: /avatars/458c90151614bc7f116943b6e67d6b8a.svg
paper_authors.fullname: du
paper_authors.type: user
paper_authors.user: dododododo
paper_authors._id: 66c5940c83672a810a1d9867
paper_authors.name: Di Liang
paper_authors._id: 66c5940c83672a810a1d9868
paper_authors.name: Daixin Shu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T16:31:22.314Z
paper_authors._id: 66a752b21833ce4daf0c1d7c
paper_authors.avatarUrl: /avatars/b878c17877e4c75cf9595b8fc73e9aec.svg
paper_authors.fullname: Daixin Shu
paper_authors.type: user
paper_authors.user: sdxvv
paper_authors._id: 66c5940c83672a810a1d9869
paper_authors.name: Xianfu Cheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T16:31:15.677Z
paper_authors._id: 66c596fccf0439733e38913f
paper_authors.avatarUrl: /avatars/f4805286596305996613689868c57fa2.svg
paper_authors.fullname: Xianfu Cheng
paper_authors.type: user
paper_authors.user: BuaaCXF
paper_authors._id: 66c5940c83672a810a1d986a
paper_authors.name: Tianzhen Sun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T16:31:09.606Z
paper_authors._id: 66c5bde784526581561b70d7
paper_authors.avatarUrl: /avatars/943ed21bdc28fe6e41e88ac9db265ebd.svg
paper_authors.fullname: tianzhen sun
paper_authors.type: user
paper_authors.user: tianzhen2024
paper_authors._id: 66c5940c83672a810a1d986b
paper_authors.name: Guanglin Niu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T16:31:02.185Z
paper_authors._id: 66c5f5a8b67d2b050b854888
paper_authors.avatarUrl: /avatars/6208398d750b692852c40f2c3087b427.svg
paper_authors.fullname: Guanglin Niu
paper_authors.type: user
paper_authors.user: ngl567
paper_authors._id: 66c5940c83672a810a1d986c
paper_authors.name: Tongliang Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-30T16:30:46.842Z
paper_authors._id: 663860b8d37124438e73fc13
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VCrseZtFRCfcm7iIZP1SP.jpeg
paper_authors.fullname: Tongliang Li
paper_authors.type: user
paper_authors.user: tonyliangli
paper_authors._id: 66c5940c83672a810a1d986d
paper_authors.name: Zhoujun Li
paper_title: TableBench: A Comprehensive and Complex Benchmark for Table Question
    Answering
paper_summary: Recent advancements in Large Language Models (LLMs) have markedly enhanced
  the interpretation and processing of tabular data, introducing previously
  unimaginable capabilities. Despite these achievements, LLMs still encounter
  significant challenges when applied in industrial scenarios, particularly due
  to the increased complexity of reasoning required with real-world tabular data,
  underscoring a notable disparity between academic benchmarks and practical
  applications. To address this discrepancy, we conduct a detailed investigation
  into the application of tabular data in industrial scenarios and propose a
  comprehensive and complex benchmark TableBench, including 18 fields within four
  major categories of table question answering (TableQA) capabilities.
  Furthermore, we introduce TableLLM, trained on our meticulously constructed
  training set TableInstruct, achieving comparable performance with GPT-3.5.
  Massive experiments conducted on TableBench indicate that both open-source and
  proprietary LLMs still have significant room for improvement to meet real-world
  demands, where the most advanced model, GPT-4, achieves only a modest score
  compared to humans.
paper_ai_keywords: TableLLM TableQA TableBench TableInstruct GPT-3.5 GPT-4