TableBench: A Comprehensive and Complex Benchmark for Table Question
  Answering
Recent advancements in Large Language Models (LLMs) have markedly enhanced
the interpretation and processing of tabular data, introducing previously
unimaginable capabilities. Despite these achievements, LLMs still encounter
significant challenges when applied in industrial scenarios, particularly due
to the increased complexity of reasoning required with real-world tabular data,
underscoring a notable disparity between academic benchmarks and practical
applications. To address this discrepancy, we conduct a detailed investigation
into the application of tabular data in industrial scenarios and propose a
comprehensive and complex benchmark TableBench, including 18 fields within four
major categories of table question answering (TableQA) capabilities.
Furthermore, we introduce TableLLM, trained on our meticulously constructed
training set TableInstruct, achieving comparable performance with GPT-3.5.
Massive experiments conducted on TableBench indicate that both open-source and
proprietary LLMs still have significant room for improvement to meet real-world
demands, where the most advanced model, GPT-4, achieves only a modest score
compared to humans.
66c5940c83672a810a1d9861
Xianjie Wu
66c5940c83672a810a1d9862
Jian Yang
claimed_verified
2024-10-14T19:04:48.724Z
64ccb9bfead94891d12aef42
/avatars/c54809d43d93d3f0766bd2555cecc4e3.svg
Yang Jian
user
CSJianYang
66c5940c83672a810a1d9863
Linzheng Chai
claimed_verified
2024-08-22T19:50:49.335Z
64ba096e760936217a3ad2e2
https://cdn-avatars.huggingface.co/v1/production/uploads/64ba096e760936217a3ad2e2/aNQK83Jg5PsBkY0UDg-RA.jpeg
Linzheng Chai
user
Challenging666
66c5940c83672a810a1d9864
Ge Zhang
claimed_verified
2024-08-22T19:50:52.645Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
user
zhangysk
66c5940c83672a810a1d9865
Jiaheng Liu
admin_assigned
2024-08-30T16:31:42.952Z
65377c30e48353201e6fdda0
/avatars/a8f803b6f2e598eaee9c52c0d2ddfc16.svg
Jiaheng Liu
user
CheeryLJH
66c5940c83672a810a1d9866
Xinrun Du
claimed_verified
2024-11-03T14:24:51.742Z
654907a4a1faff97850c4eff
/avatars/458c90151614bc7f116943b6e67d6b8a.svg
du
user
dododododo
66c5940c83672a810a1d9867
Di Liang
66c5940c83672a810a1d9868
Daixin Shu
admin_assigned
2024-08-30T16:31:22.314Z
66a752b21833ce4daf0c1d7c
/avatars/b878c17877e4c75cf9595b8fc73e9aec.svg
Daixin Shu
user
sdxvv
66c5940c83672a810a1d9869
Xianfu Cheng
admin_assigned
2024-08-30T16:31:15.677Z
66c596fccf0439733e38913f
/avatars/f4805286596305996613689868c57fa2.svg
Xianfu Cheng
user
BuaaCXF
66c5940c83672a810a1d986a
Tianzhen Sun
admin_assigned
2024-08-30T16:31:09.606Z
66c5bde784526581561b70d7
/avatars/943ed21bdc28fe6e41e88ac9db265ebd.svg
tianzhen sun
user
tianzhen2024
66c5940c83672a810a1d986b
Guanglin Niu
admin_assigned
2024-08-30T16:31:02.185Z
66c5f5a8b67d2b050b854888
/avatars/6208398d750b692852c40f2c3087b427.svg
Guanglin Niu
user
ngl567
66c5940c83672a810a1d986c
Tongliang Li
admin_assigned
2024-08-30T16:30:46.842Z
663860b8d37124438e73fc13
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VCrseZtFRCfcm7iIZP1SP.jpeg
Tongliang Li
user
tonyliangli
66c5940c83672a810a1d986d
Zhoujun Li
TableBench: A Comprehensive and Complex Benchmark for Table Question
  Answering
Recent advancements in Large Language Models (LLMs) have markedly enhanced
the interpretation and processing of tabular data, introducing previously
unimaginable capabilities. Despite these achievements, LLMs still encounter
significant challenges when applied in industrial scenarios, particularly due
to the increased complexity of reasoning required with real-world tabular data,
underscoring a notable disparity between academic benchmarks and practical
applications. To address this discrepancy, we conduct a detailed investigation
into the application of tabular data in industrial scenarios and propose a
comprehensive and complex benchmark TableBench, including 18 fields within four
major categories of table question answering (TableQA) capabilities.
Furthermore, we introduce TableLLM, trained on our meticulously constructed
training set TableInstruct, achieving comparable performance with GPT-3.5.
Massive experiments conducted on TableBench indicate that both open-source and
proprietary LLMs still have significant room for improvement to meet real-world
demands, where the most advanced model, GPT-4, achieves only a modest score
compared to humans.
TableLLM, trained on TableInstruct, benchmarks performance in tabular data reasoning and highlights the need for improvement in real-world applications.
Large Language Models tabular data TableBench table question answering TableQA TableLLM TableInstruct GPT-3.5 GPT-4