arXiVeri: Automatic table verification with GPT
Without accurate transcription of numerical data in scientific documents, a
scientist cannot draw accurate conclusions. Unfortunately, the process of
copying numerical data from one paper to another is prone to human error. In
this paper, we propose to meet this challenge through the novel task of
automatic table verification (AutoTV), in which the objective is to verify the
accuracy of numerical data in tables by cross-referencing cited sources. To
support this task, we propose a new benchmark, arXiVeri, which comprises
tabular data drawn from open-access academic papers on arXiv. We introduce
metrics to evaluate the performance of a table verifier in two key areas: (i)
table matching, which aims to identify the source table in a cited document
that corresponds to a target table, and (ii) cell matching, which aims to
locate shared cells between a target and source table and identify their row
and column indices accurately. By leveraging the flexible capabilities of
modern large language models (LLMs), we propose simple baselines for table
verification. Our findings highlight the complexity of this task, even for
state-of-the-art LLMs like OpenAI's GPT-4. The code and benchmark will be made
publicly available.
64891cbe563d6345d8ea4b66
Gyungin Shin
admin_assigned
2023-06-14T11:47:17.339Z
6238c507254d95e12bd1952b
/avatars/8cea1c761c7b2e996277727143bfbea2.svg
Gyungin Shin
user
noelshin
64891cbe563d6345d8ea4b67
Weidi Xie
admin_assigned
2023-06-14T11:47:52.962Z
63f1841f2f7c0152e8748ec4
/avatars/5398658c7d0cf556531a625a4ca5d18a.svg
Xie
user
Weidi
64891cbe563d6345d8ea4b68
Samuel Albanie
admin_assigned
2023-06-14T11:51:18.735Z
62f3efefd6ba2ee26651f44a
https://cdn-avatars.huggingface.co/v1/production/uploads/1660153837083-noauth.png
Samuel Albanie
user
albanie
arXiVeri: Automatic table verification with GPT
Without accurate transcription of numerical data in scientific documents, a
scientist cannot draw accurate conclusions. Unfortunately, the process of
copying numerical data from one paper to another is prone to human error. In
this paper, we propose to meet this challenge through the novel task of
automatic table verification (AutoTV), in which the objective is to verify the
accuracy of numerical data in tables by cross-referencing cited sources. To
support this task, we propose a new benchmark, arXiVeri, which comprises
tabular data drawn from open-access academic papers on arXiv. We introduce
metrics to evaluate the performance of a table verifier in two key areas: (i)
table matching, which aims to identify the source table in a cited document
that corresponds to a target table, and (ii) cell matching, which aims to
locate shared cells between a target and source table and identify their row
and column indices accurately. By leveraging the flexible capabilities of
modern large language models (LLMs), we propose simple baselines for table
verification. Our findings highlight the complexity of this task, even for
state-of-the-art LLMs like OpenAI's GPT-4. The code and benchmark will be made
publicly available.
The paper introduces a task called automatic table verification (AutoTV) to ensure the accuracy of numerical data in tables through source cross-referencing, proposing a benchmark called arXiVeri and evaluating table and cell matching using LLMs.
automatic table verification AutoTV tabular data table matching cell matching metrics large language models LLMs GPT-4