ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning
<|reference_start|>ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning: Recently, many versatile Multi-modal Large Language Models (MLLMs) have emerged continuously. However, their capacity to query information depicted in visual charts and engage in reasoning based on the queried contents remains under-explored. In this paper, to comprehensively and rigorously benchmark the ability of the off-the-shelf MLLMs in the chart domain, we construct ChartX, a multi-modal evaluation set covering 18 chart types, 7 chart tasks, 22 disciplinary topics, and high-quality chart data. Besides, we develop ChartVLM to offer a new perspective on handling multi-modal tasks that strongly depend on interpretable patterns, such as reasoning tasks in the field of charts or geometric images. We evaluate the chart-related ability of mainstream MLLMs and our ChartVLM on the proposed ChartX evaluation set. Extensive experiments demonstrate that ChartVLM surpasses both versatile and chart-related large models, achieving results comparable to GPT-4V. We believe that our study can pave the way for further exploration in creating a more comprehensive chart evaluation set and developing more interpretable multi-modal models. Both ChartX and ChartVLM are available at: https://github.com/UniModal4Reasoning/ChartVLM<|reference_end|>
@article{xia2024chartx,
    title={ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for
  Complicated Chart Reasoning},
    author={Renqiu Xia, Bo Zhang, Hancheng Ye, Xiangchao Yan, Qi Liu, Hongbin
  Zhou, Zijun Chen, Min Dou, Botian Shi, Junchi Yan, Yu Qiao},
    journal={arXiv preprint arXiv:2402.12185},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.12185},
    primaryClass={cs.CV}
}