Faithful Chart Summarization with ChaTS-Pi
<|reference_start|>Faithful Chart Summarization with ChaTS-Pi: Chart-to-summary generation can help explore data, communicate insights, and help the visually impaired people. Multi-modal generative models have been used to produce fluent summaries, but they can suffer from factual and perceptual errors. In this work we present CHATS-CRITIC, a reference-free chart summarization metric for scoring faithfulness. CHATS-CRITIC is composed of an image-to-text model to recover the table from a chart, and a tabular entailment model applied to score the summary sentence by sentence. We find that CHATS-CRITIC evaluates the summary quality according to human ratings better than reference-based metrics, either learned or n-gram based, and can be further used to fix candidate summaries by removing not supported sentences. We then introduce CHATS-PI, a chart-to-summary pipeline that leverages CHATS-CRITIC during inference to fix and rank sampled candidates from any chart-summarization model. We evaluate CHATS-PI and CHATS-CRITIC using human raters, establishing state-of-the-art results on two popular chart-to-summary datasets.<|reference_end|>
@article{krichene2024faithful,
    title={Faithful Chart Summarization with ChaTS-Pi},
    author={Syrine Krichene, Francesco Piccinno, Fangyu Liu, Julian Martin
  Eisenschlos},
    journal={arXiv preprint arXiv:2405.19094},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.19094},
    primaryClass={cs.CL cs.AI}
}