title: RICO: Improving Accuracy and Completeness in Image Recaptioning via
    Visual Reconstruction
summary: Image recaptioning is widely used to generate training datasets with enhanced
  quality for various multimodal tasks. Existing recaptioning methods typically
  rely on powerful multimodal large language models (MLLMs) to enhance textual
  descriptions, but often suffer from inaccuracies due to hallucinations and
  incompleteness caused by missing fine-grained details. To address these
  limitations, we propose RICO, a novel framework that refines captions through
  visual reconstruction. Specifically, we leverage a text-to-image model to
  reconstruct a caption into a reference image, and prompt an MLLM to identify
  discrepancies between the original and reconstructed images to refine the
  caption. This process is performed iteratively, further progressively promoting
  the generation of more faithful and comprehensive descriptions. To mitigate the
  additional computational cost induced by the iterative process, we introduce
  RICO-Flash, which learns to generate captions like RICO using DPO. Extensive
  experiments demonstrate that our approach significantly improves caption
  accuracy and completeness, outperforms most baselines by approximately 10% on
  both CapsBench and CompreCap. Code released at
  https://github.com/wangyuchi369/RICO.
paper_authors._id: 6837d79d4d9866c160d8f43b
paper_authors.name: Yuchi Wang
paper_authors._id: 6837d79d4d9866c160d8f43c
paper_authors.name: Yishuo Cai
paper_authors._id: 6837d79d4d9866c160d8f43d
paper_authors.name: Shuhuai Ren
paper_authors._id: 6837d79d4d9866c160d8f43e
paper_authors.name: Sihan Yang
paper_authors._id: 6837d79d4d9866c160d8f43f
paper_authors.name: Linli Yao
paper_authors._id: 6837d79d4d9866c160d8f440
paper_authors.name: Yuanxin Liu
paper_authors._id: 6837d79d4d9866c160d8f441
paper_authors.name: Yuanxing Zhang
paper_authors._id: 6837d79d4d9866c160d8f442
paper_authors.name: Pengfei Wan
paper_authors._id: 6837d79d4d9866c160d8f443
paper_authors.name: Xu Sun
paper_title: RICO: Improving Accuracy and Completeness in Image Recaptioning via
    Visual Reconstruction
paper_summary: Image recaptioning is widely used to generate training datasets with enhanced
  quality for various multimodal tasks. Existing recaptioning methods typically
  rely on powerful multimodal large language models (MLLMs) to enhance textual
  descriptions, but often suffer from inaccuracies due to hallucinations and
  incompleteness caused by missing fine-grained details. To address these
  limitations, we propose RICO, a novel framework that refines captions through
  visual reconstruction. Specifically, we leverage a text-to-image model to
  reconstruct a caption into a reference image, and prompt an MLLM to identify
  discrepancies between the original and reconstructed images to refine the
  caption. This process is performed iteratively, further progressively promoting
  the generation of more faithful and comprehensive descriptions. To mitigate the
  additional computational cost induced by the iterative process, we introduce
  RICO-Flash, which learns to generate captions like RICO using DPO. Extensive
  experiments demonstrate that our approach significantly improves caption
  accuracy and completeness, outperforms most baselines by approximately 10% on
  both CapsBench and CompreCap. Code released at
  https://github.com/wangyuchi369/RICO.
paper_ai_summary: A novel iterative framework, RICO, improves image caption accuracy by using visual reconstruction and a text-to-image model to refine discrepancies, while RICO-Flash enhances efficiency using DPO.
paper_ai_keywords: multimodal large language models text-to-image model DPO CapsBench CompreCap