title: Scaling Inference-Time Search with Vision Value Model for Improved
    Visual Comprehension
summary: Despite significant advancements in vision-language models (VLMs), there
  lacks effective approaches to enhance response quality by scaling
  inference-time computation. This capability is known to be a core step towards
  the self-improving models in recent large language model studies. In this
  paper, we present Vision Value Model (VisVM) that can guide VLM inference-time
  search to generate responses with better visual comprehension. Specifically,
  VisVM not only evaluates the generated sentence quality in the current search
  step, but also anticipates the quality of subsequent sentences that may result
  from the current step, thus providing a long-term value. In this way, VisVM
  steers VLMs away from generating sentences prone to hallucinations or
  insufficient detail, thereby producing higher quality responses. Experimental
  results demonstrate that VisVM-guided search significantly enhances VLMs'
  ability to generate descriptive captions with richer visual details and fewer
  hallucinations, compared with greedy decoding and search methods with other
  visual reward signals. Furthermore, we find that self-training the model with
  the VisVM-guided captions improve VLM's performance across a wide range of
  multimodal benchmarks, indicating the potential for developing self-improving
  VLMs. Our value model and code are available at
  https://github.com/si0wang/VisVM.
paper_authors._id: 675320b0cb11c5f0308bf62b
paper_authors.name: Wang Xiyao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-06T17:14:30.976Z
paper_authors._id: 655fed9fdef5905d38b84af3
paper_authors.avatarUrl: /avatars/2cda4182dfd11a1e94743639e62328ea.svg
paper_authors.fullname: Xiyao Wang
paper_authors.type: user
paper_authors.user: russwang
paper_authors._id: 675320b0cb11c5f0308bf62c
paper_authors.name: Yang Zhengyuan
paper_authors._id: 675320b0cb11c5f0308bf62d
paper_authors.name: Li Linjie
paper_authors._id: 675320b0cb11c5f0308bf62e
paper_authors.name: Lu Hongjin
paper_authors._id: 675320b0cb11c5f0308bf62f
paper_authors.name: Xu Yuancheng
paper_authors._id: 675320b0cb11c5f0308bf630
paper_authors.name: Lin Chung-Ching Lin
paper_authors._id: 675320b0cb11c5f0308bf631
paper_authors.name: Lin Kevin
paper_authors._id: 675320b0cb11c5f0308bf632
paper_authors.name: Huang Furong
paper_authors._id: 675320b0cb11c5f0308bf633
paper_authors.name: Wang Lijuan
paper_title: Scaling Inference-Time Search with Vision Value Model for Improved
    Visual Comprehension
paper_summary: Despite significant advancements in vision-language models (VLMs), there
  lacks effective approaches to enhance response quality by scaling
  inference-time computation. This capability is known to be a core step towards
  the self-improving models in recent large language model studies. In this
  paper, we present Vision Value Model (VisVM) that can guide VLM inference-time
  search to generate responses with better visual comprehension. Specifically,
  VisVM not only evaluates the generated sentence quality in the current search
  step, but also anticipates the quality of subsequent sentences that may result
  from the current step, thus providing a long-term value. In this way, VisVM
  steers VLMs away from generating sentences prone to hallucinations or
  insufficient detail, thereby producing higher quality responses. Experimental
  results demonstrate that VisVM-guided search significantly enhances VLMs'
  ability to generate descriptive captions with richer visual details and fewer
  hallucinations, compared with greedy decoding and search methods with other
  visual reward signals. Furthermore, we find that self-training the model with
  the VisVM-guided captions improve VLM's performance across a wide range of
  multimodal benchmarks, indicating the potential for developing self-improving
  VLMs. Our value model and code are available at
  https://github.com/si0wang/VisVM.
paper_ai_summary: VisVM enhances vision-language model inference by guiding sentence generation to improve visual comprehension and reduce hallucinations.
paper_ai_keywords: Vision Value Model VLMs inference-time search generated sentence quality visual comprehension hallucinations visual reward signals multimodal benchmarks self-improving VLMs