title: To Trust Or Not To Trust Your Vision-Language Model's Prediction
summary: Vision-Language Models (VLMs) have demonstrated strong capabilities in
  aligning visual and textual modalities, enabling a wide range of applications
  in multimodal understanding and generation. While they excel in zero-shot and
  transfer learning scenarios, VLMs remain susceptible to misclassification,
  often yielding confident yet incorrect predictions. This limitation poses a
  significant risk in safety-critical domains, where erroneous predictions can
  lead to severe consequences. In this work, we introduce TrustVLM, a
  training-free framework designed to address the critical challenge of
  estimating when VLM's predictions can be trusted. Motivated by the observed
  modality gap in VLMs and the insight that certain concepts are more distinctly
  represented in the image embedding space, we propose a novel confidence-scoring
  function that leverages this space to improve misclassification detection. We
  rigorously evaluate our approach across 17 diverse datasets, employing 4
  architectures and 2 VLMs, and demonstrate state-of-the-art performance, with
  improvements of up to 51.87% in AURC, 9.14% in AUROC, and 32.42% in FPR95
  compared to existing baselines. By improving the reliability of the model
  without requiring retraining, TrustVLM paves the way for safer deployment of
  VLMs in real-world applications. The code will be available at
  https://github.com/EPFL-IMOS/TrustVLM.
paper_authors._id: 683910d50df60182c0dd5b62
paper_authors.name: Hao Dong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:53:56.719Z
paper_authors._id: 6649fb62a460da1da20f66d0
paper_authors.avatarUrl: /avatars/6afa26922ba8abe8c9603e6f7222531b.svg
paper_authors.fullname: Hao Dong
paper_authors.type: user
paper_authors.user: hdong51
paper_authors._id: 683910d50df60182c0dd5b63
paper_authors.name: Moru Liu
paper_authors._id: 683910d50df60182c0dd5b64
paper_authors.name: Jian Liang
paper_authors._id: 683910d50df60182c0dd5b65
paper_authors.name: Eleni Chatzi
paper_authors._id: 683910d50df60182c0dd5b66
paper_authors.name: Olga Fink
paper_title: To Trust Or Not To Trust Your Vision-Language Model's Prediction
paper_summary: Vision-Language Models (VLMs) have demonstrated strong capabilities in
  aligning visual and textual modalities, enabling a wide range of applications
  in multimodal understanding and generation. While they excel in zero-shot and
  transfer learning scenarios, VLMs remain susceptible to misclassification,
  often yielding confident yet incorrect predictions. This limitation poses a
  significant risk in safety-critical domains, where erroneous predictions can
  lead to severe consequences. In this work, we introduce TrustVLM, a
  training-free framework designed to address the critical challenge of
  estimating when VLM's predictions can be trusted. Motivated by the observed
  modality gap in VLMs and the insight that certain concepts are more distinctly
  represented in the image embedding space, we propose a novel confidence-scoring
  function that leverages this space to improve misclassification detection. We
  rigorously evaluate our approach across 17 diverse datasets, employing 4
  architectures and 2 VLMs, and demonstrate state-of-the-art performance, with
  improvements of up to 51.87% in AURC, 9.14% in AUROC, and 32.42% in FPR95
  compared to existing baselines. By improving the reliability of the model
  without requiring retraining, TrustVLM paves the way for safer deployment of
  VLMs in real-world applications. The code will be available at
  https://github.com/EPFL-IMOS/TrustVLM.
paper_ai_summary: TrustVLM enhances the reliability of Vision-Language Models by estimating prediction trustworthiness without retraining, improving misclassification detection in multimodal tasks.
paper_ai_keywords: Vision-Language Models VLMs modality gap image embedding space confidence-scoring function AURC AUROC FPR95