## Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation

XIAOYE QU , Huazhong University of Science and Technology, China QIYUAN CHEN , † Zhejiang University, China WEI WEI , ∗ Huazhong University of Science and Technology, China JIASHUO SUN, Xiamen University, China JIANFENG DONG, Zhejiang Gongshang University, China

†

Despite the remarkable ability of large vision-language models (LVLMs) in image comprehension, these models frequently generate plausible yet factually incorrect responses, a phenomenon known as hallucination. Recently, in large language models (LLMs), augmenting LLMs by retrieving information from external knowledge resources has been proven as a promising solution to mitigate hallucinations. However, the retrieval augmentation in LVLM significantly lags behind the widespread applications of LVLM. Moreover, when transferred to augmenting LVLMs, sometimes the hallucination degree of the model is even exacerbated. Motivated by the research gap and counter-intuitive phenomenon, we introduce a novel framework, the Active Retrieval-Augmented large vision-language model (ARA), specifically designed to address hallucinations by incorporating three critical dimensions: (i) dissecting the retrieval targets based on the inherent hierarchical structures of images. (ii) pinpointing the most effective retrieval methods and filtering out the reliable retrieval results. (iii) timing the retrieval process to coincide with episodes of low certainty, while circumventing unnecessary retrieval during periods of high certainty. To assess the capability of our proposed ARAmodelin reducing hallucination, we employ three widely used LVLM models (LLaVA-1.5, Qwen-VL, and mPLUG-Owl2) across four benchmarks. Our empirical observations suggest that by utilizing fitting retrieval mechanisms and timing the retrieval judiciously, we can effectively mitigate the hallucination problem. We hope that this study can provide deeper insights into how to adapt the retrieval augmentation to LVLMs for reducing hallucinations with more effective retrieval and minimal retrieval occurrences.

## ACMReference Format:

Xiaoye Qu , Qiyuan Chen † † , Wei Wei , Jiashuo Sun, and Jianfeng Dong. 2018. Alleviating Hallucination in ∗ Large Vision-Language Models with Active Retrieval Augmentation. J. ACM 37, 4, Article 111 (August 2018), 20 pages. https://doi.org/XXXXXXX.XXXXXXX

† The first two authors contribute equally.

∗ Corresponding author.

Authors' Contact Information: Xiaoye Qu † , xiaoye@hust.edu.cn, Huazhong University of Science and Technology, China; Qiyuan Chen , chenqiyuan1012@gmail.com, Zhejiang University, China; Wei Wei , weiw@hust.edu.cn, Huazhong Uni-† ∗ versity of Science and Technology, China; Jiashuo Sun, gasolsun36@gmail.com, Xiamen University, China; Jianfeng Dong, dongjf24@gmail.com, Zhejiang Gongshang University, China.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

© 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM.

ACM 1557-735X/2018/8-ART111

https://doi.org/XXXXXXX.XXXXXXX

Fig. 1. Given an input image and corresponding query, the vanilla LVLM fails to identify the clock within the image and outputs a hallucinated answer. However, by employing the retrieval augmentation strategy proposed in this paper, the LVLM model can accurately answer the query.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00555v1-figure-294fd249-ac7b-4d04-8965-b34da1e41a49.jpg)

## 1 INTRODUCTION

Recently, Large Vision-Language Models (LVLMs) [2, 4, 8, 22, 26, 47, 56] have gained widespread application across various scenarios due to their significant capacity to generate contextually appropriate text for visual inputs. Benefiting from the advancements in model architecture, training methods, and data diversity, LVLMs exhibit superior performance for various tasks such as visual question answering [11, 32] and image captioning [36, 51]. Despite these advancements, LVLMs still suffer from a significant challenge termed 'hallucination', whereby the models produce semantically plausible but factually inaccurate text, misaligned with the ground-truth content of the associated image [7, 12, 18, 49, 55, 57]. This problem damages the practical employment of LVLMs, particularly in fields that require accurate content generation, such as medical [19, 45] and robotics [27, 39], potentially resulting in severe consequences. Hence, addressing the hallucination problem is paramount for bolstering the credibility of LVLMs across real-world scenarios.

To mitigate hallucinations in LVLMs, a series of attempts have been recently proposed and can be broadly classified into two categories. The first class of approach retrains the LVLMs with constructed hallucination-related datasets by supervised fine-tuning (SFT) [10, 24, 44], or Reinforcement Learning from Human Feedback (RLHF) [42, 50]. Although effective, these methods introduce significantly additional training costs. The other solutions focus on designing more robust decoding strategies [7, 12, 18, 49]. For example, VCD [18] introduces visual contrastive decoding to contrast output distributions derived from original and distorted visual inputs. In this way, the LVLMreduces the over-reliance on statistical bias and unimodal prior. Although these methods are training-free, they still suffer from the limitations of LVLMs' static parametric capacity. Recently, in the realm of large language models (LMs), augmenting LLMs [14, 16, 34] by retrieving information from external knowledge resources has shown promise in reducing language hallucinations. Furthermore, this retrieval augmentation serves as a flexible way to extend beyond the model's inherent knowledge without the burden of extensive training costs. In this paper, we aim to propose a novel framework to augment LVLMs with external knowledge by introducing an Active Retrieval-Augmented large vision-language model (ARA) for mitigating hallucinations. As shown in Figure 1, given an input image and query, the LVLM tends to produce hallucinated answers. Instead, our model can accurately identify the most relevant pairs and output the correct answer.

Concretely, our proposed ARA is grounded in three critical dimensions for mitigating hallucinations in LVLMs. Firstly, given the inherently hierarchical nature of images, simple full-image retrieval may result in noise and irrelevant outcomes, thus it is imperative to decompose the target object causing hallucination from the input image for more accurate retrieval. Secondly, in contrast to augmenting LLMs, LVLMs offer diverse retrieval modalities due to their multimodal

inputs. In this way, discerning the most effective retrieval technique and securing trustworthy results is crucial for ensuring retrieving performance. Lastly, excessive retrieval activations may lead to undue time expenditure and unnecessary retrieval. By initiating retrieval only during periods of low LVLM certainty and knowledge deficiency, one can circumvent unnecessary or improper information retrieval.

Based on the above analysis, given the input image and query, our proposed ARA first devises a coarse-to-fine retrieving paradigm. The target object is first extracted from the input query and subsequently positioned within a particular region of the input image. To build a robust retrieving system, we simultaneously engage in coarse-grained and fine-grained retrieval process which retrieves the full image as well as the specific regions pertaining to the target object. Recognizing the plurality of retrieval strategies due to the multimodal input, an extensive analysis of different methods is conducted to determine the most optimal retrieving approach. Subsequent to acquiring pertinent text and image pairs from external databases, a reranking strategy is employed to eliminate unreliable outcomes, thereby augmenting LVLMs with the appropriate external information. Particularly, retrieval operations are circumvented when the LVLM exhibits high certainty, to prevent the inclusion of superfluous retrieval that may introduce redundant information. In this context, retrieval is activated using a difficulty metric that relies on the mutual information between multimodal inputs.

To assess the efficacy of ARA, empirical evaluations are performed using three prevalent LVLMs across four benchmarks related to hallucination challenges. The promising results demonstrate that our framework can effectively mitigate hallucinations. To sum up, our contributions are summarized as follows:

- · Our research thoroughly investigates the three critical dimensions of augmenting LVLMs with external knowledge, including accurately dissecting the retrieval targets, pinpointing the most effective retrieval methods, and the sensible triggering of the retrieval process.
- · Weintroduce an Active Retrieval-Augmented large vision-language model (ARA) for mitigating hallucination. Our empirical evidence indicates that, by employing appropriate retrieval mechanisms and carefully timing the retrievals, we can significantly address the issue of hallucination while maintaining a moderate frequency of retrieval.
- · Through comprehensive experiments, we demonstrate the superior performance of our proposed ARA in alleviating hallucinations using three well-known LVLMs across four hallucinationrelated benchmarks. The results in our paper also suggest that with optimal retrieval settings, the potential of augmenting LVLMs can be more effectively harnessed.

## 2 RELATED WORK

## 2.1 Large Visual-Language Models

Recently, large-scale models have attracted significant attention and gained widespread applications [30, 40, 41, 46, 58]. LVLMs significantly enhance the interaction between humans and AI in a more natural and intuitive ways and demonstrate remarkable capabilities in understanding and generating multi-modal content. With the aid of advanced Large Language Models like LLaMA [43] and Qwen [1], a batch of LVLMs such as LLaVA-1.5 [26], Qwen-VL [2], and mPLUGOwl2 [47] have emerged, which can comprehend and generate a wide array of content by utilizing information from distinct modalities like texts and images. The training of LVLMs is divided into two critical phases: feature alignment and instruction fine-tuning. The former aims to model visual and textual inputs coherently, while the latter strives to enhance the ability of LVLMs to comprehend user queries. Despite the advancements, these LVLMs continue to face significant challenges

with hallucination issues. Thus, in this paper, we focus on solving hallucination problems to promote the use of LVLMs in practical scenarios.

## 2.2 Hallucination in LVLMs

In LVLMs, hallucination refers to models that generate seemingly plausible outputs inclusive of objects, attributes, or relations that do not correspond with the images [20, 48, 52]. Approaches to address hallucinations in LVLMs have largely fallen into two camps. The first emphasizes model refinement through supervised fine-tuning or application of reinforcement learning techniques. For instance, LRV [25] seeks to diminish hallucinatory outputs by utilizing broad and varied supervised fine-tuning datasets. For methods based on reinforcement learning, LLaVA-RLHF [42] is the pioneer in applying Reinforcement Learning with Human Feedback (RLHF) to mitigate hallucination in LVLMs. This approach is further refined by RLHF-V [50], which incorporates detailed corrective human feedback. Considering the instability and training difficulty of RLHF, Zhao et al. [53] employ Direct Preference Optimization (DPO) and build a hallucination-aware dataset for alleviating hallucination. Although While these methods have yielded notable advancements, they also introduce significant training overheads and are prone to overfitting the training data. The second category consists of training-free strategies designed to circumvent hallucination without incurring additional training expenses. VCD [18] endeavors to rectify the model's over-reliance on linguistic priors and statistical leanings by comparing the output distributions from both unaltered and visually perturbed inputs. OPERA [12] mitigates overconfidence in model predictions by integrating a penalty term into the model logits during beam search decoding, further enhanced with a strategic rollback feature. Although these methods are effective, they still suffer from the limitations of LVLMs' static parametric capacity. In this paper, different from the above two paradigms, we focus on augmenting LVLMs with external knowledge to mitigate hallucination in LVLMs.

## 2.3 Retrieval-Augmented Generation

In the realm of large language models (LLMs), Retrieval-Augmented Generation (RAG) has been widely used and shown promising results in mitigating hallucinations. RAG in LLMs retrieves relevant information from an external knowledge base before LLMs respond to a query [14, 16, 34], thereby enabling them to collaboratively generate responses by leveraging the retrieved external non-parameterized knowledge alongside their internal parameterized knowledge. However, its application in LVLMs has not been thoroughly explored. Since LVLMs present a multimodal nature, findings pertinent to LLMs cannot be indiscriminately extrapolated to LVLMs. Predominantly, existing research on retrieval augmentation in multimodal tasks has been confined to image captioning [3, 35-37] or image generation [6], with a narrow focus that overlooks the broader implications of RAG in LVLMs, especially concerning its capacity for hallucination reduction. In this paper, our research conducts an extensive evaluation of RAG's applicability in LVLMs to mitigate hallucination by tailoring retrieval strategies and optimizing active retrieval processes. It is worth noting that the active retrieving strategy has also been discussed in [15] in LLM. However, they propose a token-level confidence for triggering the retrieval, which is not applicable in LVLMs and we will investigate it in the ablation study.

## 3 METHOD

In this section, we first describe the generation procedure of the LVLMs, laying the groundwork for comprehending our approach. Subsequently, we will introduce our Active Retrieval-Augmentation large vision-language model (ARA) in detail. As shown in Figure 2, given the input image and query, the LVLM first analyzes them based on mutual information and determines whether to trigger the retrieval process. If the retrieval is necessary, our ARA will follow a coarse-to-fine paradigm

Fig. 2. The overall pipeline of our Active Retrieval-Augmented large vision-language model (ARA). Given the input image and query, our model first actively determines whether the input pair requires retrieval. If retrieval is not necessary, the LVLM will directly answer the input query based on the image. Otherwise, our model will perform a coarse-to-fine retrieval based on the input image. Subsequently, the retrieved image and text pairs are further reranked according to the caption of the input image. Finally, the retrieved coarsegrained and fine-grained pairs are jointly decoded to produce the final answer.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00555v1-figure-57567b86-5f8b-42dd-a4b5-ba41b71b3c42.jpg)

to dissect the retrieval targets and conduct jointly coarse-grained and fine-grained retrieval. Following this, a reranking of the retrieval outcomes is performed for additional refinement. Finally, the LVLM harnesses this externally sourced knowledge to generate the final response.

## 3.1 LVLM Input and Decoding

The input of LVLMs contains both image and text. Typically, the LVLMs use a vision encoder (e.g. CLIP) to extract visual tokens from the input images, and then map them into the input space of LLMs with a linear projection module. Subsequently, the mapped visual tokens are used as part of the LLM input, along with the text input. In this paper, we denote the visual tokens as x /u1D463 = { /u1D465 0 , /u1D465 1 , . . . , /u1D465 /u1D441 -1 } . Here /u1D441 is the length of the visual tokens and it is a fixed number in most cases. Correspondingly, the input text is tokenized and we denote it as x /u1D45D = { /u1D465 0 , /u1D465 1 , . . . , /u1D465 /u1D440 -1 } . The image and text tokens are concatenated as the final input sequence and we denote it as { /u1D465 /u1D456 } /u1D447 -1 /u1D461 = 0 that /u1D447 = /u1D441 + /u1D440 .

During decoding, LVLM gets the probabilities for the next token prediction, formally:

$$p ( x _ { t } | x _ { < t } ) = \text{SoftMax} [ L V L M ( h _ { t } ) ] _ { x _ { t } }, \ \ x _ { t } \in \mathcal { V },$$

where ℎ /u1D461 denotes the output hidden states at time step t for the input image and query, /u1D465 &lt; /u1D461 to simplify the sequence { /u1D465 /u1D456 } /u1D461 -1 /u1D456 = 0 and V means the whole vocabulary set. For greedy decoding, the token with the highest probability will be chosen.

## 3.2 Active Triggering Retrieval

As shown in Figure 2, given the input image and query, our ARA model first determines whether the input pair requires retrieval. In this way, we can circumvent unnecessary retrieval during periods of high certainty. In this section, we propose three active retrieval methods to explore the

difficulty metrics that influence when to retrieve, including Confidence-aware Active Retrieval, Question-aware Active Retrieval, and Image-aware Active Retrieval. The first method relies on the confidence of the model output, and the latter two matrices depend on the mutual information between the inputs. An ideal metric should effectively reduce the frequency of retrievals and can easily apply to different LVLMs.

Confidence-aware Active Retrieval . We feed the image and query into LVLMs to generate the answer. If the LVLM is confident about the answer, we accept it without retrieving additional information. Specifically, we actively trigger retrieval if any token of the answer has a confidence lower than the threshold /u1D703 . In this method, /u1D703 = 0 indicates retrieval will never be used and /u1D703 = 1 means that all inputs will use the retrieval. This strategy is inspired by [15] in LLM.

Query-aware Active Retrieval . To better evaluate the difficulty of queries from a view of language bias, we evaluate the degree of LVLMs answering relying on the input image. To this end, we define a difficulty metric using mutual information based on the difference below:

$$M _ { i j } = \log \frac { P ( a _ { i j } | V _ { i }, Q _ { i } ) } { P ( a _ { i j } ) } - \log \frac { P ( a _ { i j } | Q _ { i } ) } { P ( a _ { i j } ) } = \log \frac { P ( a _ { i j } | V _ { i }, Q _ { i } ) } { P ( a _ { i j } | Q _ { i } ) }. \quad \ \ ( 2 )$$

where /u1D443 ( /u1D44E /u1D456 /u1D457 | /u1D444 /u1D456 ) indicates the output probability with only query input to the LVLMs and /u1D443 ( /u1D44E /u1D456 /u1D457 | /u1D449 /u1D456 , /u1D444 /u1D456 ) means the output probability with both query and image as model input. The higher /u1D440 /u1D456 /u1D457 indicates more visual reasoning is needed to answer. /u1D440 /u1D456 /u1D457 &lt; 0 illustrates that the answer excessively depends on the input query. Thus, we define a threshold /u1D703 and trigger retrieval if /u1D440 /u1D456 /u1D457 &lt; /u1D703 , where insufficient visual information is provided for answering the query. Incorporating retrieved information will enhance the confidence of LVLMs to answer.

Image-aware Active Retrieval . Similar to the above query-aware active retrieval, we use the probability difference between the original and noisy image to evaluate the image information to answer the query. During the implementation, we use the method in [18] to add noise.

$$D _ { i j } = \log \frac { P ( a _ { i j } | V _ { i }, Q _ { i } ) } { P ( a _ { i j } ) } - \log \frac { P ( a _ { i j } | V _ { i } ^ { \prime }, Q _ { i } ) } { P ( a _ { i j } ) } = \log \frac { P ( a _ { i j } | V _ { i }, Q _ { i } ) } { P ( a _ { i j } | V _ { i } ^ { \prime }, Q _ { i } ) }. \quad \ ( 3 )$$

where /u1D443 ( /u1D44E /u1D456 /u1D457 | /u1D449 /u1D456 , /u1D444 /u1D456 ) indicates the output probability with query and input image, /u1D443 ( /u1D44E /u1D456 /u1D457 | /u1D449 ′ /u1D456 , /u1D444 /u1D456 ) means the output probability with query and noisy image. Analogously, we define a boundary /u1D703 and trigger retrieval if /u1D437 /u1D456 /u1D457 &lt; /u1D703 .

## 3.3 Coarse-to-Fine Hierarchical Retrieval

Given the inherently hierarchical nature of images, simple full-image retrieval may result in noise and irrelevant outcomes. As shown in Figure 2, using the full image for retrieval can not obtain the most relevant information for reasoning, namely 'red shirt". Thus it is imperative to decompose the target object causing hallucination from the input image for more accurate retrieval. In the following, we will first describe the coarse-grained retrieval and then introduce the fine-grained retrieve.

Coarse-grained Retrieve. Initially, based on the input image, we use the CLIP to extract the visual embedding. Similarly, we build the embeddings for the images and corresponding texts in the knowledge base. Then, we can retrieve the relevant images from knowledge bases with the similarity between images. Finally, the top-K image and text pairs will be fetched for the following reasoning. Benefiting from the multimodal encoding ability of CLIP, we can also retrieve the top-K pairs by matching the input image with texts in the database. We will compare different retrieving methods in the ablation study.

Fine-grained Retrieve. While full-image retrieval can incorporate valuable information, it may overlook fine-grained details, such as diminutive objects and intricate attribute data. Therefore, it

is crucial to propose a fine-grained retrieval mechanism that concentrates on the distinct objects within an image. To conduct fine-grained retrieval, we initially extract key entities from the input query utilizing the large language model LLaMA2-7B [43]. This extraction is facilitated by incontext learning [9], where demonstrations are presented to the LLM, which subsequently outputs the specific entities from the input query. Following this, Grounding Dino [28] is deployed to identify the objects in the image that correspond to the extracted entities. As illustrated in Figure 2, wecrop the targeted object and then perform retrieval in a manner akin to the previously described coarse-grained retrieval.

## 3.4 Reranking Retrieving Results

Throughout the above retrieving process, the similarity is computed by the CLIP embeddings between images. In this way, the visually similar but semantically different pairs may be recalled. To avoid noisy retrieving results which bring different semantic, after obtaining the retrieved pairs for both coarse-grained and fine-grained retrieval, we subsequently propose a reranking strategy designed to recalibrate the retrieved outcomes. Specifically, we adopt the LVLM to describe the input image and generate a detailed caption. For coarse-grained retrieval, the input full image is characterized, while describing the cropped image for the fine-grained retrieval. In this way, we are able to derive the semantics of the input image. Then, we extract the textual embeddings for both image captions and captions of retrieved images with the CLIP. Finally, we can rerank the retrieved pairs according to the semantic similarity between these captions. In this stage, we also explore other reranking methods, such as k-reciprocal encoding [54] or using an external visual entailment tool [23]. We will compare them in the ablation study.

## 3.5 Joint Coarse-grained and Fine-grained Decoding

After retrieving and ranking the pairs, we can conduct a joint coarse-grained and fine-grained decoding to yield the final responses. According to the different ways of using the retrieved results, we have two schemes including probability-level fusion and instance-level fusion. The former integrates the results of coarse-grained and fine-grained retrieval into a single prompt for inference, while the latter conducts separate inferences on the coarse-grained and fine-grained retrieval results, followed by a probabilistic-level weighting.

Probability-level Fusion. In this paradigm, we use the prompt for both coarse-grained retrieval and fine-grained retrieval: 'Here are the image-caption pairs similar to the test image: 〈 Retrieval Pairs . Based on these pairs and this image: 〉 〈 Input Image . Answer this question: 〉 〈 Input query ". 〉 Here we denote the retrieval-augmented LVLMs as below:

$$p ^ { \text{coarse} } ( x _ { t } | x _ { < t } ) = \text{SoftMax} [ L V L M ( [ R _ { I } \colon h _ { t } ] ) ] _ { x _ { t } }, \ \ x _ { t } \in \mathcal { V },$$

where /u1D445 /u1D43C indicates the hidden states of retrieved image-level pairs. For simplicity, we directly use [:] here. We use /u1D45D /u1D450/u1D45C /u1D44E/u1D45F /u1D460 /u1D452 to represent decoding with coarse-grained retrieval. Similarly,

$$p ^ { f i n e } ( x _ { t } | x _ { < t } ) = \text{SoftMax} [ L V L M ( [ R _ { O } \colon h _ { t } ] ) ] _ { x _ { t } }, \ \ x _ { t } \in \mathcal { V },$$

where /u1D445 /u1D442 indicates the retrieved fine-grained object-level pairs. We represent the decoding with object-level fine retrieval by /u1D45D /u1D453 /u1D456 /u1D45B/u1D452 ( /u1D465 /u1D461 | /u1D465 &lt; /u1D461 ) . For the probability-level fusion, we can combine them with a hyperparameter /u1D6FC in [0,1] to control the importance of two kinds of retrieval.

$$p ^ { c o a r s e \& f i n e } ( x _ { t } | x _ { < t } ) = \alpha p ^ { c o a r s e } ( x _ { t } | x _ { < t } ) + ( 1 - \alpha ) p ^ { f i n e } ( x _ { t } | x _ { < t } ),$$

Instance-level Fusion. For this kind of fusion, we can integrate both coarse-grained retrieval and fine-grained retrieval with prompts: 'Here are the image-caption pairs similar to the test image:

〈 Coarse-grained Retrieval Pairs . Here are the image-caption pairs: 〉 〈 Fine-grained Retrieval Pairs 〉 similar to the 〈 entity 〉 in the input image. Based on these pairs and this input image: 〈 Input Image . 〉 Answer this question: 〈 Input query ." Here we extract the entity from the input query and fill it in 〉 the prompt.

$$p ^ { c o a r s e + f i n e } ( x _ { t } | x _ { < t } ) = \text{SoftMax} [ L V L M ( [ R _ { I } \colon R _ { O } \colon h _ { t } ] ) ] _ { x _ { t } }, \ \ x _ { t } \in \mathcal { V },$$

## 4 EXPERIMENT

This section will elaborate on the evaluation of our proposed Active Retrieval-Augmented large vision-language model (ARA) across different LVLMs.

## 4.1 Evaluation Metrics

POPE [20], the Polling-based Object Probing Evaluation proposed a simplified method for evaluating object hallucinations. In this evaluation framework, the LVLM is tasked with determining the presence of specific objects in a given image, with a balanced distribution of queries for object presence and absence. The evaluation protocol encompasses three distinct sampling settings: random, popular, and adversarial, each employing unique strategies for constructing negative samples. In the random setting, objects not present in the image are chosen randomly. The popular setting selects missing objects from a high-frequency pool, whereas the adversarial setting prioritizes coexisting objects that could be easily mistaken for existing objects in the image. The POPE benchmark aggregates data from three diverse sources: MSCOCO [21], A-OKVQA [38], and GQA [13]. Within each dataset, every sampling setting comprises 500 images, with 6 questions posed for each image, resulting in a total of 27,000 question-answer pairs derived from the development sets of these datasets. The evaluation predominantly assesses metrics such as accuracy, precision, recall, and F1 score.

MME [48] is a comprehensive benchmark designed specifically for evaluating LVLMs on multiple dimensions. It includes ten sub-tasks related to perception and four sub-tasks emphasizing cognition. According to previous studies [7, 18, 49], we utilize the existence and count subsets for object-level hallucination evaluation, and the position and color subsets for attribute-level hallucination evaluation. Performance is quantified by accuracy and accuracy+, which is also the official evaluation method.

MMStar [5], foundational to vision-reliant multi-modal evaluation, consists of 1,500 challenge samples meticulously chosen by human experts. It is tailored to systematically benchmark six key capabilities alongside eighteen granular facets, aiming to rigorously assess the multi-modal proficiency of LVLMs using a selection of samples that have been carefully refined and equilibrated.

MMbench [29] is collected from multiple sources, including public datasets and the Internet, and currently, contains 2974 multiple-choice questions, covering 20 ability dimensions. Similar to MME, we focus on hallucination-related ability. Thus, we select 4 ability subsets including object localization, attribute recognition, spatial relationship, and action recognition as our evaluation benchmarks. These subsets form a comprehensive assessment of hallucinations, addressing issues at the object-level, attribute-level, and relation-level comprehensively.

## 4.2 Implementation Details

In the case of coarse-grained retrieval, we use the COCO dataset [21] as the retrieval database. For fine-grained retrieval, we use the VisualGenome [17] to build the fine-grained retrieval database as it provides a detailed caption for each grounding box in the image. CLIP-Large is used to extract image embeddings for retrieval. To decompose target objects from the input query, we extract key

Table 1. Results on POPE. Regular decoding denotes greedy decoding, whereas RAR refers to our method with augmentation.

| Dataset   | Setting     | Model      | Decoding    | Accuracy ↑   | Precision   | Recall      | F1 Score ↑   |
|-----------|-------------|------------|-------------|--------------|-------------|-------------|--------------|
|           |             |            | Regular     | 86.50        | 97.40       | 75.00       | 84.75        |
|           |             | LLaVA1.5   | RAR         | 89.43        | 95.96       | 82.33       | 88.63        |
|           | Random      |            | Regular     | 82.13        | 98.88       | 65.00       | 78.44        |
|           |             | Qwen-VL    | RAR         | 84.27        | 98.22       | 69.80       | 81.61        |
|           |             |            | Regular     | 85.97        | 95.15       | 75.80       | 84.38        |
|           |             | mPLUG-Owl2 | RAR         | 89.60        | 94.33       | 84.27       | 89.01        |
|           |             |            | Regular     | 85.53        | 95.02       | 75.00       | 83.83        |
|           |             | LLaVA1.5   | RAR         | 87.47        | 92.07       | 82.00       | 86.74        |
|           | Popular     | Qwen-VL    | Regular     | 81.93        | 98.29       | 65.00       | 78.25        |
| MSCOCO    |             |            | RAR         | 84.03        | 97.58       | 69.80       | 81.38        |
|           |             |            | Regular     | 84.63        | 92.06       | 75.80       | 83.14        |
|           |             | mPLUG-Owl2 | RAR         | 86.10        | 87.68       | 84.00       | 85.80        |
|           |             | LLaVA1.5   | Regular     | 83.60        | 90.71       | 74.87       | 82.03        |
|           |             |            | RAR         | 84.53        | 87.37       | 80.73       | 83.92        |
|           | Adversarial |            | Regular     | 80.97        | 95.41       | 65.07       | 77.37        |
|           |             | Qwen-VL    | RAR         | 82.90        | 94.58       | 69.80       | 80.32        |
|           |             | mPLUG-Owl2 | Regular     | 82.37        | 87.32       | 75.73       | 81.11        |
|           |             |            | RAR         | 82.77        | 82.40       | 83.33 83.00 | 82.86        |
|           |             | LLaVA1.5   | Regular     | 88.63        | 93.54       |             | 87.95        |
|           |             |            | RAR Regular | 90.27 83.60  | 90.81 95.90 | 89.60       | 90.20        |
|           | Random      | Qwen-VL    | RAR         | 86.53        | 95.74       | 70.20 76.47 | 81.06 85.03  |
|           |             |            | Regular     | 86.47        | 91.07       | 80.87       | 85.66        |
|           |             | mPLUG-Owl2 | RAR         | 88.60        | 89.33       | 87.67       | 88.49        |
|           |             |            | Regular     | 85.53        | 87.43       | 83.00       | 85.16        |
|           |             | LLaVA1.5   | RAR         | 85.63        | 85.85       | 85.33       | 85.59        |
|           |             |            | Regular     | 83.47        | 95.55       | 70.20       | 80.94        |
| A-OKVQA   | Popular     | Qwen-VL    |             |              | 94.40       | 76.47       | 84.49        |
|           |             |            | RAR         | 85.97        |             |             |              |
|           |             | mPLUG-Owl2 | Regular RAR | 82.43 82.77  | 83.48 80.17 | 80.87       | 82.15 83.48  |
|           |             |            | Regular     | 79.13        | 77.04       | 87.07 83.00 | 79.91        |
|           |             | LLaVA1.5   | RAR         | 79.33        | 76.96       | 83.73       | 80.20        |
|           |             |            | Regular     | 78.97        | 85.13       | 70.20       | 76.95        |
|           | Adversarial | Qwen-VL    | RAR         | 80.50        | 83.18       | 76.47       | 79.68        |
|           |             |            | Regular     | 74.70        | 73.38       | 80.87       | 76.94        |
|           |             | mPLUG-Owl2 | RAR         | 75.63        | 71.54       | 85.13       | 77.75        |
|           |             |            | Regular     | 88.87        | 93.64       | 83.40       | 88.22        |
|           |             | LLaVA1.5   | RAR         | 90.10        | 91.57       | 88.33       | 89.92        |
|           |             |            | Regular     | 82.63        | 95.45       | 68.53       | 79.78        |
|           | Random      | Qwen-VL    | RAR         | 85.60        | 96.27       | 74.07       | 83.72        |
|           |             |            | Regular     | 85.17        | 91.05       | 78.00       | 84.02        |
|           |             | mPLUG-Owl2 | RAR         | 86.90        | 89.51       | 83.60       | 86.45        |
|           |             | LLaVA1.5   | Regular     | 84.57        | 85.39       | 83.40       | 84.38        |
|           |             |            | RAR         | 84.70        | 84.27       | 85.33       | 84.80        |
| GQA       | Popular     | Qwen-VL    | Regular     | 80.40 83.43  | 89.86 91.41 | 68.53 73.80 | 77.76        |
|           |             |            | RAR Regular | 78.67        | 79.05       | 78.0        | 81.67 78.52  |
|           |             | mPLUG-Owl2 | RAR         | 79.70        | 78.07       | 82.60       |              |
|           |             |            |             |              |             |             | 80.27        |
|           |             | LLaVA1.5   | Regular     | 81.63        | 80.55       | 83.40       | 81.95 82.24  |
|           |             |            | RAR         | 81.83        | 80.43       | 84.13       | 76.04        |
|           | Adversarial | Qwen-VL    | Regular     | 78.40        | 85.38       | 68.53       |              |
|           |             |            | RAR         | 80.93        | 85.86       | 74.07       | 79.53        |
|           |             |            | Regular     | 76.40        | 75.58       | 78.00       | 76.77        |
|           |             | mPLUG-Owl2 | RAR         | 76.53        | 75.38       | 78.80       | 77.05        |

Table 2. Results on the hallucination subset of MME.

| Model      | Decoding   | Object-level   | Object-level   | Attribute-level   | Attribute-level   | Total Scores ↑   |
|------------|------------|----------------|----------------|-------------------|-------------------|------------------|
| Model      | Decoding   | Existence ↑    | Count ↑        | Position ↑        | Color ↑           | Total Scores ↑   |
| LLaVA-1.5  | Regular    | 195.00         | 126.67         | 106.67            | 140.00            | 605.00           |
| LLaVA-1.5  | RAR        | 195.00         | 145.00         | 133.33            | 175.00            | 648.33           |
| Qwen-VL    | Regular    | 165.00         | 145.00         | 103.33            | 185.00            | 598.33           |
| Qwen-VL    | RAR        | 170.00         | 140.00         | 118.33            | 195.00            | 623.33           |
| mPLUG-Owl2 | Regular    | 170.00         | 145.00         | 76.67             | 163.33            | 555.00           |
| mPLUG-Owl2 | RAR        | 175.00         | 148.33         | 78.33             | 163.33            | 565.00           |

entities from the input query utilizing the large language model LLaMA2-7B [43]. Grounding Dino [28] is deployed to identify the objects in the image that correspond to the extracted entities. If the objects can not be located in the image, we only use coarse-grained retrieval for the following reasoning process. In this paper, we implement our ARA model on three LVLMs, including LLaVA 1.5, Qwen-VL, and mPLUG-Owl2 to evaluate the effectiveness of our method. For LLaVA 1.5 and mPLUG-Owl2, we use the retrieved texts to augment, while we apply the retrieved image and text pairs to boost Qwen-VL. For these three LVLMs, we implement augmentation using 3, 4, and 5 retrieval components respectively. For active retrieval, we use the query-aware active retrieval. For the hyperparameter /u1D6FC presented in Equation 6, we apply a value of 0.8 for the POPE, MMstar, and the object-level subsets within the MME benchmark, as well as for OL, SR, and ACR in the MMbench. A value of 0.4 is utilized for the remaining evaluation subsets. For active triggering retrieval, we use query-aware active retrieval for its simplicity and effectiveness. The threshold of active triggering is determined by grid search.

In our experiments, to have a strict comparison of our RAR and vanilla LVLMs, we consistently utilize greedy decoding instead of other probability sampling methods. With this strategy, the token with the highest probability from the post-softmax distribution is directly selected as the next token.

## 4.3 Experiment Results

Results on POPE. Experimental results on POPE under the random, popular, and adversarial settings are summarized in Table 1. This benchmark mainly focuses on the object-level hallucination. Specifically, the performances of our ARA consistently surpass the baseline results on all of the LVLMs. This suggests its pivotal role in augmenting LVLMs with retrieval, thereby reducing instances of object-level hallucination. In a more detailed model-specific analysis, RAR demonstrates varied effects across different LVLMs. For all three LVLMs, the F1 score elevation is predominantly driven by a recall boost (e.g., up to 10 points for mPLUG-Owl2), showcasing retrieving to external knowledge base can effectively help detect object presences.

Results on MME Hallucination Subset. The MME subset evaluations extend beyond POPE's scope, encompassing both object-level and attribute-level hallucination. Results in Table 2 show that our ARA leads to a uniform enhancement in addressing object-level hallucination for all models, except the count score of Qwen-VL. Meanwhile, our ARA demonstrates significant improvements in attribute-level Color scores, contributing to substantial overall performance gains. These improvements are attributable to our designed coarse-to-fine retrieval paradigms, which effectively focus on the target object.

Table 3. Results on the MMStar benchmark. CP (coarse perception), FP (fine-grained perception), IR (instance reasoning), LR (logical reasoning), MA (mathematics), ST (science &amp; technology).

| Model      | Decoding   |    CP |   FGP |    IR |    LR |    MA |    ST |   Ave. |
|------------|------------|-------|-------|-------|-------|-------|-------|--------|
| LLaVA-1.5  | Regular    | 0.556 | 0.264 | 0.388 | 0.276 | 0.236 | 0.208 |  0.321 |
| LLaVA-1.5  | RAR        | 0.66  | 0.416 | 0.464 | 0.34  | 0.32  | 0.252 |  0.409 |
| Qwen-VL    | Regular    | 0.54  | 0.348 | 0.46  | 0.288 | 0.292 | 0.204 |  0.355 |
| Qwen-VL    | RAR        | 0.584 | 0.428 | 0.524 | 0.324 | 0.26  | 0.28  |  0.4   |
| mPLUG-Owl2 | Regular    | 0.544 | 0.268 | 0.42  | 0.328 | 0.236 | 0.172 |  0.328 |
| mPLUG-Owl2 | RAR        | 0.636 | 0.396 | 0.48  | 0.392 | 0.312 | 0.252 |  0.411 |

Table 4. Results on the hallucination subsets of MMbench benchmark. OL (object localization), ATR (attribute recognition), SR (spatial relationship), and ACR (action recognition).

| Model      | Decoding   |     OL |    ATR |     SR |    ACR |   Ave. |
|------------|------------|--------|--------|--------|--------|--------|
| LLaVA1.5   | Regular    | 0.6032 | 0.875  | 0.3785 | 0.9581 | 0.7147 |
| LLaVA1.5   | RAR        | 0.654  | 0.8902 | 0.435  | 0.9628 | 0.7293 |
| Qwen-VL    | Regular    | 0.5397 | 0.8144 | 0.3842 | 0.9116 | 0.6684 |
| Qwen-VL    | RAR        | 0.6413 | 0.8523 | 0.4407 | 0.9349 | 0.7271 |
| mPLUG-Owl2 | Regular    | 0.5206 | 0.7955 | 0.3672 | 0.9209 | 0.656  |
| mPLUG-Owl2 | RAR        | 0.5873 | 0.8182 | 0.4237 | 0.9581 | 0.7024 |

Results on MMStar. In a more difficult benchmark MMStar which rigorously assesses the multimodal proficiency, as depicted in Table 3, our method ARA secures a notable enhancement across all six subsets. For instance, LLaVA-1.5 realizes an average improvement of 8.8 points. Unlike POPE, which may include statistical bias, answering questions in MMStar necessitates reliance on image information. Hence, our retrieval augmentation method introducing external image and text knowledge significantly bolsters model performance.

Results on MMBench Hallucination Subset. This benchmark extends our hallucination evaluation to incorporate the relation level in addition to the above object and attribute levels, providing a more holistic assessment for reducing hallucinations. As evidenced in Table 4, benefiting from coarse-grained retrieval, our ARA manifests discernible enhancements in the object and relation-level subsets, such as OL, SR, and ACR. For instance, our ARA model surpasses the vanilla LLaVA-1.5 with 5.08 percent points on object localization (OL). Concurrently, the integration of fine-grained retrieval into our model facilitates superior performance in attribute-level subsets, namely attribute recognition (ATR).

## 5 ABLATION STUDY

In this section, we undertake a series of ablation studies with the aim of identifying the critical elements integral to constructing a resilient and efficient retrieval-augmented large vision-language model.

Table 5. Ablation studies on different retrieving methods. The results are obtained on POPE MSCOCO Popular by LLaVA 1.5. For this study, we only conduct coarse-grained retrieval.

| Methods     |   Accuracy |   Precision |   Recall |   F1 Score |
|-------------|------------|-------------|----------|------------|
| T → T       |      86.67 |       93.72 |    78.6  |      85.5  |
| T → I       |      51.6  |       50.81 |    99.87 |      67.36 |
| I → I       |      86.93 |       91.41 |    81.53 |      86.19 |
| I → T       |      86.67 |       90.92 |    81.47 |      85.94 |
| DINOv2-Base |      86.07 |       90.37 |    80.73 |      85.28 |
| CLIP-Base   |      86.33 |       90.79 |    80.87 |      85.54 |
| CLIP-Large  |      86.93 |       91.41 |    81.53 |      86.19 |

## 5.1 Ablation of Coarse-to-Fine Retrieval

In this section, in order to explore the effects of different factors on coarse-to-fine retrieval in a simple and effective way, we did not use the rerank method or active trigger strategy.

Which kind of retrieval is better? Due to the multimodal characteristics of the input, there are various retrieval methods at our disposal. As shown in Table 5, we attempt to use the input query and image to search for images and descriptions in the database. For this study, we only conduct coarse-grained retrieval and we use the COCO dataset as the database which contains a caption for each image. As a result, there are four retrieval methods available to us. As reflected in the results within this table, we adopt the input image as our primary retrieval modality for images in the database throughout our series of experiments. Surprisingly, using the input query to search the database for image leads to substantially poorer outcomes, achieving only 51.6% accuracy. This lower performance may stem from the fact that, despite the presence of relevant keywords, the images retrieved this way tend to be surrounded by a significant degree of noise.

Additionally, we investigate the influence of different embedding models on the efficacy of image-to-image retrieval. In our comparison, we examine three models: CLIP-Base/Large [33] and DINOV2-Base [31]. Among them, we notice that CLIP-base performs better than DINO (86.33 vs 86.07) and this may be related to their different training methods. Moreover, when comparing CLIP models of various sizes, we observe that the larger-scale CLIP models contribute to more pronounced improvements in performance. Therefore, in the context of this study, CLIP-Large is selected as the image embedding model for retrieval purposes.

How to augment the LVLM? After retrieving, we can obtain retrieved images and paired captions. In this section, we explore how to augment the LVLM with this external information. For a fair comparison, in this study, we fix the number of retrieved pairs to 3 for both coarse-grained and fine-grained retrieval. As demonstrated in Table 6, when using coarse-grained retrieval, we observe that the image and caption pair perform worse than the single caption (86.93 vs 87.43). However, the opposite results are observed in fine-grained retrieval and the Fine (I+T) achieves 86.53 compared with 86.20 of Fine (T). These results imply that retrieval augmentation is sensitive to specific configurations. In this experiment, we do not carefully compare with Coarse (I) or Fine (I) as we have found that the LVLMs we use in our experiments are still quite weak in multi-image reasoning.

In Section 3.5, we propose a joint coarse-grained and fine-grained decoding, designed to optimally utilize the two retrieval mechanisms. In Table 6, we also investigate the effectiveness of probabilistic-level fusion denoted as Coarse &amp; Fine and instance-level fusion labeled as Coarse+Fine.

Table 6. Performance on the LLaVA-1.5. In this paper, we compare different kinds of retrieval schemes and regular decoding. 'I" indicates augmentation with only retrieved texts and 'I+T" means enhancing with both texts and images. The experiments are performed on the popular part of POPE MSCOCO dataset. Coarse and Fine indicate coarse-grained and fine-grained retrieval, separately. Coarse+Fine means the instance-level fusion, while Coarse &amp; Fine corresponds to the probabilistic-level fusion.

Fig. 3. Performance of three LVLMs corresponding to different numbers of retrieval pairs. The performance trends of accuracy and F1 score are the same. The experiments are performed on the popular part of the POPE MSCOCO dataset.

| Method              |   Accuracy |   Precision |   Recall |    F1 |
|---------------------|------------|-------------|----------|-------|
| Regular             |      85.53 |       95.02 |    75    | 83.83 |
| Coarse (T)          |      86.43 |       93.03 |    80.93 | 86.56 |
| Coarse (I+T)        |      86.93 |       91.41 |    81.53 | 86.19 |
| Fine (T)            |      86.2  |       89.46 |    82.07 | 85.61 |
| Fine (I+T)          |      86.53 |       93.08 |    78.93 | 85.43 |
| Coarse+Fine (T)     |      86.27 |       89.59 |    82.07 | 85.66 |
| Coarse+Fine (I+T)   |      86.77 |       91.88 |    80.67 | 85.91 |
| Coarse & Fine (T)   |      87.47 |       92.07 |    82    | 86.74 |
| Coarse & Fine (I+T) |      87.23 |       92.47 |    81.07 | 86.39 |

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00555v1-figure-9f4c86f1-bf3e-4b71-8f89-eff950b234c4.jpg)

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00555v1-figure-ac8ddb43-8f2e-41fd-a725-3032a4321d37.jpg)

As shown from the results, we observe that Coarse+Fine even performs than Coarse retrieval, signifying that simply integrating the coarse-grained and fine-grained retrieved results in a prompt fails to enhance performance, given the distinct granularity of the information retrieved through these two approaches. Instead, the probabilistic-level fusion Coarse &amp; Fine achieves better accuracy than both two kinds of retrieval. Thus, in our paper, we use probabilistic-level fusion for the following experiments. It is worth noting that Coarse &amp; Fine (T) performs better than Coarse &amp; Fine (I+T) for LLaVA-1.5 but Coarse &amp; Fine (I+T) achieves better for Qwen-VL (we will describe in Figure 3 later). This result further validates the necessity of carefully selecting retrieval configurations to obtain the most favorable experimental results.

Howmuchexternalinformationdoweneed? With the above retrieval, we can obtain multiple image-caption pairs from the database. This research delves into the efficacy of varying retrieved pair configurations. Considering that different LVLMs may be sensitive to retrieval configurations, our study incorporates both (T) and (I+T) configurations for a thorough evaluation. The analysis is constrained to examining between one to five retrieved pairs due to the input token limitations inherent in contemporary LVLMs. As depicted in Figure 3, LLaVA (T) reaches peak accuracy with

Fig. 4. The detailed reranking process in our method. With reranking, the retrieved five pairs are sorted in a new order. This example is from the POPE benchmark and the LVLM used here is LLaVA 1.5.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00555v1-figure-73adc05e-8662-40e3-ae1e-342ff6425edd.jpg)

Table 7. Ablation studies on different reranking methods. The results are obtained on POPE MSCOCO Popular by LLaVA-1.5. In this experiment, we incorporate a reranking strategy on top of the coarse-grained retrieval. Thus, the result of the first line (None) aligns with the results in Table 5.

| Methods                  |   Accuracy |   Precision |   Recall |   F1 Score |
|--------------------------|------------|-------------|----------|------------|
| None                     |      86.93 |       91.41 |    81.53 |      86.19 |
| /u1D458 -Reciprocal [54] |      87.03 |       91.36 |    81.8  |      86.32 |
| VSR [23]                 |      86.97 |       91.29 |    81.73 |      86.25 |
| RAR                      |      87.17 |       91.57 |    81.87 |      86.45 |

three retrieved texts, whereas mPLUG (T) begins to saturate when there are four retrieval texts. Conversely, QwenVL (I+T) demonstrates improved outcomes with a greater number of retrieval pairs; however, it still lags considerably behind the performance achieved by the other two LVLM models.

## 5.2 The effectiveness of Reranking Method

To substantiate the efficacy and indispensability of reranking, we begin by providing a case study within the POPE benchmark. As depicted in Figure 4, in instances where an image portrays people in a pool, leveraging CLIP for image retrieval may inadvertently introduce irrelevant content due to its predominant emphasis on visual congruence. For instance, it might retrieve an incongruent image captioned 'fruit and a bowl of bread sit on the table." Therefore, an adept reranking methodology can proficiently filter and organize the retrieved images in a more relevant manner. In this particular scenario, our reranking strategy aligns the input image caption with those of the retrieved images. By assessing these similarity metrics, we are able to reorganize the pool of retrieved images into a more accurately ranked series, thereby enhancing the precision of the retrieval process.

In order to further quantify the effect of reranking and compare different reranking methods, we conduct experiments in Table 7. The result reveals that the omission of re-ranking precipitates a decline in accuracy, dropping from 87.17% to 86.93% This suggests that re-ranking effectively mitigates noise, particularly when employing the top three re-ordered retrieval pairs. For comparative purposes, we executed two alternative methodologies. The k-Reciprocal approach [54], a prevalent offline person re-identification method, re-ranks images using the k-Reciprocal nearest neighbor algorithm. While this method offers a swifter execution by circumventing the captioning process of input images, its efficacy slightly underperforms our strategy. Additionally, we examined the employment of the external tool VSR [23], which determines the entailment between the caption of the input image and the retrieved images. As shown in Table 7, this method gains only a slight

Fig. 5. Different strategies for triggering retrieval on the popular part of POPE MSCOCO. The horizontal axis represents the difficulty threshold, the left vertical axis represents the proportion of queries that trigger retrieval out of all queries, and the right vertical axis shows the values of F1 score and accuracy. From top to bottom, the three lines respectively represent: confidence-aware active retrieval, query-aware active retrieval, and image-aware active retrieval.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00555v1-figure-d47dddcb-e477-43d5-9a2d-9de3cd0a0053.jpg)

Table 8. The performance comparison between our method RAR and VCD in POPE MSCOCO benchmark. VCD is the state-of-the-art method published in CVPR24. The results of VCD-sampling are from their paper. For a fair comparison with our method, we rerun VCD with greedy decoding.

| Setting     | Decoding          |   Accuracy ↑ |   Precision |   Recall |   F1 Score ↑ |
|-------------|-------------------|--------------|-------------|----------|--------------|
| Random      | VCD-greedy [18]   |        85.37 |       93.24 |    76.27 |        83.9  |
| Random      | VCD-sampling [18] |        87.73 |       91.42 |    83.28 |        87.16 |
| Random      | RAR               |        89.43 |       95.96 |    82.33 |        88.63 |
| Popular     | VCD-greedy [18]   |        83.3  |       88.75 |    76.27 |        82.04 |
| Popular     | VCD-sampling [18] |        85.38 |       86.92 |    83.28 |        85.06 |
| Popular     | RAR               |        87.47 |       92.07 |    82    |        86.74 |
| Adversarial | VCD-greedy [18]   |        80    |       82.37 |    76.33 |        79.24 |
| Adversarial | VCD-sampling [18] |        80.88 |       79.45 |    83.29 |        81.33 |
| Adversarial | RAR               |        84.58 |       87.17 |    80.73 |        83.92 |

increase in accuracy (86.93 vs 86.97). This may be due to the visual entailment model not being sufficiently versatile.

Table 9. Text generation quality comparison. Results on the MS-COCO dataset.

| Model      | Decoding   |   B@4 |   METEOR |   ROUGE-L |   CIDEr |   SPICE |   Ave. |
|------------|------------|-------|----------|-----------|---------|---------|--------|
| LLaVA-1.5  | Regular    |  22.3 |     28   |      50.9 |    75.3 |    22.1 |   39.7 |
| LLaVA-1.5  | RAR        |  25.8 |     28.3 |      53.2 |    94.2 |    22.2 |   44.7 |
| Qwen-VL    | Regular    |  25.3 |     25.5 |      50.3 |    96.4 |    19.8 |   43.4 |
| Qwen-VL    | RAR        |  40.8 |     29.8 |      60.5 |   137.2 |    23.4 |   58.3 |
| mPLUG-Owl2 | Regular    |  28.4 |     27.3 |      54.1 |    96.5 |    19   |   45.1 |
| mPLUG-Owl2 | RAR        |  32.1 |     28.5 |      56.3 |   103.7 |    20.8 |   48.3 |

## 5.3 Ablation of Active Triggering Retrieval

As excessive retrieval activations may lead to undue time expenditure and unnecessary retrieval, in this paper, retrieval is activated using a difficulty metric. In this ablation study, we compare three kinds of active retrieval methods, including confidence-aware active retrieval, query-aware active retrieval, and image-aware active retrieval. As shown in Figure 5, we have drawn the proportion of queries that need to be retrieved to achieve peak performance. Firstly, it is quite intuitive, the fluctuations of confidence-aware active retrieval among different models are very significant. For mPLUG-Owl2, we even need to retrieve all the queries, which indicates that confidence-aware active retrieval is not a good indicator of whether to perform a retrieval. For both query-aware active retrieval and image-aware active retrieval, the LVLMs can better determine a trigger threshold. Considering that query-aware active retrieval is more concise and the process of adding noise is omitted, we use query-aware active retrieval in all our experiments. It is worth noting that the very slight decline in the model's performance after continuing to use retrieval beyond the trigger threshold is due to the fact that the model itself has a high level of confidence, but the retrieval has introduced some redundant noise information, which we cannot filter out through reranking.

## 5.4 Comparison with State-of-the-Art Method

In this paper, we augment LVLMs with external knowledge to mitigate hallucinations. To further demonstrate the effectiveness of our method, we compare it with VCD [18] and the results are demonstrated in Table 8. VCD is a state-of-the-art training-free method that explores intra-model knowledge with visual contrastive decoding. From the results, we can notice that our method significantly surpasses VCD in all three settings. For instance, in popular settings, our method gains a 4.06 percent improvement. It is worth noting that VCD performs comparably with the vanilla LLaVA-1.5 when using greedy decoding. This phenomenon is also observed in [7]. Instead, our method significantly surpasses the vanilla LLaVA-1.5 model as presented in Tabe 1. This result further verifies the robust improvement of introducing external information when applying our ARA method.

## 5.5 Text Generation Quality Evaluation

In this section, we evaluated the effectiveness of our method in improving text generation quality on the MS-COCO validation set 1 . As shown in Table 9, all models LLaVA, Qwen-VL, and mPLUG exhibited significant improvements after applying our method, with an average improvement of 7.7% across the three models. Notably, the improvement was most pronounced for Qwen-VL, with an average increase of approximately 14.9%.

1 We use the official COCO evaluation package.

Fig. 6. Examples from POPE benchmark and MME. We present both coarse-grained and fine-grained retrieval for each instance.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00555v1-figure-f750aef1-ded9-46f8-be6f-11cef57d359e.jpg)

| Query                                           |        | Retrieved Results   | Retrieved Results                                          | Retrieved Results                                                    | Retrieved Results                                                             |
|-------------------------------------------------|--------|---------------------|------------------------------------------------------------|----------------------------------------------------------------------|-------------------------------------------------------------------------------|
|                                                 | Coarse |                     | Caption: two monitors with a keyboard and mouse on a desk. | Caption: two imac computers are on a desk with a mouse and keyboard. | Caption: a turned on computer sitting on top of a desk.                       |
| Is there a mouse in the image?                  | Fine   |                     | Caption: a white computer mouse.                           | Caption: white mouse on grey mousepad.                               | Caption: white wireless mouse in front of tape.                               |
|                                                 | Coarse |                     | Caption: a laptop and some computer screens on a desk.     | Caption: office cubicle with a computer and a laptop .               | Caption: a wooden table topped with a laptop computer and a desktop computer. |
| Is there a laptop in the image?                 | Fine   |                     | Caption: a silver laptop that is switched on.              | Caption: laptop is visible on a desk.                                | Caption: open black laptop sitting on floor.                                  |
|                                                 | Coarse |                     | Caption: a picture of the big ben clock tower in london.   | Caption: a large tower with a clock at the top.                      | Caption: an old clock tower is seen next to a grassy area.                    |
| Is there a clock in the image?                  | Fine   |                     | Caption: black numbers on clock face.                      | Caption: clock in tower to tell time to people outside.              | Caption: black and white clock .                                              |
|                                                 | Coarse |                     | Caption: a slender vase filled with four red flowers.      | Caption: a clear vase has red and yellow flowers .                   | Caption: a group of red flowers with long green stems in a glass vase.        |
| Are there any green flowers in the image?       | Fine   |                     | Caption: a vase filled with flowers .                      | Caption: tan curtains behind table and flower vases                  | Caption: 10 roses in a vase.                                                  |
|                                                 | Coarse |                     | Caption: three dogs running together in a field of grass.  | Caption: a dog returning an orange ball to his owner.                | Caption: four dogs are running and bathing in the ocean.                      |
| Are there only three dogs appear in this image? | Fine   |                     | Caption: there are two dogs in the picture.                | Caption: two dogs playfully running.                                 | Caption: two dogs on grass.                                                   |

## 5.6 Qualitative Results

To qualitatively verify the effectiveness of our ARA method on downstream tasks, we presented five examples from the POPE MSCOCO dataset and MME benchmark. As shown in Figure 6, we present three image-caption pairs for both coarse-grained retrieval and fine-grained retrieval. In the above three examples of determining the existence of objects, fine-grained retrieval helps to increase the model's existence information about specific objects. For the last two examples, the coarse-grained retrieval contributed effectively to capturing the overall information of the images. Thus, with both kinds of retrieval, our RAR model can effectively increase the accuracy compared with vanilla LLaVA-1.5.

## 6 CONCLUSION

In this paper, we propose a novel framework to augment LVLMs with external knowledge by introducing an Active Retrieval-Augmented large vision-language model (ARA) for mitigating hallucinations. Specifically, our proposed ARA is grounded in three critical dimensions for mitigating hallucinations in LVLMs, including coarse-to-fine retrieval for dissecting the retrieval targets, pinpointing the most effective retrieval methods, and triggering the retrieval process properly. Our findings indicate that by utilizing fitting retrieval mechanisms and timing the retrieval judiciously, we can effectively mitigate the hallucination problem. Our extensive experiments across four benchmarks and three widely-used LVLM confirm ARA's efficacy in reducing hallucination.

## REFERENCES

- [1] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609 (2023).
- [2] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. (2023).
- [3] Jingwen Chen, Yingwei Pan, Yehao Li, Ting Yao, Hongyang Chao, and Tao Mei. 2023. Retrieval augmented convolutional encoder-decoder networks for video captioning. ACMTransactions on Multimedia Computing, Communications and Applications 19, 1s (2023), 1-24.
- [4] Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXiv:2310.09478 (2023).
- [5] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. 2024. Are We on the Right Way for Evaluating Large Vision-Language Models? arXiv preprint arXiv:2403.20330 (2024).
- [6] Wenhu Chen, Hexiang Hu, Chitwan Saharia, and William W Cohen. 2022. Re-imagen: Retrieval-augmented text-toimage generator. arXiv preprint arXiv:2209.14491 (2022).
- [7] Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. 2024. HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding. arXiv preprint arXiv:2403.00425 (2024).
- [8] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. 2024. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems 36 (2024).
- [9] Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234 (2022).
- [10] Anisha Gunjal, Jihan Yin, and Erhan Bas. 2024. Detecting and preventing hallucinations in large vision language models. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 18135-18143.
- [11] Jiaxian Guo, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Boyang Li, Dacheng Tao, and Steven CH Hoi. 2022. From images to textual prompts: Zero-shot vqa with frozen large language models. arXiv preprint arXiv:2212.10846 (2022).
- [12] Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. 2023. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. arXiv preprint arXiv:2311.17911 (2023).
- [13] Drew A Hudson and Christopher D Manning. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 6700-6709.
- [14] Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Atlas: Few-shot learning with retrieval augmented language models. arXiv preprint arXiv:2208.03299 (2022).
- [15] Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. arXiv preprint arXiv:2305.06983 (2023).
- [16] Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906 (2020).
- [17] Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision 123 (2017), 32-73.

- [18] Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2023. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. arXiv preprint arXiv:2311.16922 (2023).
- [19] Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2024. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems 36 (2024).
- [20] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355 (2023).
- [21] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 . Springer, 740-755.
- [22] Daizong Liu, Mingyu Yang, Xiaoye Qu, Pan Zhou, Wei Hu, and Yu Cheng. 2024. A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends. arXiv preprint arXiv:2407.07403 (2024).
- [23] Fangyu Liu, Guy Edward Toh Emerson, and Nigel Collier. 2023. Visual Spatial Reasoning. Transactions of the Association for Computational Linguistics (2023).
- [24] Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. 2023. Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv:2306.14565 (2023).
- [25] Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. 2023. Mitigating hallucination in large multi-modal models via robust instruction tuning. In The Twelfth International Conference on Learning Representations .
- [26] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024. Visual instruction tuning. Advances in neural information processing systems 36 (2024).
- [27] Kangcheng Liu, Xinhu Zheng, Chaoqun Wang, Hesheng Wang, Ming Liu, and Kai Tang. 2024. Online Robot Navigation and and Manipulation with Distilled Vision-Language Models. arXiv preprint arXiv:2401.17083 (2024).
- [28] Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. 2023. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499 (2023).
- [29] Yuanzhan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. 2023. MMBench: Is Your Multi-modal Model an All-around Player? ArXiv abs/2307.06281 (2023). https://api.semanticscholar.org/CorpusID:259837088
- [30] Zhenyi Lu, Jie Tian, Wei Wei, Xiaoye Qu, Yu Cheng, Dangyang Chen, et al. 2024. Mitigating Boundary Ambiguity and Inherent Bias for Text Classification in the Era of Large Language Models. arXiv preprint arXiv:2406.07001 (2024).
- [31] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. 2023. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023).
- [32] Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal. 2023. Rephrase, augment, reason: Visual grounding of questions for vision-language models. arXiv preprint arXiv:2310.05861 (2023).
- [33] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning . PMLR, 8748-8763.
- [34] Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics 11 (2023), 1316-1331.
- [35] Rita Ramos, Bruno Martins, and Desmond Elliott. 2023. LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting. arXiv preprint arXiv:2305.19821 (2023).
- [36] Rita Ramos, Bruno Martins, Desmond Elliott, and Yova Kementchedjhieva. 2023. Smallcap: lightweight image captioning prompted with retrieval augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2840-2849.
- [37] Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Alessandro Nicolosi, and Rita Cucchiara. 2024. Towards RetrievalAugmented Architectures for Image Captioning. ACM Transactions on Multimedia Computing, Communications and Applications (2024).
- [38] Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. A-okvqa: A benchmark for visual question answering using world knowledge. In European Conference on Computer Vision . Springer, 146-162.
- [39] Dhruv Shah, Błażej Osiński, Sergey Levine, et al. 2023. Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action. In Conference on robot learning . PMLR, 492-504.
- [40] Zhaochen Su, Juntao Li, Jun Zhang, Tong Zhu, Xiaoye Qu, Pan Zhou, Yan Bowen, Yu Cheng, et al. 2024. Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning? arXiv preprint arXiv:2406.09072 (2024).

- [41] Zhaochen Su, Jun Zhang, Tong Zhu, Xiaoye Qu, Juntao Li, Min Zhang, and Yu Cheng. 2024. Timo: Towards Better Temporal Reasoning for Language Models. arXiv preprint arXiv:2406.14192 (2024).
- [42] Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, YuXiong Wang, Yiming Yang, et al. 2023. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525 (2023).
- [43] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023).
- [44] Lei Wang, Jiabang He, Shenshen Li, Ning Liu, and Ee-Peng Lim. 2024. Mitigating fine-grained hallucination by fine-tuning large vision-language models with caption rewrites. In International Conference on Multimedia Modeling . Springer, 32-45.
- [45] Sheng Wang, Zihao Zhao, Xi Ouyang, Qian Wang, and Dinggang Shen. 2023. Chatcad: Interactive computer-aided diagnosis on medical image using large language models. arXiv preprint arXiv:2302.07257 (2023).
- [46] Yue Wang, Tianfan Fu, Yinlong Xu, Zihan Ma, Hongxia Xu, Bang Du, Yingzhou Lu, Honghao Gao, Jian Wu, and Jintai Chen. 2024. TWIN-GPT: Digital Twins for Clinical Trials via Large Language Model. ACMTransactions on Multimedia Computing, Communications and Applications (2024).
- [47] Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2023. mplugowl2: Revolutionizing multi-modal large language model with modality collaboration. arXiv preprint arXiv:2311.04257 (2023).
- [48] Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. A survey on multimodal large language models. arXiv preprint arXiv:2306.13549 (2023).
- [49] Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. 2023. Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.16045 (2023).
- [50] Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. 2023. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. arXiv preprint arXiv:2312.00849 (2023).
- [51] Zequn Zeng, Yan Xie, Hao Zhang, Chiyu Chen, Zhengjue Wang, and Bo Chen. 2024. MeaCap: Memory-Augmented Zero-shot Image Captioning. arXiv preprint arXiv:2403.03715 (2024).
- [52] Bohan Zhai, Shijia Yang, Xiangchen Zhao, Chenfeng Xu, Sheng Shen, Dongdi Zhao, Kurt Keutzer, Manling Li, Tan Yan, and Xiangjun Fan. 2023. Halle-switch: Rethinking and controlling object existence hallucinations in large vision language models for detailed caption. arXiv preprint arXiv:2310.01779 (2023).
- [53] Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. 2023. Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization. arXiv preprint arXiv:2311.16839 (2023).
- [54] Zhun Zhong, Liang Zheng, Donglin Cao, and Shaozi Li. 2017. Re-ranking person re-identification with k-reciprocal encoding. In Proceedings of the IEEE conference on computer vision and pattern recognition . 1318-1327.
- [55] Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2023. Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754 (2023).
- [56] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 (2023).
- [57] Lanyun Zhu, Deyi Ji, Tianrun Chen, Peng Xu, Jieping Ye, and Jun Liu. 2024. IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding. arXiv preprint arXiv:2402.18476 (2024).
- [58] Linhai Zhuo, Yuqian Fu, Jingjing Chen, Yixin Cao, and Yu-Gang Jiang. 2024. Unified View Empirical Study for Large Pretrained Model on Cross-Domain Few-Shot Learning. ACM Transactions on Multimedia Computing, Communications and Applications (2024).
Xiaoye Qu Qiyuan Chen Wei Wei Jishuo Sun Jianfeng Dong
Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation
Despite the remarkable ability of large vision-language models (LVLMs) in
image comprehension, these models frequently generate plausible yet factually
incorrect responses, a phenomenon known as hallucination.Recently, in large
language models (LLMs), augmenting LLMs by retrieving information from external
knowledge resources has been proven as a promising solution to mitigate
hallucinations.However, the retrieval augmentation in LVLM significantly lags
behind the widespread applications of LVLM. Moreover, when transferred to
augmenting LVLMs, sometimes the hallucination degree of the model is even
exacerbated.Motivated by the research gap and counter-intuitive phenomenon, we
introduce a novel framework, the Active Retrieval-Augmented large
vision-language model (ARA), specifically designed to address hallucinations by
incorporating three critical dimensions: (i) dissecting the retrieval targets
based on the inherent hierarchical structures of images. (ii) pinpointing the
most effective retrieval methods and filtering out the reliable retrieval
results. (iii) timing the retrieval process to coincide with episodes of low
certainty, while circumventing unnecessary retrieval during periods of high
certainty. To assess the capability of our proposed ARA model in reducing
hallucination, we employ three widely used LVLM models (LLaVA-1.5, Qwen-VL, and
mPLUG-Owl2) across four benchmarks. Our empirical observations suggest that by
utilizing fitting retrieval mechanisms and timing the retrieval judiciously, we
can effectively mitigate the hallucination problem. We hope that this study can
provide deeper insights into how to adapt the retrieval augmentation to LVLMs
for reducing hallucinations with more effective retrieval and minimal retrieval
occurrences.