12.2 Hallucination in Other VL Tasks
In addition to image captioning, hallucination has also been observed in other VL tasks and raised
as an open research question. For example, in open-ended visual question answering, Figure 3 (left
and right) shows that the model could generate seem likely answers when we only see the text,
however wrong when given the image. Moreover, Figure 3 (middle) indicates that hallucination can
also be triggered by adversarially prompting an unanswerable question. The model will imagine an
unsupported answer that commonly matches the given visual scene.

Fig. 3. Examples of hallucination in visual question answering (taken from [1]). The bold text is the output
generated by the model and the part before it is the input prompt.
SurveyofHallucinationinNatural Language Generation