Investigating Hallucination in Conversations for Low Resource Languages
Large Language Models (LLMs) have demonstrated remarkable proficiency in
generating text that closely resemble human writing. However, they often
generate factually incorrect statements, a problem typically referred to as
'hallucination'. Addressing hallucination is crucial for enhancing the
reliability and effectiveness of LLMs. While much research has focused on
hallucinations in English, our study extends this investigation to
conversational data in three languages: Hindi, Farsi, and Mandarin. We offer a
comprehensive analysis of a dataset to examine both factual and linguistic
errors in these languages for GPT-3.5, GPT-4o, Llama-3.1, Gemma-2.0,
DeepSeek-R1 and Qwen-3. We found that LLMs produce very few hallucinated
responses in Mandarin but generate a significantly higher number of
hallucinations in Hindi and Farsi.
689036f00a411b3b8d28d6d1
Amit Das
689036f00a411b3b8d28d6d2
Md. Najib Hasan
689036f00a411b3b8d28d6d3
Souvika Sarkar
689036f00a411b3b8d28d6d4
Zheng Zhang
689036f00a411b3b8d28d6d5
Fatemeh Jamshidi
689036f00a411b3b8d28d6d6
Tathagata Bhattacharya
689036f00a411b3b8d28d6d7
Nilanjana Raychawdhury
689036f00a411b3b8d28d6d8
Dongji Feng
689036f00a411b3b8d28d6d9
Vinija Jain
689036f00a411b3b8d28d6da
Aman Chadha
Investigating Hallucination in Conversations for Low Resource Languages
Large Language Models (LLMs) have demonstrated remarkable proficiency in
generating text that closely resemble human writing. However, they often
generate factually incorrect statements, a problem typically referred to as
'hallucination'. Addressing hallucination is crucial for enhancing the
reliability and effectiveness of LLMs. While much research has focused on
hallucinations in English, our study extends this investigation to
conversational data in three languages: Hindi, Farsi, and Mandarin. We offer a
comprehensive analysis of a dataset to examine both factual and linguistic
errors in these languages for GPT-3.5, GPT-4o, Llama-3.1, Gemma-2.0,
DeepSeek-R1 and Qwen-3. We found that LLMs produce very few hallucinated
responses in Mandarin but generate a significantly higher number of
hallucinations in Hindi and Farsi.
LLMs generate fewer hallucinations in Mandarin compared to Hindi and Farsi across multiple models.
Large Language Models LLMs hallucination GPT-3.5 GPT-4o Llama-3.1 Gemma-2.0 DeepSeek-R1 Qwen-3