title: Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language
    Models
summary: The burgeoning interest in Multimodal Large Language Models (MLLMs), such as
  OpenAI's GPT-4V(ision), has significantly impacted both academic and industrial
  realms. These models enhance Large Language Models (LLMs) with advanced visual
  understanding capabilities, facilitating their application in a variety of
  multimodal tasks. Recently, Google introduced Gemini, a cutting-edge MLLM
  designed specifically for multimodal integration. Despite its advancements,
  preliminary benchmarks indicate that Gemini lags behind GPT models in
  commonsense reasoning tasks. However, this assessment, based on a limited
  dataset (i.e., HellaSWAG), does not fully capture Gemini's authentic
  commonsense reasoning potential. To address this gap, our study undertakes a
  thorough evaluation of Gemini's performance in complex reasoning tasks that
  necessitate the integration of commonsense knowledge across modalities. We
  carry out a comprehensive analysis of 12 commonsense reasoning datasets,
  ranging from general to domain-specific tasks. This includes 11 datasets
  focused solely on language, as well as one that incorporates multimodal
  elements. Our experiments across four LLMs and two MLLMs demonstrate Gemini's
  competitive commonsense reasoning capabilities. Additionally, we identify
  common challenges faced by current LLMs and MLLMs in addressing commonsense
  problems, underscoring the need for further advancements in enhancing the
  commonsense reasoning abilities of these models.
paper_authors._id: 65922d2ac0a266442631c4ce
paper_authors.name: Yuqing Wang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-20T13:41:29.074Z
paper_authors._id: 65b9738b011deaafb432a053
paper_authors.avatarUrl: /avatars/1929f20b242d4fd9f083963e0bed0ccf.svg
paper_authors.fullname: Yuqing Wang
paper_authors.type: user
paper_authors.user: EternityYW
paper_authors._id: 65922d2ac0a266442631c4cf
paper_authors.name: Yun Zhao
paper_title: Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language
    Models
paper_summary: The burgeoning interest in Multimodal Large Language Models (MLLMs), such as
  OpenAI's GPT-4V(ision), has significantly impacted both academic and industrial
  realms. These models enhance Large Language Models (LLMs) with advanced visual
  understanding capabilities, facilitating their application in a variety of
  multimodal tasks. Recently, Google introduced Gemini, a cutting-edge MLLM
  designed specifically for multimodal integration. Despite its advancements,
  preliminary benchmarks indicate that Gemini lags behind GPT models in
  commonsense reasoning tasks. However, this assessment, based on a limited
  dataset (i.e., HellaSWAG), does not fully capture Gemini's authentic
  commonsense reasoning potential. To address this gap, our study undertakes a
  thorough evaluation of Gemini's performance in complex reasoning tasks that
  necessitate the integration of commonsense knowledge across modalities. We
  carry out a comprehensive analysis of 12 commonsense reasoning datasets,
  ranging from general to domain-specific tasks. This includes 11 datasets
  focused solely on language, as well as one that incorporates multimodal
  elements. Our experiments across four LLMs and two MLLMs demonstrate Gemini's
  competitive commonsense reasoning capabilities. Additionally, we identify
  common challenges faced by current LLMs and MLLMs in addressing commonsense
  problems, underscoring the need for further advancements in enhancing the
  commonsense reasoning abilities of these models.
paper_ai_summary: Gemini, a multimodal large language model, demonstrates competitive commonsense reasoning abilities across a variety of datasets, highlighting the need for ongoing advancements in this area.
paper_ai_keywords: multimodal large language models MLLMs GPT-4V Gemini commonsense reasoning HellaSWAG multimodal integration large language models LLMs