title: G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
summary: Large language models (LLMs) have shown remarkable proficiency in human-level
  reasoning and generation capabilities, which encourages extensive research on
  their application in mathematical problem solving. However, current work has
  been largely focused on text-based mathematical problems, with limited
  investigation in problems involving geometric information. Addressing this gap,
  we aim to enable LLMs to solve geometric problems by understanding image input.
  We first analyze the limitations of current Multimodal Large Language Models
  (MLLMs) in this area: they struggle to accurately comprehending basic geometric
  elements and their relationships. To overcome these challenges, we take
  advantage of the unique characteristics of geometric problems (such as unique
  geometric logical form, and geometric scalability) and the capacity of the
  textual LLMs to build an enriched multimodal geometry dataset based on existing
  data. The augmented dataset, Geo170K, contains more than 170K geometric
  image-caption and question-answer pairs. Utilizing our constructed Geo170K
  dataset, we develop G-LLaVA, which demonstrates exceptional performance in
  solving geometric problems, significantly outperforming GPT-4-V on the
  MathVista benchmark with only 7B parameters.
paper_authors._id: 65812a95defc9c0d30099a06
paper_authors.name: Jiahui Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T08:50:56.699Z
paper_authors._id: 645da35aa19f3e64bbf294de
paper_authors.avatarUrl: /avatars/4a80eed597a92d00a162db062ac1517f.svg
paper_authors.fullname: gaojiahui
paper_authors.type: user
paper_authors.user: gaojiahui1
paper_authors._id: 65812a95defc9c0d30099a07
paper_authors.name: Renjie Pi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T08:51:04.996Z
paper_authors._id: 63f45b8d520c14618930d175
paper_authors.avatarUrl: /avatars/a20994594579b52a8be8bd2c4acbb913.svg
paper_authors.fullname: renjie
paper_authors.type: user
paper_authors.user: renjiepi
paper_authors._id: 65812a95defc9c0d30099a08
paper_authors.name: Jipeng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T08:51:17.524Z
paper_authors._id: 627a65adfc495b60d7513b75
paper_authors.avatarUrl: /avatars/65eddcb68bda5c5c7fccbf34157107ef.svg
paper_authors.fullname: ZHANG Jipeng
paper_authors.type: user
paper_authors.user: 2003pro
paper_authors._id: 65812a95defc9c0d30099a09
paper_authors.name: Jiacheng Ye
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T08:51:24.189Z
paper_authors._id: 632b2bb06110e37dba3c75cf
paper_authors.avatarUrl: /avatars/c91a5774cfec04efc3c986c871e8197f.svg
paper_authors.fullname: Jiacheng Ye
paper_authors.type: user
paper_authors.user: Jiacheng513
paper_authors._id: 65812a95defc9c0d30099a0a
paper_authors.name: Wanjun Zhong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-19T08:51:32.289Z
paper_authors._id: 643f956635e2b54a42e7feba
paper_authors.avatarUrl: /avatars/c6185f81ae8499ae866ad451c1cbf43b.svg
paper_authors.fullname: Wanjun Zhong
paper_authors.type: user
paper_authors.user: WanjunZhong
paper_authors._id: 65812a95defc9c0d30099a0b
paper_authors.name: Yufei Wang
paper_authors._id: 65812a95defc9c0d30099a0c
paper_authors.name: Lanqing Hong
paper_authors._id: 65812a95defc9c0d30099a0d
paper_authors.name: Jianhua Han
paper_authors._id: 65812a95defc9c0d30099a0e
paper_authors.name: Hang Xu
paper_authors._id: 65812a95defc9c0d30099a0f
paper_authors.name: Zhenguo Li
paper_authors._id: 65812a95defc9c0d30099a10
paper_authors.name: Lingpeng Kong
paper_title: G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
paper_summary: Large language models (LLMs) have shown remarkable proficiency in human-level
  reasoning and generation capabilities, which encourages extensive research on
  their application in mathematical problem solving. However, current work has
  been largely focused on text-based mathematical problems, with limited
  investigation in problems involving geometric information. Addressing this gap,
  we aim to enable LLMs to solve geometric problems by understanding image input.
  We first analyze the limitations of current Multimodal Large Language Models
  (MLLMs) in this area: they struggle to accurately comprehending basic geometric
  elements and their relationships. To overcome these challenges, we take
  advantage of the unique characteristics of geometric problems (such as unique
  geometric logical form, and geometric scalability) and the capacity of the
  textual LLMs to build an enriched multimodal geometry dataset based on existing
  data. The augmented dataset, Geo170K, contains more than 170K geometric
  image-caption and question-answer pairs. Utilizing our constructed Geo170K
  dataset, we develop G-LLaVA, which demonstrates exceptional performance in
  solving geometric problems, significantly outperforming GPT-4-V on the
  MathVista benchmark with only 7B parameters.
paper_ai_summary: G-LLaVA outperforms GPT-4-V on solving geometric problems using an enriched multimodal geometry dataset, Geo170K.
paper_ai_keywords: Multimodal Large Language Models geometric elements geometric logical form geometric scalability text-based mathematical problems image-caption question-answer pairs G-LLaVA MathVista benchmark