G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
Large language models (LLMs) have shown remarkable proficiency in human-level
reasoning and generation capabilities, which encourages extensive research on
their application in mathematical problem solving. However, current work has
been largely focused on text-based mathematical problems, with limited
investigation in problems involving geometric information. Addressing this gap,
we aim to enable LLMs to solve geometric problems by understanding image input.
We first analyze the limitations of current Multimodal Large Language Models
(MLLMs) in this area: they struggle to accurately comprehending basic geometric
elements and their relationships. To overcome these challenges, we take
advantage of the unique characteristics of geometric problems (such as unique
geometric logical form, and geometric scalability) and the capacity of the
textual LLMs to build an enriched multimodal geometry dataset based on existing
data. The augmented dataset, Geo170K, contains more than 170K geometric
image-caption and question-answer pairs. Utilizing our constructed Geo170K
dataset, we develop G-LLaVA, which demonstrates exceptional performance in
solving geometric problems, significantly outperforming GPT-4-V on the
MathVista benchmark with only 7B parameters.
65812a95defc9c0d30099a06
Jiahui Gao
admin_assigned
2023-12-19T08:50:56.699Z
645da35aa19f3e64bbf294de
/avatars/4a80eed597a92d00a162db062ac1517f.svg
gaojiahui
user
gaojiahui1
65812a95defc9c0d30099a07
Renjie Pi
admin_assigned
2023-12-19T08:51:04.996Z
63f45b8d520c14618930d175
/avatars/a20994594579b52a8be8bd2c4acbb913.svg
renjie
user
renjiepi
65812a95defc9c0d30099a08
Jipeng Zhang
admin_assigned
2023-12-19T08:51:17.524Z
627a65adfc495b60d7513b75
/avatars/65eddcb68bda5c5c7fccbf34157107ef.svg
ZHANG Jipeng
user
2003pro
65812a95defc9c0d30099a09
Jiacheng Ye
admin_assigned
2023-12-19T08:51:24.189Z
632b2bb06110e37dba3c75cf
/avatars/c91a5774cfec04efc3c986c871e8197f.svg
Jiacheng Ye
user
Jiacheng513
65812a95defc9c0d30099a0a
Wanjun Zhong
admin_assigned
2023-12-19T08:51:32.289Z
643f956635e2b54a42e7feba
/avatars/c6185f81ae8499ae866ad451c1cbf43b.svg
Wanjun Zhong
user
WanjunZhong
65812a95defc9c0d30099a0b
Yufei Wang
65812a95defc9c0d30099a0c
Lanqing Hong
65812a95defc9c0d30099a0d
Jianhua Han
65812a95defc9c0d30099a0e
Hang Xu
65812a95defc9c0d30099a0f
Zhenguo Li
65812a95defc9c0d30099a10
Lingpeng Kong
G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
Large language models (LLMs) have shown remarkable proficiency in human-level
reasoning and generation capabilities, which encourages extensive research on
their application in mathematical problem solving. However, current work has
been largely focused on text-based mathematical problems, with limited
investigation in problems involving geometric information. Addressing this gap,
we aim to enable LLMs to solve geometric problems by understanding image input.
We first analyze the limitations of current Multimodal Large Language Models
(MLLMs) in this area: they struggle to accurately comprehending basic geometric
elements and their relationships. To overcome these challenges, we take
advantage of the unique characteristics of geometric problems (such as unique
geometric logical form, and geometric scalability) and the capacity of the
textual LLMs to build an enriched multimodal geometry dataset based on existing
data. The augmented dataset, Geo170K, contains more than 170K geometric
image-caption and question-answer pairs. Utilizing our constructed Geo170K
dataset, we develop G-LLaVA, which demonstrates exceptional performance in
solving geometric problems, significantly outperforming GPT-4-V on the
MathVista benchmark with only 7B parameters.
G-LLaVA outperforms GPT-4-V on solving geometric problems using an enriched multimodal geometry dataset, Geo170K.
Multimodal Large Language Models geometric elements geometric logical form geometric scalability text-based mathematical problems image-caption question-answer pairs G-LLaVA MathVista benchmark