GeoX: Geometric Problem Solving Through Unified Formalized
  Vision-Language Pre-training
Despite their proficiency in general tasks, Multi-modal Large Language Models
(MLLMs) struggle with automatic Geometry Problem Solving (GPS), which demands
understanding diagrams, interpreting symbols, and performing complex reasoning.
This limitation arises from their pre-training on natural images and texts,
along with the lack of automated verification in the problem-solving process.
Besides, current geometric specialists are limited by their task-specific
designs, making them less effective for broader geometric problems. To this
end, we present GeoX, a multi-modal large model focusing on geometric
understanding and reasoning tasks. Given the significant differences between
geometric diagram-symbol and natural image-text, we introduce unimodal
pre-training to develop a diagram encoder and symbol decoder, enhancing the
understanding of geometric images and corpora. Furthermore, we introduce
geometry-language alignment, an effective pre-training paradigm that bridges
the modality gap between unimodal geometric experts. We propose a
Generator-And-Sampler Transformer (GS-Former) to generate discriminative
queries and eliminate uninformative representations from unevenly distributed
geometric signals. Finally, GeoX benefits from visual instruction tuning,
empowering it to take geometric images and questions as input and generate
verifiable solutions. Experiments show that GeoX outperforms both generalists
and geometric specialists on publicly recognized benchmarks, such as GeoQA,
UniGeo, Geometry3K, and PGPS9k.
676221f793b104717dc78d85
Renqiu Xia
admin_assigned
2024-12-18T13:38:03.402Z
65b7ae76768464877cdb2e39
https://cdn-avatars.huggingface.co/v1/production/uploads/65b7ae76768464877cdb2e39/uAWPo4tpkqbZoDeEkc7y0.jpeg
Renqiu Xia
user
renqiux0302
676221f793b104717dc78d86
Mingsheng Li
admin_assigned
2024-12-18T13:38:10.777Z
674d8c97d086436faf52c890
/avatars/c4bde24b0796f725a9f4e68f72fe6a44.svg
mingsheng li
user
mingsheng1
676221f793b104717dc78d87
Hancheng Ye
admin_assigned
2024-12-18T13:38:19.064Z
6597f70d260bcd5fb5529613
/avatars/8fd7a210f3cb0de1229b49a3cd1ad8a0.svg
Hancheng Ye
user
HankYe
676221f793b104717dc78d88
Wenjie Wu
676221f793b104717dc78d89
Hongbin Zhou
claimed_verified
2024-12-20T13:56:02.541Z
646db87ee4fca2e115db4594
/avatars/f2f3825e3c0c6ac99cd89024570f4063.svg
Hongbin
user
VictorChew
676221f793b104717dc78d8a
Jiakang Yuan
admin_assigned
2024-12-18T13:38:53.074Z
64a3d1ddb3239f3e3892b24b
/avatars/7ce585f5fc1d077fb1d70cc18c4da2c1.svg
Jiakang Yuan
user
JiakangYuan
676221f793b104717dc78d8b
Tianshuo Peng
admin_assigned
2024-12-18T13:39:00.721Z
6538dd471ad9b3ba7c2df861
https://cdn-avatars.huggingface.co/v1/production/uploads/6538dd471ad9b3ba7c2df861/MbEa7KHAK6u7PRb7WiPUC.jpeg
Tianshuo Peng
user
Potentialts
676221f793b104717dc78d8c
Xinyu Cai
admin_assigned
2024-12-18T13:39:06.933Z
64521a2a7ba5cb58fabefcae
/avatars/caff912e95348c36b34dfc4f1509d42b.svg
XinyuCai
user
XinyuCai
676221f793b104717dc78d8d
Xiangchao Yan
admin_assigned
2024-12-18T13:39:14.242Z
65b88b92e0bde92c176a888a
/avatars/fc1cb54328ca93860e97fc73a3c1eb2f.svg
Xiangchao Yan
user
yxc97
676221f793b104717dc78d8e
Bin Wang
676221f793b104717dc78d8f
Conghui He
admin_assigned
2024-12-18T13:39:21.989Z
63f9fca8d4349b157a109eec
/avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
Conghui He
user
conghui
676221f793b104717dc78d90
Botian Shi
admin_assigned
2024-12-18T13:39:30.085Z
643df87f7cd64d872cb9fabd
/avatars/c53bfabcee08de448dde973915e8b31d.svg
Botian Shi
user
friskit
676221f793b104717dc78d91
Tao Chen
676221f793b104717dc78d92
Junchi Yan
admin_assigned
2024-12-18T13:39:37.318Z
667289f903c802764985d8c6
/avatars/916befcbf0e52ce56be49617f31c7bb2.svg
Junchi Yan
user
Rethinker
676221f793b104717dc78d93
Bo Zhang
claimed_verified
2024-12-18T09:52:49.814Z
643dfd235aafbdca3a5792c0
/avatars/ce8553cf5936012c692e08054ee27937.svg
Bo Zhang
user
BoZhang
GeoX: Geometric Problem Solving Through Unified Formalized
  Vision-Language Pre-training
Despite their proficiency in general tasks, Multi-modal Large Language Models
(MLLMs) struggle with automatic Geometry Problem Solving (GPS), which demands
understanding diagrams, interpreting symbols, and performing complex reasoning.
This limitation arises from their pre-training on natural images and texts,
along with the lack of automated verification in the problem-solving process.
Besides, current geometric specialists are limited by their task-specific
designs, making them less effective for broader geometric problems. To this
end, we present GeoX, a multi-modal large model focusing on geometric
understanding and reasoning tasks. Given the significant differences between
geometric diagram-symbol and natural image-text, we introduce unimodal
pre-training to develop a diagram encoder and symbol decoder, enhancing the
understanding of geometric images and corpora. Furthermore, we introduce
geometry-language alignment, an effective pre-training paradigm that bridges
the modality gap between unimodal geometric experts. We propose a
Generator-And-Sampler Transformer (GS-Former) to generate discriminative
queries and eliminate uninformative representations from unevenly distributed
geometric signals. Finally, GeoX benefits from visual instruction tuning,
empowering it to take geometric images and questions as input and generate
verifiable solutions. Experiments show that GeoX outperforms both generalists
and geometric specialists on publicly recognized benchmarks, such as GeoQA,
UniGeo, Geometry3K, and PGPS9k.
GeoX is a multi-modal model designed for geometric problem solving, incorporating unimodal pre-training, geometry-language alignment, and visual instruction tuning to enhance performance on benchmarks like GeoQA, UniGeo, Geometry3K, and PGPS9k.
Multi-modal Large Language Models automatic Geometry Problem Solving diagram encoder symbol decoder geometry-language alignment Generator-And-Sampler Transformer GS-Former visual instruction tuning