Do Large Language Models Truly Understand Geometric Structures?
Xiaofeng Wang Yiming Wang Wenhong Zhu Rui Wang
Geometric ability is a significant challenge for large language models (LLMs) due to the need for advanced spatial comprehension and abstract thinking. Existing datasets primarily evaluate LLMs on their final answers, but they cannot truly measure their true understanding of geometric structures, as LLMs can arrive at correct answers by coincidence. To fill this gap, we introduce the GeomRel dataset, designed to evaluate LLMs’ understanding of geometric structures by isolating the core step of geometric relationship identification in problem-solving. Using this benchmark, we conduct thorough evaluations of diverse LLMs and identify key limitations in understanding geometric structures. We further propose the Geometry Chain-of-Thought (GeoCoT) method, which enhances LLMs’ ability to identify geometric relationships, resulting in significant performance improvements.
Large Language Models, Geometric Ability Evaluation, Geometric Relationship Identification
This paper introduces a dataset to evaluate large language models’ understanding of geometric structures and proposes a method to enhance their ability to identify geometric relationships.