TrustGeoGen: Scalable and Formal-Verified Data Engine for Trustworthy
  Multi-modal Geometric Problem Solving
Mathematical geometric problem solving (GPS) often requires effective
integration of multimodal information and verifiable logical coherence. Despite
the fast development of large language models in general problem solving, it
remains unresolved regarding with both methodology and benchmarks, especially
given the fact that exiting synthetic GPS benchmarks are often not
self-verified and contain noise and self-contradicted information due to the
illusion of LLMs. In this paper, we propose a scalable data engine called
TrustGeoGen for problem generation, with formal verification to provide a
principled benchmark, which we believe lays the foundation for the further
development of methods for GPS. The engine synthesizes geometric data through
four key innovations: 1) multimodal-aligned generation of diagrams, textual
descriptions, and stepwise solutions; 2) formal verification ensuring
rule-compliant reasoning paths; 3) a bootstrapping mechanism enabling
complexity escalation via recursive state generation and 4) our devised
GeoExplore series algorithms simultaneously produce multi-solution variants and
self-reflective backtracking traces. By formal logical verification,
TrustGeoGen produces GeoTrust-200K dataset with guaranteed modality integrity,
along with GeoTrust-test testset. Experiments reveal the state-of-the-art
models achieve only 49.17\% accuracy on GeoTrust-test, demonstrating its
evaluation stringency. Crucially, models trained on GeoTrust achieve OOD
generalization on GeoQA, significantly reducing logical inconsistencies
relative to pseudo-label annotated by OpenAI-o1. Our code is available at
https://github.com/Alpha-Innovator/TrustGeoGen
68104f85b442ffc234b670cd
Daocheng Fu
68104f85b442ffc234b670ce
Zijun Chen
68104f85b442ffc234b670cf
Renqiu Xia
admin_assigned
2025-04-29T12:21:32.294Z
65b7ae76768464877cdb2e39
https://cdn-avatars.huggingface.co/v1/production/uploads/65b7ae76768464877cdb2e39/uAWPo4tpkqbZoDeEkc7y0.jpeg
Renqiu Xia
user
renqiux0302
68104f85b442ffc234b670d0
Qi Liu
68104f85b442ffc234b670d1
Yuan Feng
68104f85b442ffc234b670d2
Hongbin Zhou
admin_assigned
2025-04-29T12:22:30.987Z
67ea0f099d4f014f8dea2065
/avatars/11e9b31ec4d4fffc2370e135a2b78461.svg
zhouhongbin
user
zhoubb2010
68104f85b442ffc234b670d3
Renrui Zhang
68104f85b442ffc234b670d4
Shiyang Feng
68104f85b442ffc234b670d5
Peng Gao
68104f85b442ffc234b670d6
Junchi Yan
admin_assigned
2025-04-29T12:22:54.567Z
667289f903c802764985d8c6
/avatars/916befcbf0e52ce56be49617f31c7bb2.svg
Junchi Yan
user
Rethinker
68104f85b442ffc234b670d7
Botian Shi
admin_assigned
2025-04-29T12:23:00.081Z
643df87f7cd64d872cb9fabd
/avatars/c53bfabcee08de448dde973915e8b31d.svg
Botian Shi
user
friskit
68104f85b442ffc234b670d8
Bo Zhang
admin_assigned
2025-04-29T12:23:32.125Z
643dfd235aafbdca3a5792c0
/avatars/ce8553cf5936012c692e08054ee27937.svg
Bo Zhang
user
BoZhang
68104f85b442ffc234b670d9
Yu Qiao
TrustGeoGen: Scalable and Formal-Verified Data Engine for Trustworthy
  Multi-modal Geometric Problem Solving
Mathematical geometric problem solving (GPS) often requires effective
integration of multimodal information and verifiable logical coherence. Despite
the fast development of large language models in general problem solving, it
remains unresolved regarding with both methodology and benchmarks, especially
given the fact that exiting synthetic GPS benchmarks are often not
self-verified and contain noise and self-contradicted information due to the
illusion of LLMs. In this paper, we propose a scalable data engine called
TrustGeoGen for problem generation, with formal verification to provide a
principled benchmark, which we believe lays the foundation for the further
development of methods for GPS. The engine synthesizes geometric data through
four key innovations: 1) multimodal-aligned generation of diagrams, textual
descriptions, and stepwise solutions; 2) formal verification ensuring
rule-compliant reasoning paths; 3) a bootstrapping mechanism enabling
complexity escalation via recursive state generation and 4) our devised
GeoExplore series algorithms simultaneously produce multi-solution variants and
self-reflective backtracking traces. By formal logical verification,
TrustGeoGen produces GeoTrust-200K dataset with guaranteed modality integrity,
along with GeoTrust-test testset. Experiments reveal the state-of-the-art
models achieve only 49.17\% accuracy on GeoTrust-test, demonstrating its
evaluation stringency. Crucially, models trained on GeoTrust achieve OOD
generalization on GeoQA, significantly reducing logical inconsistencies
relative to pseudo-label annotated by OpenAI-o1. Our code is available at
https://github.com/Alpha-Innovator/TrustGeoGen
TrustGeoGen is a scalable data engine providing a formally verified benchmark for GPS through innovative multimodal generation and logical verification, improving model accuracy and OOD generalization.
multimodal-aligned generation formal verification rule-compliant reasoning paths bootstrapping mechanism recursive state generation self-reflective backtracking GeoExplore GeoTrust-200K GeoTrust-test state-of-the-art accuracy OOD generalization logical inconsistencies