title: TrustGeoGen: Scalable and Formal-Verified Data Engine for Trustworthy
    Multi-modal Geometric Problem Solving
summary: Mathematical geometric problem solving (GPS) often requires effective
  integration of multimodal information and verifiable logical coherence. Despite
  the fast development of large language models in general problem solving, it
  remains unresolved regarding with both methodology and benchmarks, especially
  given the fact that exiting synthetic GPS benchmarks are often not
  self-verified and contain noise and self-contradicted information due to the
  illusion of LLMs. In this paper, we propose a scalable data engine called
  TrustGeoGen for problem generation, with formal verification to provide a
  principled benchmark, which we believe lays the foundation for the further
  development of methods for GPS. The engine synthesizes geometric data through
  four key innovations: 1) multimodal-aligned generation of diagrams, textual
  descriptions, and stepwise solutions; 2) formal verification ensuring
  rule-compliant reasoning paths; 3) a bootstrapping mechanism enabling
  complexity escalation via recursive state generation and 4) our devised
  GeoExplore series algorithms simultaneously produce multi-solution variants and
  self-reflective backtracking traces. By formal logical verification,
  TrustGeoGen produces GeoTrust-200K dataset with guaranteed modality integrity,
  along with GeoTrust-test testset. Experiments reveal the state-of-the-art
  models achieve only 49.17\% accuracy on GeoTrust-test, demonstrating its
  evaluation stringency. Crucially, models trained on GeoTrust achieve OOD
  generalization on GeoQA, significantly reducing logical inconsistencies
  relative to pseudo-label annotated by OpenAI-o1. Our code is available at
  https://github.com/Alpha-Innovator/TrustGeoGen
paper_authors._id: 68104f85b442ffc234b670cd
paper_authors.name: Daocheng Fu
paper_authors._id: 68104f85b442ffc234b670ce
paper_authors.name: Zijun Chen
paper_authors._id: 68104f85b442ffc234b670cf
paper_authors.name: Renqiu Xia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:21:32.294Z
paper_authors._id: 65b7ae76768464877cdb2e39
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65b7ae76768464877cdb2e39/uAWPo4tpkqbZoDeEkc7y0.jpeg
paper_authors.fullname: Renqiu Xia
paper_authors.type: user
paper_authors.user: renqiux0302
paper_authors._id: 68104f85b442ffc234b670d0
paper_authors.name: Qi Liu
paper_authors._id: 68104f85b442ffc234b670d1
paper_authors.name: Yuan Feng
paper_authors._id: 68104f85b442ffc234b670d2
paper_authors.name: Hongbin Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:22:30.987Z
paper_authors._id: 67ea0f099d4f014f8dea2065
paper_authors.avatarUrl: /avatars/11e9b31ec4d4fffc2370e135a2b78461.svg
paper_authors.fullname: zhouhongbin
paper_authors.type: user
paper_authors.user: zhoubb2010
paper_authors._id: 68104f85b442ffc234b670d3
paper_authors.name: Renrui Zhang
paper_authors._id: 68104f85b442ffc234b670d4
paper_authors.name: Shiyang Feng
paper_authors._id: 68104f85b442ffc234b670d5
paper_authors.name: Peng Gao
paper_authors._id: 68104f85b442ffc234b670d6
paper_authors.name: Junchi Yan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:22:54.567Z
paper_authors._id: 667289f903c802764985d8c6
paper_authors.avatarUrl: /avatars/916befcbf0e52ce56be49617f31c7bb2.svg
paper_authors.fullname: Junchi Yan
paper_authors.type: user
paper_authors.user: Rethinker
paper_authors._id: 68104f85b442ffc234b670d7
paper_authors.name: Botian Shi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:23:00.081Z
paper_authors._id: 643df87f7cd64d872cb9fabd
paper_authors.avatarUrl: /avatars/c53bfabcee08de448dde973915e8b31d.svg
paper_authors.fullname: Botian Shi
paper_authors.type: user
paper_authors.user: friskit
paper_authors._id: 68104f85b442ffc234b670d8
paper_authors.name: Bo Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-29T12:23:32.125Z
paper_authors._id: 643dfd235aafbdca3a5792c0
paper_authors.avatarUrl: /avatars/ce8553cf5936012c692e08054ee27937.svg
paper_authors.fullname: Bo Zhang
paper_authors.type: user
paper_authors.user: BoZhang
paper_authors._id: 68104f85b442ffc234b670d9
paper_authors.name: Yu Qiao
paper_title: TrustGeoGen: Scalable and Formal-Verified Data Engine for Trustworthy
    Multi-modal Geometric Problem Solving
paper_summary: Mathematical geometric problem solving (GPS) often requires effective
  integration of multimodal information and verifiable logical coherence. Despite
  the fast development of large language models in general problem solving, it
  remains unresolved regarding with both methodology and benchmarks, especially
  given the fact that exiting synthetic GPS benchmarks are often not
  self-verified and contain noise and self-contradicted information due to the
  illusion of LLMs. In this paper, we propose a scalable data engine called
  TrustGeoGen for problem generation, with formal verification to provide a
  principled benchmark, which we believe lays the foundation for the further
  development of methods for GPS. The engine synthesizes geometric data through
  four key innovations: 1) multimodal-aligned generation of diagrams, textual
  descriptions, and stepwise solutions; 2) formal verification ensuring
  rule-compliant reasoning paths; 3) a bootstrapping mechanism enabling
  complexity escalation via recursive state generation and 4) our devised
  GeoExplore series algorithms simultaneously produce multi-solution variants and
  self-reflective backtracking traces. By formal logical verification,
  TrustGeoGen produces GeoTrust-200K dataset with guaranteed modality integrity,
  along with GeoTrust-test testset. Experiments reveal the state-of-the-art
  models achieve only 49.17\% accuracy on GeoTrust-test, demonstrating its
  evaluation stringency. Crucially, models trained on GeoTrust achieve OOD
  generalization on GeoQA, significantly reducing logical inconsistencies
  relative to pseudo-label annotated by OpenAI-o1. Our code is available at
  https://github.com/Alpha-Innovator/TrustGeoGen
paper_ai_summary: TrustGeoGen is a scalable data engine providing a formally verified benchmark for GPS through innovative multimodal generation and logical verification, improving model accuracy and OOD generalization.
paper_ai_keywords: multimodal-aligned generation formal verification rule-compliant reasoning paths bootstrapping mechanism recursive state generation self-reflective backtracking GeoExplore GeoTrust-200K GeoTrust-test state-of-the-art accuracy OOD generalization logical inconsistencies