title: SeePhys: Does Seeing Help Thinking? -- Benchmarking Vision-Based Physics
    Reasoning
summary: We present SeePhys, a large-scale multimodal benchmark for LLM reasoning
  grounded in physics questions ranging from middle school to PhD qualifying
  exams. The benchmark covers 7 fundamental domains spanning the physics
  discipline, incorporating 21 categories of highly heterogeneous diagrams. In
  contrast to prior works where visual elements mainly serve auxiliary purposes,
  our benchmark features a substantial proportion of vision-essential problems
  (75\%) that mandate visual information extraction for correct solutions.
  Through extensive evaluation, we observe that even the most advanced visual
  reasoning models (e.g., Gemini-2.5-pro and o4-mini) achieve sub-60\% accuracy
  on our benchmark. These results reveal fundamental challenges in current large
  language models' visual understanding capabilities, particularly in: (i)
  establishing rigorous coupling between diagram interpretation and physics
  reasoning, and (ii) overcoming their persistent reliance on textual cues as
  cognitive shortcuts.
paper_authors._id: 68367c93f6cadba33fdfb17c
paper_authors.name: Kun Xiang
paper_authors._id: 68367c93f6cadba33fdfb17d
paper_authors.name: Heng Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:57:31.603Z
paper_authors._id: 67604fe49dec814e4b7b772e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/67604fe49dec814e4b7b772e/XQNDrAIj4NIP-ZZsRQg_2.jpeg
paper_authors.fullname: HengLi
paper_authors.type: user
paper_authors.user: HengLi29
paper_authors._id: 68367c93f6cadba33fdfb17e
paper_authors.name: Terry Jingchen Zhang
paper_authors._id: 68367c93f6cadba33fdfb17f
paper_authors.name: Yinya Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:57:28.968Z
paper_authors._id: 6628c1d30ccfcdcc321fc624
paper_authors.avatarUrl: /avatars/016674fdb50219847e20aa1130a9e882.svg
paper_authors.fullname: Yinya Eleanor Huang
paper_authors.type: user
paper_authors.user: yinyahuang
paper_authors._id: 68367c93f6cadba33fdfb180
paper_authors.name: Zirong Liu
paper_authors._id: 68367c93f6cadba33fdfb181
paper_authors.name: Peixin Qu
paper_authors._id: 68367c93f6cadba33fdfb182
paper_authors.name: Jixi He
paper_authors._id: 68367c93f6cadba33fdfb183
paper_authors.name: Jiaqi Chen
paper_authors._id: 68367c93f6cadba33fdfb184
paper_authors.name: Yu-Jie Yuan
paper_authors._id: 68367c93f6cadba33fdfb185
paper_authors.name: Jianhua Han
paper_authors._id: 68367c93f6cadba33fdfb186
paper_authors.name: Hang Xu
paper_authors._id: 68367c93f6cadba33fdfb187
paper_authors.name: Hanhui Li
paper_authors._id: 68367c93f6cadba33fdfb188
paper_authors.name: Mrinmaya Sachan
paper_authors._id: 68367c93f6cadba33fdfb189
paper_authors.name: Xiaodan Liang
paper_title: SeePhys: Does Seeing Help Thinking? -- Benchmarking Vision-Based Physics
    Reasoning
paper_summary: We present SeePhys, a large-scale multimodal benchmark for LLM reasoning
  grounded in physics questions ranging from middle school to PhD qualifying
  exams. The benchmark covers 7 fundamental domains spanning the physics
  discipline, incorporating 21 categories of highly heterogeneous diagrams. In
  contrast to prior works where visual elements mainly serve auxiliary purposes,
  our benchmark features a substantial proportion of vision-essential problems
  (75\%) that mandate visual information extraction for correct solutions.
  Through extensive evaluation, we observe that even the most advanced visual
  reasoning models (e.g., Gemini-2.5-pro and o4-mini) achieve sub-60\% accuracy
  on our benchmark. These results reveal fundamental challenges in current large
  language models' visual understanding capabilities, particularly in: (i)
  establishing rigorous coupling between diagram interpretation and physics
  reasoning, and (ii) overcoming their persistent reliance on textual cues as
  cognitive shortcuts.
paper_ai_summary: SeePhys, a multimodal benchmark, highlights challenges in LLMs' visual reasoning and physics-grounded problem-solving capabilities, especially in interpreting diagrams and reducing reliance on textual cues.
paper_ai_keywords: LLM reasoning multimodal benchmark vision-essential problems visual information extraction visual reasoning models diagram interpretation physics reasoning cognitive shortcuts