PHYSICS: Benchmarking Foundation Models on University-Level Physics
  Problem Solving
We introduce PHYSICS, a comprehensive benchmark for university-level physics
problem solving. It contains 1297 expert-annotated problems covering six core
areas: classical mechanics, quantum mechanics, thermodynamics and statistical
mechanics, electromagnetism, atomic physics, and optics. Each problem requires
advanced physics knowledge and mathematical reasoning. We develop a robust
automated evaluation system for precise and reliable validation. Our evaluation
of leading foundation models reveals substantial limitations. Even the most
advanced model, o3-mini, achieves only 59.9% accuracy, highlighting significant
challenges in solving high-level scientific problems. Through comprehensive
error analysis, exploration of diverse prompting strategies, and
Retrieval-Augmented Generation (RAG)-based knowledge augmentation, we identify
key areas for improvement, laying the foundation for future advancements.
67e9ffab6887b70da56d0de5
Kaiyue Feng
admin_assigned
2025-03-31T08:30:14.677Z
668b6668cc2c0b4ae303bdb8
/avatars/2a4e30c0a5ee76b66232f425d5e62747.svg
Kaiyue Feng
user
Carrie777
67e9ffab6887b70da56d0de6
Yilun Zhao
claimed_verified
2025-03-31T08:11:42.629Z
62f662bcc58915315c4eccea
https://cdn-avatars.huggingface.co/v1/production/uploads/62f662bcc58915315c4eccea/zOAQLONfMP88zr70sxHK-.jpeg
Yilun Zhao
user
yilunzhao
67e9ffab6887b70da56d0de7
Yixin Liu
admin_assigned
2025-03-31T08:30:33.598Z
6244de1c1c560fb11edfca44
/avatars/36558928bd04be7f49837d4c603681d7.svg
Yixin Liu
user
henryL7
67e9ffab6887b70da56d0de8
Tianyu Yang
67e9ffab6887b70da56d0de9
Chen Zhao
67e9ffab6887b70da56d0dea
John Sous
admin_assigned
2025-03-31T08:30:58.653Z
6626e136cee7ea5738a8442b
/avatars/8c1537773e2c70f9c10b51a004380824.svg
John Sous
user
jsous
67e9ffab6887b70da56d0deb
Arman Cohan
admin_assigned
2025-03-31T08:31:04.921Z
5f5ba21188f57f65f951f255
https://cdn-avatars.huggingface.co/v1/production/uploads/1599840760465-noauth.png
Arman Cohan
user
armanc
PHYSICS: Benchmarking Foundation Models on University-Level Physics
  Problem Solving
We introduce PHYSICS, a comprehensive benchmark for university-level physics
problem solving. It contains 1297 expert-annotated problems covering six core
areas: classical mechanics, quantum mechanics, thermodynamics and statistical
mechanics, electromagnetism, atomic physics, and optics. Each problem requires
advanced physics knowledge and mathematical reasoning. We develop a robust
automated evaluation system for precise and reliable validation. Our evaluation
of leading foundation models reveals substantial limitations. Even the most
advanced model, o3-mini, achieves only 59.9% accuracy, highlighting significant
challenges in solving high-level scientific problems. Through comprehensive
error analysis, exploration of diverse prompting strategies, and
Retrieval-Augmented Generation (RAG)-based knowledge augmentation, we identify
key areas for improvement, laying the foundation for future advancements.
A comprehensive physics problem-solving benchmark reveals limitations in existing foundation models, guiding future improvements through error analysis and knowledge augmentation techniques.
Retrieval-Augmented Generation (RAG)