A study re-evaluating frontier language models on physics benchmarks found that reported low scores reflect flawed evaluations rather than model limitations. After expert review corrected errors in reference solutions and problematic questions, GPT-5.6-Sol's performance improved dramatically, suggesting current benchmarks substantially underestimate these models' physics reasoning abilities.
By Ansari; Ali; Sun; Haoran; Liu; Andy Zeyi; Jabbour; Mark; Ding; Yongshan; Girvin; Steven; Yu; Ismail-Beigi; Sohrab; Kubica; Aleksander; Miller; Owen D; O'Hern; Corey; Ozolins; Vidvuds; Poland; David; Stone; A Douglas; Bosch; Frank C van den; Wright; Logan; Akbari; Navid; Antu; Santanu; Cai; Kangle; Calabrese-Day; Andrew; Wuttig; Mateo Cárdenes; Cheng; Meng; Chiang; Barry T; Ghorashi; Gu; Shouzhen; Huang; Haoyang; Zhibo; Kienesberger; Lukas; Hantian; Lomba; Charles; Zhongling; Wenchao; McIntosh; Rohin E; McKinney; Evan; Rojkov; Ivan; Xulei; Tokayer; Yarone Meir; Umasankar; Naveen Balaji; Varma; Mira; Wang; Leda; Qimin; Tyler; Wei; Haoyu; Yang; Jinming; Zhao; Jinchen; Sherlock Tingrui; Zheng; Qinyuan; Zou; Jay S; Baker; Lucas; Cohan; Arman; Sous; John