FrontierPhysics benchmark launched for end-to-end AI physics research
A paper submitted to arXiv on 4 November 2026 introduces FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Scoring combines deterministic verifiers with expert-authored rubrics to judge whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors. The benchmark team, drawn from physics labs at Berkeley, Caltech and Harvard, said it recruited 42 physicists from 26 institutions; Artificial Analysis said it was excited to partner to host the evaluation on its platform soon.
Existing physics benchmarks evaluate only fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. FrontierPhysics instead tests whether agents can complete realistic, end-to-end physics research and produce results that are correct and publishable.
Computer Science > Artificial Intelligence
arXiv:2611.06910 (cs)
[Submitted on 4 Nov 2026]
Title:FrontierPhysics: Evaluating Agents for End-to-End Physics Research
Authors:Bingran You, Wenjun Ke, Ria Rosenauer, Bo Liu, Wenbo Chen, Min Chen, Shuqi Xu, Yue Ma, Jiechao Feng, Bo Chen, Xiangkai Sun, Jiarui Liu, Jinen Guo, Zeyu Zhou, Xu Huang, Zhuohan Li, Haoyu Li, Heguang Lin, Lijie Ding, Yadong Zeng, Kangqi Zhang, Xinge Liu, Hanwen Liu, Zhenghong Li, Alexander von Recum, Yuchen Zhu, Felix Schwarzfischer, Yizhou Zhao, Jicheng Wang, Binxu Li, Qian Yu, Chan Li, Cheng Dai, Di Wang, Haoyu Zhang, Jiankai Sun, Kaituo Feng, Mian Qin, Ruihong Yin, Yiqun Luo, Weiwei Wu, Shihong Pan, Tsung Yeh Hsieh, Xiao Huang, Yan Zhang, Sihan Li, Zirui Ren, Zengji Tu, Kyoung Whan Choe, Ke Sun, Vincenzo Galia, Vaisakhi Mishra, Jiashu He, Hongyang Wang, Zhiheng Yang, Shubham Gaur, Yifan Gu, Shuyi Wang, Zihao Zuo, Steven Dillmann, Yiyou Sun, Xinyang Han, Hanchung Lee, Molei Tao
View a PDF of the paper titled FrontierPhysics: Evaluating Agents for End-to-End Physics Research, by Bingran You and 71 other authors
Some source content is omitted. Continue reading on the original site.
#FrontierPhysics#Benchflow AI#Artificial Analysis#UC Berkeley#Caltech#Harvard University