Epi2Diff predicts human item difficulty from LLM reasoning traces
Researchers from Virginia Tech, the University of Maryland, MBZUAI and the University of Pittsburgh proposed Epi2Diff, a framework that splits LLM reasoning traces into cognitive episodes and combines them with item content to predict human difficulty. The research team reported first-place results across the main metrics for SAT Mathematics, SAT Reading and Writing, Cambridge and USMLE, with SAT Mathematics accuracy reaching 73.0%. Its paper was accepted by Transactions on Machine Learning Research.
Epi2Diff adds interpretable process signals—such as reasoning duration, analysis or execution emphasis and strategy switching—to item-difficulty prediction, offering test-bank teams additional evidence to complement human response data.
一道试题对人来说有多难，能否从大模型的解题过程中找到线索？
来自弗吉尼亚理工大学、马里兰大学、MBZUAI与匹兹堡大学的研究团队提出Epi2Diff：将大模型生成的推理轨迹拆分为“认知片段”，结合题目内容预测人类作答难度。
在SAT数学、SAT阅读与写作、Cambridge和USMLE四个真实测评数据集上，Epi2Diff在单次主实验的各项指标中均排名第一。其中，SAT数学难度分类准确率达到73.0%，比最强基线高3.2个百分点。
相关论文已被机器学习期刊Transactions on Machine Learning Research（TMLR）接收。
研究背景：题目语义之外，还有解题过程
判断新题难度是组卷和题库建设的基础工作。传统教育测量依赖考生作答数据标定，可靠但预试成本较高；仅分析题干语义，又难以判断解题过程究竟在哪里费力。
#Epi2Diff#Virginia Tech#University of Maryland#MBZUAI#University of Pittsburgh#QwQ-32B#Qwen3-32B#RoBERTa#Sentence-BERT#XGBoost