Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (4): 734-759.DOI: 10.3778/j.issn.1673-9418.2108086

• Surveys and Frontiers • Previous Articles     Next Articles

Review of Question Difficulty Evaluation Approaches

XU Jia1,2,3,+(), WEI Tingting1, YU Ge4, HUANG Xinyue1, LYU Pin1,2,3   

  1. 1. School of Computer Electronics and Information, Guangxi University, Nanning 530004, China
    2. Guangxi Key Laboratory of Multimedia Communications and Network Technology, Guangxi University, Nanning 530004, China
    3. Guangxi Colleges and University Key Laboratory of Parallel and Distributed Computing, Guangxi University, Nanning 530004, China
    4. School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China
  • Received:2021-08-23 Revised:2021-11-24 Online:2022-04-01 Published:2021-12-01
  • About author:XU Jia, born in 1984, Ph.D., associate professor, M.S. supervisor, senior member of CCF, member of CCF Database Committee. Her research interests include database theory and technology, educational data analysis and mining, etc.
    WEI Tingting, born in 1996, M.S. candidate, student member of CCF. Her research interests include exercise difficulty prediction and personalized recommendation.
    YU Ge, born in 1962, Ph.D., professor, Ph.D. supervisor, fellow of CCF. His research interests include database theory and technology, parallel and distributed computing, etc.
    HUANG Xinyue, born in 1997, M.S. candidate, student member of CCF. Her research interests include big data in education and data mining.
    LYU Pin, born in 1983, Ph.D., associate professor, M.S. supervisor, senior member of CCF, member of CCF Cooperative Computing. His research interests include Internet of things, educational big data analysis, etc.
  • Supported by:
    National Natural Science Foundation of China(62067001);National Natural Science Foundation of China(U1811261);Special Funds for Guangxi BaGui Scholars, the Projects of Higher Education Undergraduate Teaching Reform in Guangxi(2020JGA116);Special Funds for Guangxi BaGui Scholars, the Projects of Higher Education Undergraduate Teaching Reform in Guangxi(2017JGZ103);Innovation Project of Guangxi Graduate Education(JGY2021003);Natural Science Foundation of Guangxi(2019JJA170045)

题目难度评估方法研究综述

许嘉1,2,3,+(), 韦婷婷1, 于戈4, 黄欣悦1, 吕品1,2,3   

  1. 1.广西大学 计算机与电子信息学院,南宁 530004
    2.广西大学 广西多媒体通信网络技术重点实验室,南宁 530004
    3.广西大学 广西高校并行与分布式计算重点实验室,南宁 530004
    4.东北大学 计算机科学与工程学院,沈阳 110819
  • 通讯作者: + E-mail: xujia@gxu.edu.cn
  • 作者简介:许嘉(1984—),女,山东荣成人,博士,副教授,硕士生导师,CCF高级会员,CCF数据库专委会委员,主要研究方向为数据库理论与技术、教育数据分析挖掘等。
    韦婷婷(1996—),女,广西桂平人,硕士研究生,CCF学生会员,主要研究方向为习题难度预测、个性化推荐。
    于戈(1962—),男,辽宁大连人,博士,教授,博士生导师,CCF会士,主要研究方向为数据库理论与技术、并行分布式计算等。
    黄欣悦(1997—),女,广西人,硕士研究生,CCF学生会员,主要研究方向为教育大数据、数据挖掘。
    吕品(1983—),男,山东滨州人,博士,副研究员,硕士生导师,CCF高级会员,CCF协同计算专委会委员,主要研究方向为物联网、教育大数据分析等。
  • 基金资助:
    国家自然科学基金(62067001);国家自然科学基金(U1811261);“广西八桂学者”专项经费;广西高等教育本科教学改革工程项目(2020JGA116);“广西八桂学者”专项经费;广西高等教育本科教学改革工程项目(2017JGZ103);广西研究生教育创新计划资助项目(JGY2021003);广西自然科学基金(2019JJA170045)

Abstract:

Difficulty of a question is not only the key information to ensure the rationality of a test paper and the fairness of a test, but also acts as a critical parameter in intelligent tutoring system (ITS), effectively supporting many intelligent teaching functions, such as intelligent paper forming, automatic question generation, and perso-nalized exercise recommendation. Therefore, question difficulty evaluation has become an important research dire-ction in the field of educational data mining, and has a lot of research work. This paper comprehensively reviews the research progress of question difficulty evaluation in recent ten years, divides the question difficulty into two cate-gories: absolute difficulty and relative difficulty, sorts out and classifies exsiting evaluation approaches of question difficulty, and mainly explains deep learning based approaches for both question absolute difficulty prediction and question relative difficulty prediction. Specifically, important approaches of deep learning based question relative difficulty prediction are experimentally analyzed. Meanwhile, related datasets and evaluation metrics of question difficulty prediction approaches are summarized. Finally, the future research directions of question difficulty eva-luation are prospected.

Key words: question difficulty, difficulty evaluation, machine learning, deep learning, knowledge tracking, cogni-tive diagnosis

摘要:

题目难度是保证试卷合理性及考试公平性的关键信息,也是智能教学系统(ITS)中的关键参数,有效支撑着包括智能组卷、题目自动生成和个性化习题推荐在内的多项智能教学功能。因此,题目难度评估已成为教育数据挖掘领域的一个重要研究方向,拥有大量研究工作。全面回顾了近十年题目难度评估研究领域的研究进展,将题目难度分为题目绝对难度和题目相对难度两类,并对现有的题目难度评估方法进行了整理和分类,其中重点分析了基于深度学习的题目绝对难度预测方法和基于深度学习的题目相对难度预测方法,并对后者包含的重要方法进行了实验分析。同时,对题目难度预测的相关数据集和模型评价指标等进行了总结。最后,对题目难度评估的未来研究方向进行了展望。

关键词: 题目难度, 难度评估, 机器学习, 深度学习, 知识追踪, 认知诊断

CLC Number: