计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (4): 734-759.DOI: 10.3778/j.issn.1673-9418.2108086
许嘉1,2,3,+(), 韦婷婷1, 于戈4, 黄欣悦1, 吕品1,2,3
收稿日期:
2021-08-23
修回日期:
2021-11-24
出版日期:
2022-04-01
发布日期:
2021-12-01
通讯作者:
+ E-mail: xujia@gxu.edu.cn作者简介:
许嘉(1984—),女,山东荣成人,博士,副教授,硕士生导师,CCF高级会员,CCF数据库专委会委员,主要研究方向为数据库理论与技术、教育数据分析挖掘等。基金资助:
XU Jia1,2,3,+(), WEI Tingting1, YU Ge4, HUANG Xinyue1, LYU Pin1,2,3
Received:
2021-08-23
Revised:
2021-11-24
Online:
2022-04-01
Published:
2021-12-01
About author:
XU Jia, born in 1984, Ph.D., associate professor, M.S. supervisor, senior member of CCF, member of CCF Database Committee. Her research interests include database theory and technology, educational data analysis and mining, etc.Supported by:
摘要:
题目难度是保证试卷合理性及考试公平性的关键信息,也是智能教学系统(ITS)中的关键参数,有效支撑着包括智能组卷、题目自动生成和个性化习题推荐在内的多项智能教学功能。因此,题目难度评估已成为教育数据挖掘领域的一个重要研究方向,拥有大量研究工作。全面回顾了近十年题目难度评估研究领域的研究进展,将题目难度分为题目绝对难度和题目相对难度两类,并对现有的题目难度评估方法进行了整理和分类,其中重点分析了基于深度学习的题目绝对难度预测方法和基于深度学习的题目相对难度预测方法,并对后者包含的重要方法进行了实验分析。同时,对题目难度预测的相关数据集和模型评价指标等进行了总结。最后,对题目难度评估的未来研究方向进行了展望。
中图分类号:
许嘉, 韦婷婷, 于戈, 黄欣悦, 吕品. 题目难度评估方法研究综述[J]. 计算机科学与探索, 2022, 16(4): 734-759.
XU Jia, WEI Tingting, YU Ge, HUANG Xinyue, LYU Pin. Review of Question Difficulty Evaluation Approaches[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 734-759.
模型 | 文献 |
---|---|
回归分析 | [21,24-25,45-47,49-52,54-56] |
支持向量机 | [24-25,42,49-53,57] |
决策树 | [25,44,47,49,52-53] |
随机森林 | [24-25,43,49-52] |
浅层BP神经网络 | [41,53] |
表1 题目绝对难度预测常用的机器学习模型
Table1 Frequently-used machine learning models for question absolute difficulty prediction
模型 | 文献 |
---|---|
回归分析 | [21,24-25,45-47,49-52,54-56] |
支持向量机 | [24-25,42,49-53,57] |
决策树 | [25,44,47,49,52-53] |
随机森林 | [24-25,43,49-52] |
浅层BP神经网络 | [41,53] |
模型/框架简称 | 年份 | 使用模型 | 方法简单描述 | 优点 | 局限性和使用场景 |
---|---|---|---|---|---|
TACNN[ | 2017 | CNN | 基于CNN提出了一种预测英语考试中阅读理解题难度的方法 | 引入注意力机制限定阅读材料中不同语句对题目的贡献 | 只利用了题目文本的语义信息,只适用于题目文本信息较为丰富的阅读题 |
C-MIDP、 R-MIDP、 H-MIDP[ | 2019 | CNN、LSTM | 提出了基于CNN和RNN的数学题目难度预测模型C-MIDP和R-MIDP,以及二者的混合模型H-MIDP | 利用了题目文本的语义特征,并且考虑了题目的序列语义和逻辑信息 | 只利用题目中的文本信息,无法处理题目中的图片等信息,适用于题目文本信息较为丰富的阅读题 |
TCN-DPN[ | 2019 | LSTM | 利用深度学习技术提取中文阅读理解题的特征,特征提取阶段训练模型利用了数据量较大的语料库 | 无需手动提取文本特征,利用了题目文本的语义信息 | 只利用了题目文本信息,且只适用于题目文本信息较为丰富的阅读题 |
DAN[ | 2019 | CNN、 Bi-LSTM | 基于文档增强注意的神经网络提出了针对医学考试中选择题的难度预测模型 | 构建相关的文档数据库用于丰富输入,框架的召回模块和困惑模块能模拟学生的答题行为 | 只适用于可建立扩展文档数据库的选择题,且无法处理带图片的选择题 |
BEDP[ | 2019 | Capsule neural network | 提出一个融合深度多模态嵌入模型和贝叶斯推理的题目难度预测框架,深度多模态嵌入模型获得题目的统一多模态表示,分类器预测题目绝对难度 | 能处理包含图片的题目,同时利用题目中的文本特征和图像特征 | 能处理的题目类型有限,如无法提取编程题代码中的逻辑结构信息,适用于带有图片的题目 |
表2 基于深度学习的题目绝对难度预测模型对比
Table 2 Comparison of deep learning based question absolute difficulty prediction models
模型/框架简称 | 年份 | 使用模型 | 方法简单描述 | 优点 | 局限性和使用场景 |
---|---|---|---|---|---|
TACNN[ | 2017 | CNN | 基于CNN提出了一种预测英语考试中阅读理解题难度的方法 | 引入注意力机制限定阅读材料中不同语句对题目的贡献 | 只利用了题目文本的语义信息,只适用于题目文本信息较为丰富的阅读题 |
C-MIDP、 R-MIDP、 H-MIDP[ | 2019 | CNN、LSTM | 提出了基于CNN和RNN的数学题目难度预测模型C-MIDP和R-MIDP,以及二者的混合模型H-MIDP | 利用了题目文本的语义特征,并且考虑了题目的序列语义和逻辑信息 | 只利用题目中的文本信息,无法处理题目中的图片等信息,适用于题目文本信息较为丰富的阅读题 |
TCN-DPN[ | 2019 | LSTM | 利用深度学习技术提取中文阅读理解题的特征,特征提取阶段训练模型利用了数据量较大的语料库 | 无需手动提取文本特征,利用了题目文本的语义信息 | 只利用了题目文本信息,且只适用于题目文本信息较为丰富的阅读题 |
DAN[ | 2019 | CNN、 Bi-LSTM | 基于文档增强注意的神经网络提出了针对医学考试中选择题的难度预测模型 | 构建相关的文档数据库用于丰富输入,框架的召回模块和困惑模块能模拟学生的答题行为 | 只适用于可建立扩展文档数据库的选择题,且无法处理带图片的选择题 |
BEDP[ | 2019 | Capsule neural network | 提出一个融合深度多模态嵌入模型和贝叶斯推理的题目难度预测框架,深度多模态嵌入模型获得题目的统一多模态表示,分类器预测题目绝对难度 | 能处理包含图片的题目,同时利用题目中的文本特征和图像特征 | 能处理的题目类型有限,如无法提取编程题代码中的逻辑结构信息,适用于带有图片的题目 |
文献 | 年份 | 方法简述 |
---|---|---|
DIRT[ | 2019 | 利用深度学习技术获取IRT所需参数,将参数输入IRT模型预测题目对于学生的相对难度 |
PMF-CD[ | 2017 | 将认知诊断模型和矩阵分解技术进行结合 |
FuzzyCDF[ | 2018 | 认知诊断模型结合模糊集理论、教育假设 |
NeuralCD[ | 2020 | 结合神经网络的认知诊断模型 |
表3 混合认知诊断
Table 3 Hybrid cognitive diagnostic
文献 | 年份 | 方法简述 |
---|---|---|
DIRT[ | 2019 | 利用深度学习技术获取IRT所需参数,将参数输入IRT模型预测题目对于学生的相对难度 |
PMF-CD[ | 2017 | 将认知诊断模型和矩阵分解技术进行结合 |
FuzzyCDF[ | 2018 | 认知诊断模型结合模糊集理论、教育假设 |
NeuralCD[ | 2020 | 结合神经网络的认知诊断模型 |
文献 | 年份 | 主要考虑的因素 | ||
---|---|---|---|---|
学生方面 | 题目方面 | 其他方面 | ||
KT-Forget、KT-Slip[ | 2011 | 遗忘因素、失误因素 | — | — |
KT-IDEM[ | 2011 | — | 题目绝对难度 | — |
Individualized BKT[ | 2013 | 学习率、知识点初始掌握程度 | — | — |
BKT-ST[ | 2014 | — | 题目相似性 | — |
EEG-KT和EEG-LRKT[ | 2014 | 心理状态 | — | — |
LF-KT[ | 2014 | 学生能力 | 题目绝对难度 | 结合潜在因子模型 |
DBN[ | 2014 | — | 知识点层次结构 | — |
KT&IRT[ | 2014 | 学生能力 | 题目绝对难度 | 结合IRT模型 |
FAST[ | 2014 | — | — | 允许将一般特征集成到该模型 |
KAT[ | 2014 | 学生行为特征 | — | — |
Spectral BKT[ | 2015 | 学习状态 | — | — |
Affective BKT[ | 2015 | 心理状态 | — | — |
Multi-Grained-BKT和Historical-BKT[ | 2016 | 遗忘因素、失误因素 | 知识点层次结构 | — |
BKT+FSA[ | 2016 | 遗忘、学生能力 | — | — |
Intervention-BKT[ | 2016 | — | — | 不同类型的教学干预 |
TLS-BKT[ | 2018 | 学习状态 | — | — |
TD-BKT[ | 2018 | — | — | 时间差异 |
MS-BKT[ | 2020 | 学习率、学习状态 | — | — |
表4 BKT扩展模型
Table 4 Extended models for BKT
文献 | 年份 | 主要考虑的因素 | ||
---|---|---|---|---|
学生方面 | 题目方面 | 其他方面 | ||
KT-Forget、KT-Slip[ | 2011 | 遗忘因素、失误因素 | — | — |
KT-IDEM[ | 2011 | — | 题目绝对难度 | — |
Individualized BKT[ | 2013 | 学习率、知识点初始掌握程度 | — | — |
BKT-ST[ | 2014 | — | 题目相似性 | — |
EEG-KT和EEG-LRKT[ | 2014 | 心理状态 | — | — |
LF-KT[ | 2014 | 学生能力 | 题目绝对难度 | 结合潜在因子模型 |
DBN[ | 2014 | — | 知识点层次结构 | — |
KT&IRT[ | 2014 | 学生能力 | 题目绝对难度 | 结合IRT模型 |
FAST[ | 2014 | — | — | 允许将一般特征集成到该模型 |
KAT[ | 2014 | 学生行为特征 | — | — |
Spectral BKT[ | 2015 | 学习状态 | — | — |
Affective BKT[ | 2015 | 心理状态 | — | — |
Multi-Grained-BKT和Historical-BKT[ | 2016 | 遗忘因素、失误因素 | 知识点层次结构 | — |
BKT+FSA[ | 2016 | 遗忘、学生能力 | — | — |
Intervention-BKT[ | 2016 | — | — | 不同类型的教学干预 |
TLS-BKT[ | 2018 | 学习状态 | — | — |
TD-BKT[ | 2018 | — | — | 时间差异 |
MS-BKT[ | 2020 | 学习率、学习状态 | — | — |
文献 | 年份 | 主要改进类型 | ||
---|---|---|---|---|
除学生答题交互序列外的模型输入数据 | 引入注意力机制 | 模型其他特性 | ||
DKT-FE[ | 2017 | 学生答题次数、请求提示的次数、答题时间等 | 否 | — |
DKT-DT[ | 2017 | 多种异构特征,如学生答题尝试次数、请求提示的次数等 | 否 | — |
NKT[ | 2017 | — | 否 | 针对DKT的长期依赖问题提出双层堆叠LSTM |
Classifier-based DKT[ | 2018 | 异构特征,如学生答题次数、请求提示次数、答题时间等 | 否 | — |
EERNN[ | 2018 | 题目文本信息 | 是 | — |
DKT-DSC[ | 2018 | 学生能力 | 否 | — |
PDKT-C[ | 2018 | 题目和知识点之间的关系、知识点之间的先决关系 | 否 | 利用 |
DKT+[ | 2018 | — | 否 | 在损失函数中引入正则化项 |
E2E-DKT[ | 2018 | 题目和知识点之间的关系 | 否 | — |
DHKT[ | 2019 | 题目和知识点之间的关系 | 否 | 利用 |
DKT+Forgetting[ | 2019 | 学生遗忘行为 | 否 | — |
DKTS[ | 2019 | 题目之间的相似关系 | 否 | — |
KQN[ | 2019 | — | 否 | 基于向量点积的自解释模型 |
BDKT[ | 2019 | 题目知识点 | 否 | 综合贝叶斯网络和DKT模型 |
AKTHE[ | 2020 | 题目和题目属性之间的关系 | 是 | 利用异构信息网络描述题目和其绝对难度、区分度之间的关系 |
GIKT[ | 2020 | 题目和知识点之间的关系 | 是 | 利用图卷积网络学习题目和知识点之间关系的嵌入 |
DynEmb[ | 2020 | 与答题相关的其他信息,如答题时间戳、知识点、题目文本等 | 否 | 利用矩阵分解技术获取题目的潜在嵌入 |
qDKT[ | 2020 | 题目相似性 | 否 | 将题目差异正则化,作为额外的损失函数,并提出一种初始化嵌入矩阵的新方法 |
A-DKT[ | 2020 | 题目相似性 | 是 | — |
EHFKT[ | 2020 | 题目的语义、知识点和绝对难度 | 否 | — |
TC-MIRT[ | 2021 | 学生答题的时间间隔 | 否 | 将项目反应理论的参数集成到改进的RNN中,增强模型的可解释性 |
表5 DKT模型的扩展模型
Table 5 Extended models of DKT model
文献 | 年份 | 主要改进类型 | ||
---|---|---|---|---|
除学生答题交互序列外的模型输入数据 | 引入注意力机制 | 模型其他特性 | ||
DKT-FE[ | 2017 | 学生答题次数、请求提示的次数、答题时间等 | 否 | — |
DKT-DT[ | 2017 | 多种异构特征,如学生答题尝试次数、请求提示的次数等 | 否 | — |
NKT[ | 2017 | — | 否 | 针对DKT的长期依赖问题提出双层堆叠LSTM |
Classifier-based DKT[ | 2018 | 异构特征,如学生答题次数、请求提示次数、答题时间等 | 否 | — |
EERNN[ | 2018 | 题目文本信息 | 是 | — |
DKT-DSC[ | 2018 | 学生能力 | 否 | — |
PDKT-C[ | 2018 | 题目和知识点之间的关系、知识点之间的先决关系 | 否 | 利用 |
DKT+[ | 2018 | — | 否 | 在损失函数中引入正则化项 |
E2E-DKT[ | 2018 | 题目和知识点之间的关系 | 否 | — |
DHKT[ | 2019 | 题目和知识点之间的关系 | 否 | 利用 |
DKT+Forgetting[ | 2019 | 学生遗忘行为 | 否 | — |
DKTS[ | 2019 | 题目之间的相似关系 | 否 | — |
KQN[ | 2019 | — | 否 | 基于向量点积的自解释模型 |
BDKT[ | 2019 | 题目知识点 | 否 | 综合贝叶斯网络和DKT模型 |
AKTHE[ | 2020 | 题目和题目属性之间的关系 | 是 | 利用异构信息网络描述题目和其绝对难度、区分度之间的关系 |
GIKT[ | 2020 | 题目和知识点之间的关系 | 是 | 利用图卷积网络学习题目和知识点之间关系的嵌入 |
DynEmb[ | 2020 | 与答题相关的其他信息,如答题时间戳、知识点、题目文本等 | 否 | 利用矩阵分解技术获取题目的潜在嵌入 |
qDKT[ | 2020 | 题目相似性 | 否 | 将题目差异正则化,作为额外的损失函数,并提出一种初始化嵌入矩阵的新方法 |
A-DKT[ | 2020 | 题目相似性 | 是 | — |
EHFKT[ | 2020 | 题目的语义、知识点和绝对难度 | 否 | — |
TC-MIRT[ | 2021 | 学生答题的时间间隔 | 否 | 将项目反应理论的参数集成到改进的RNN中,增强模型的可解释性 |
文献 | 年份 | 主要改进类型 | |
---|---|---|---|
除学生答题交互序列外的模型输入数据 | 模型整合 | ||
Colearn[ | 2018 | 学生获取答案提示的行为 | — |
EKT[ | 2019 | 题目文本、知识点 | 整合DKVMN模型和EERNN模型 |
DSCMN[ | 2019 | 学生能力 | 整合DKVMN模型和DKT-DSC模型,将学生按能力分组进行训练以隐式添加学生学习能力特征 |
SKVMN[ | 2019 | — | 整合DKVMN模型和DKT模型 |
DKVMN-CA[ | 2019 | 考虑了学生和题目方面的多个特征(包括题目文本、知识点、学生学习阶段、答题时间),并设计了基于课程概念表的存储结构 | — |
DKVMN-DT[ | 2019 | 利用决策树对学生答题行为特征进行预处理,学生答题行为特征和答题交互一起作为模型的输入 | — |
Deep-IRT[ | 2019 | — | 整合DKVMN模型和IRT理论,提高模型的可解释性 |
LFKT[ | 2021 | 考虑影响学生知识遗忘的四个因素:学生重复学习知识点的间隔时间、重复学习知识点的次数、顺序学习间隔时间以及学生对知识点的掌握程度 | — |
DKVMN-LA[ | 2021 | 学生学习能力、答题行为特征 | — |
表6 DKVMN模型的扩展模型
Table 6 Extended models of DKVMN model
文献 | 年份 | 主要改进类型 | |
---|---|---|---|
除学生答题交互序列外的模型输入数据 | 模型整合 | ||
Colearn[ | 2018 | 学生获取答案提示的行为 | — |
EKT[ | 2019 | 题目文本、知识点 | 整合DKVMN模型和EERNN模型 |
DSCMN[ | 2019 | 学生能力 | 整合DKVMN模型和DKT-DSC模型,将学生按能力分组进行训练以隐式添加学生学习能力特征 |
SKVMN[ | 2019 | — | 整合DKVMN模型和DKT模型 |
DKVMN-CA[ | 2019 | 考虑了学生和题目方面的多个特征(包括题目文本、知识点、学生学习阶段、答题时间),并设计了基于课程概念表的存储结构 | — |
DKVMN-DT[ | 2019 | 利用决策树对学生答题行为特征进行预处理,学生答题行为特征和答题交互一起作为模型的输入 | — |
Deep-IRT[ | 2019 | — | 整合DKVMN模型和IRT理论,提高模型的可解释性 |
LFKT[ | 2021 | 考虑影响学生知识遗忘的四个因素:学生重复学习知识点的间隔时间、重复学习知识点的次数、顺序学习间隔时间以及学生对知识点的掌握程度 | — |
DKVMN-LA[ | 2021 | 学生学习能力、答题行为特征 | — |
模型 | 年份 | 优点 | 局限性 |
---|---|---|---|
SAKT[ | 2019 | 能较好地处理数据稀疏性问题,即学生答题交互序列不多的情况 | 无法指出学生在具体知识点上的掌握程度;未考虑学生答题过程中的遗忘行为;模型的注意力层过浅无法捕捉题目和答题结果之间的复杂关系 |
SAINT[ | 2020 | 相较于SAKT模型,能够较好地捕捉题目和答题结果之间的复杂关系 | 未考虑知识点间的关系;无法指出学生在具体知识点上的掌握程度;未考虑学生答题过程中的遗忘行为 |
DKTT[ | 2020 | 模型不但能自动识别题目中的潜在知识点,还模拟了学生的遗忘行为 | 无法处理知识点之间的先决关系对学生答题结果的影响 |
表7 基于Transformers的知识追踪模型总结
Table 7 Summary of knowledge tracking models based on Transformers
模型 | 年份 | 优点 | 局限性 |
---|---|---|---|
SAKT[ | 2019 | 能较好地处理数据稀疏性问题,即学生答题交互序列不多的情况 | 无法指出学生在具体知识点上的掌握程度;未考虑学生答题过程中的遗忘行为;模型的注意力层过浅无法捕捉题目和答题结果之间的复杂关系 |
SAINT[ | 2020 | 相较于SAKT模型,能够较好地捕捉题目和答题结果之间的复杂关系 | 未考虑知识点间的关系;无法指出学生在具体知识点上的掌握程度;未考虑学生答题过程中的遗忘行为 |
DKTT[ | 2020 | 模型不但能自动识别题目中的潜在知识点,还模拟了学生的遗忘行为 | 无法处理知识点之间的先决关系对学生答题结果的影响 |
模型大类 | 时间 | 初始模型 | 表征学生知识状态 | 有开源代码 | 优点 | 局限性 |
---|---|---|---|---|---|---|
DKT[ | 2015 | RNN/LSTM | 单个隐藏向量 | 是 | 无需专家标注题目中的知识点,相较于BKT模型更适合应用于在线教育环境 | 模型输入较简单;使用单个隐藏层表示学生的知识状态,无法指出学生不同知识点上的掌握程度;难以捕捉长依赖关系;不适用于处理稀疏数据 |
DKVMN[ | 2017 | MANN | 每个潜在知识点对应一个知识点状态向量 | 是 | 显性维护知识点矩阵和对应的知识状态矩阵,增强了模型的可解释性;外部存储量较大,因此参数较DKT模型更少 | 模型输入较为简单;过于依赖模型本身的遗忘机制;不适用于处理稀疏数据 |
SAKT[ | 2019 | Transformers | 无 | 否 | 能较好地处理数据稀疏性问题;模型适合并行计算,训练速度较短;与RNN相比更能捕获长期依赖关系 | 无法具体指出学生在某个知识点上的掌握程度;无法处理题目知识点之间先决关系对学生答题结果的影响 |
GKT[ | 2019 | GNN | 每个知识点对应一个知识点状态向量 | 是 | 解决了无法对知识点之间的复杂关系进行建模的问题;自动提取知识点之间的关系提高了模型的可解释性 | 只考虑与题目知识点的邻近节点,并未考虑远程节点的影响 |
CKT[ | 2020 | CNN | 学生知识状态矩阵 | 是 | 对学习过程中的个性化进行建模;自动学习有意义的题目嵌入 | 模型未考虑学生答题过程中的遗忘行为 |
表8 DKT、DKVMN、SAKT、GKT和CKT模型对比
Table 8 Comparison of DKT, DKVMN, SAKT, GKT and CKT model
模型大类 | 时间 | 初始模型 | 表征学生知识状态 | 有开源代码 | 优点 | 局限性 |
---|---|---|---|---|---|---|
DKT[ | 2015 | RNN/LSTM | 单个隐藏向量 | 是 | 无需专家标注题目中的知识点,相较于BKT模型更适合应用于在线教育环境 | 模型输入较简单;使用单个隐藏层表示学生的知识状态,无法指出学生不同知识点上的掌握程度;难以捕捉长依赖关系;不适用于处理稀疏数据 |
DKVMN[ | 2017 | MANN | 每个潜在知识点对应一个知识点状态向量 | 是 | 显性维护知识点矩阵和对应的知识状态矩阵,增强了模型的可解释性;外部存储量较大,因此参数较DKT模型更少 | 模型输入较为简单;过于依赖模型本身的遗忘机制;不适用于处理稀疏数据 |
SAKT[ | 2019 | Transformers | 无 | 否 | 能较好地处理数据稀疏性问题;模型适合并行计算,训练速度较短;与RNN相比更能捕获长期依赖关系 | 无法具体指出学生在某个知识点上的掌握程度;无法处理题目知识点之间先决关系对学生答题结果的影响 |
GKT[ | 2019 | GNN | 每个知识点对应一个知识点状态向量 | 是 | 解决了无法对知识点之间的复杂关系进行建模的问题;自动提取知识点之间的关系提高了模型的可解释性 | 只考虑与题目知识点的邻近节点,并未考虑远程节点的影响 |
CKT[ | 2020 | CNN | 学生知识状态矩阵 | 是 | 对学习过程中的个性化进行建模;自动学习有意义的题目嵌入 | 模型未考虑学生答题过程中的遗忘行为 |
模型 | AUC | 训练时间/min |
---|---|---|
DKT[ | 0.804 0 | 97.02 |
DKVMN[ | 0.813 0 | 240.35 |
GKT[ | 0.671 9 | 487.50 |
CKT[ | 0.823 9 | 31.58 |
表9 重要深度知识追踪模型实验对比
Table 9 Experimental comparison of important deep knowledge tracking models
模型 | AUC | 训练时间/min |
---|---|---|
DKT[ | 0.804 0 | 97.02 |
DKVMN[ | 0.813 0 | 240.35 |
GKT[ | 0.671 9 | 487.50 |
CKT[ | 0.823 9 | 31.58 |
名称 | 特点简述 | 语言 | 链接 |
---|---|---|---|
Math | 高中生的两次数学考试,包括客观题和主观题 | 英文 | |
ASSISTments2009 | ASSISTments在线教学系统收集的2009—2010学年的小学数学题的答题记录数据,密度为0.06 | 英文 | |
ASSISTments2012 | ASSISTments在线教学系统收集的2012—2013学年的小学数学题的答题记录数据 | 英文 | |
ASSISTments2015 | ASSISTments在线教学系统收集的2014—2015学年的小学数学题的答题记录数据,密度为0.05 | 英文 | |
ASSISTment Challenges | 来自2017年ASSISTment教育数据挖掘挑战赛的数据,密度为0.81 | 英文 | |
Synthetic-5 | 模拟2 000名学生答题的数据集,该数据集中每个学生回答50道题,每道题中包含一个知识点,且数据集中包含题目绝对难度信息 | 英文 | |
Statics2011 | 数据来源某大学的工程静态课程 | 英文 | |
KDD Cup 2010 | 2010年KDD Cup比赛的开源数据集 | 英文 | |
AICFE-* | 该系列数据集共包含8个不同的数据集,分别为语文、数学、英语、物理、化学、生物、历史和地理共8个科目,数据收集时间持续近3年 | 中文 | |
EdNet | 数据来自英语教学系统Santa,是迄今为止最大的开源交互教育系统数据集 | 英文 | |
Junyi Academy | 数据来自中国台湾的均一教育平台,除EdNet外数据量最大的开源数据集 | 中文 | |
Slepemapy.cz | 数据来自一个用于练习地理学的在线系统 | 英文 | |
Anonymizeddata | 数据来自计算机编程挑战 | 英文 | |
NeurIPS 2020 Education Challenge | Eedi在NeurIPS会议上发起的预测建模挑战赛的数据,题型为选择题 | 英文 | |
Datashop | 最大的学习交互数据存储库,包含30个以上数据集,涉及多个学科的学习交互数据,数据时间跨度较大且各数据集的数据特征也不尽相同 | 英文、 中文等 | |
表10 学生交互序列公开数据集
Table 10 Public datasets of student interaction sequences
名称 | 特点简述 | 语言 | 链接 |
---|---|---|---|
Math | 高中生的两次数学考试,包括客观题和主观题 | 英文 | |
ASSISTments2009 | ASSISTments在线教学系统收集的2009—2010学年的小学数学题的答题记录数据,密度为0.06 | 英文 | |
ASSISTments2012 | ASSISTments在线教学系统收集的2012—2013学年的小学数学题的答题记录数据 | 英文 | |
ASSISTments2015 | ASSISTments在线教学系统收集的2014—2015学年的小学数学题的答题记录数据,密度为0.05 | 英文 | |
ASSISTment Challenges | 来自2017年ASSISTment教育数据挖掘挑战赛的数据,密度为0.81 | 英文 | |
Synthetic-5 | 模拟2 000名学生答题的数据集,该数据集中每个学生回答50道题,每道题中包含一个知识点,且数据集中包含题目绝对难度信息 | 英文 | |
Statics2011 | 数据来源某大学的工程静态课程 | 英文 | |
KDD Cup 2010 | 2010年KDD Cup比赛的开源数据集 | 英文 | |
AICFE-* | 该系列数据集共包含8个不同的数据集,分别为语文、数学、英语、物理、化学、生物、历史和地理共8个科目,数据收集时间持续近3年 | 中文 | |
EdNet | 数据来自英语教学系统Santa,是迄今为止最大的开源交互教育系统数据集 | 英文 | |
Junyi Academy | 数据来自中国台湾的均一教育平台,除EdNet外数据量最大的开源数据集 | 中文 | |
Slepemapy.cz | 数据来自一个用于练习地理学的在线系统 | 英文 | |
Anonymizeddata | 数据来自计算机编程挑战 | 英文 | |
NeurIPS 2020 Education Challenge | Eedi在NeurIPS会议上发起的预测建模挑战赛的数据,题型为选择题 | 英文 | |
Datashop | 最大的学习交互数据存储库,包含30个以上数据集,涉及多个学科的学习交互数据,数据时间跨度较大且各数据集的数据特征也不尽相同 | 英文、 中文等 | |
来源 | 文献 |
---|---|
专家评估 | [2,20,23,26-28,42,44,51,53-56,59,64] |
答题数据 | [1,21-24,41-42,45-46,49-50,56,58,62-63] |
表11 题目真实难度标签的来源
Table 11 Sources of true difficulty lables of questions
来源 | 文献 |
---|---|
专家评估 | [2,20,23,26-28,42,44,51,53-56,59,64] |
答题数据 | [1,21-24,41-42,45-46,49-50,56,58,62-63] |
[1] | PADÓ U. Question difficulty-how to estimate without nor-ming, how to use for automated grading[C]// Proceedings of the 2017 Workshop on Innovative Use of NLP for Building Educational Applications, Copenhagen, Sep 8, 2017. Strou-dsburg: ACL, 2017: 1-10. |
[2] | FANG J S, ZHAO W, JIA D Y. Exercise difficulty prediction in online education systems[C]// Proceedings of the 2019 In-ternational Conference on Data Mining Workshops, Beijing, Nov 8-11, 2019. Piscataway: IEEE, 2019: 311-317. |
[3] | HAN G Y, LI X Z. An intelligent test paper generation algo-rithm based on adjustment of overall difficulty degrees[J]. Applied Mechanics & Materials, 2013, 411-414:2879-2882. |
[4] | HU Z, XING C. An intelligent test paper generation method based on genetic particle swarm optimization[C]// Procee-dings of the 2016 Web Information Systems and Applica-tions Conference, Wuhan, Sep 23-25, 2016. Piscataway: IEEE, 2016: 188-194. |
[5] |
VINU E V, SREENIVASA K P. Automated generation of as-sessment tests from domain ontologies[J]. Semantic Web, 2017, 8(6):1023-1047.
DOI URL |
[6] | KHODEIR N A, ELAZHARY H, WANAS N, et al. Gene-rating story problems via controlled parameters in a web-based intelligent tutoring system[J]. The International Jour-nal of Information and Learning Technology, 2018, 35(3):199-216. |
[7] |
HUO Y J, WONG D F, NI L M, et al. Knowledge modeling via contextualized representations for LSTM-based perso-nalized exercise recommendation[J]. Information Sciences, 2020, 523:266-278.
DOI URL |
[8] | 朱天宇, 黄振亚, 陈恩红, 等. 基于认知诊断的个性化试题推荐方法[J]. 计算机学报, 2017, 40(1):176-191. |
ZHU T Y, HUANG Z Y, CHEN E H, et al. Cognitive diag-nosis based personalized question recommendation[J]. Chinese Journal of Computers, 2017, 40(1):176-191. | |
[9] |
DEVELLIS R F. Classical test theory[J]. Medical Care, 2006, 44(3):S50-9.
DOI URL |
[10] | HOLLAND P W, THAYER D T. An alternate definition of the ETS delta scale of item difficulty[J]. ETS Research Re-port Series, 1985(2): i-10. |
[11] | SUSANTI Y, TOKUNAGA T, NISHIKAWA H, et al. Cont-rolling item difficulty for automatic vocabulary question generation[J]. Research & Practice in Technology Enhanced Learning, 2017, 12(1):25. |
[12] | 佟威, 汪飞, 刘淇, 等. 数据驱动的数学试题难度预测[J]. 计算机研究与发展, 2019, 56(5):1007-1019. |
TONG W, WANG F, LIU Q, et al. Data driven prediction for the difficulty of mathematical items[J]. Journal of Com-puter Research and Development, 2019, 56(5):1007-1019. | |
[13] | ZHU G L, LIU W J, ZHANG S X. Designing personalized learning difficulty for online learners[C]// LNCS 6537: Procee-dings of the International Conference on Web-based Lear-ning, Shanghai, Dec 7-11, 2010. Berlin, Heidelberg: Springer, 2010: 264-275. |
[14] | TEUSNER R, HILLE T, HAGEDORN C. Aspects on fin-ding the optimal practical programming exercise for MOOCs[C]// Proceedings of the Frontiers in Education Con-ference, Indianapolis, Oct 18-21, 2017. Washington: IEEE Computer Society, 2017: 1-8. |
[15] |
GAN W B, SUN Y, PENG X, et al. Modeling learner’s dy-namic knowledge construction procedure and cognitive item difficulty for knowledge tracing[J]. Applied Intelligence, 2020, 50(11):3894-3912.
DOI URL |
[16] | 刘恒宇, 张天成, 武培文, 等. 知识追踪综述[J]. 华东师范大学学报(自然科学版), 2019(5):1-15. |
LIU H Y, ZHANG T C, WU P W, et al. A review of know-ledge tracking[J]. Journal of East China Normal University (Natural Sciences), 2019(5):1-15. | |
[17] | 张暖, 江波. 学习者知识追踪研究进展综述[J]. 计算机科学, 2021, 48(4):213-222. |
ZHANG N, JIANG B. Review progress of learner know-ledge tracing[J]. Computer Science, 2021, 48(4):213-222. | |
[18] | 胡学钢, 刘菲, 卜晨阳. 教育大数据中认知跟踪模型研究进展[J]. 计算机研究与发展, 2020, 57(12):2523-2546. |
HU X G, LIU F, BU C Y. Research advances on knowledge tracing models in educational big data[J]. Journal of Com-puter Research and Development, 2020, 57(12):2523-2546. | |
[19] | ALKHUZAEY S, GRASSO F, PAYNE T R, et al. A sys-tematic review of data-driven approaches to item difficulty prediction[C]// LNCS 12748: Proceedings of the 22nd International Conference on Artificial Intelligence in Education, Utrecht, Jun 14-18, 2021. Cham: Springer, 2021: 29-41. |
[20] | 刘铁园, 陈威, 常亮, 等. 基于深度学习的知识追踪研究进展[J]. 计算机研究与发展, 2022, 59(1):81-104. |
LIU T Y, CHEN W, CHANG L, et al. Research advances in the knowledge tracing based on deep learning[J]. Journal of Computer Research and Development, 2022, 59(1):81-104. | |
[21] |
CHOI I, MOON Y. Predicting the difficulty of EFL tests based on corpus linguistic features and expert judgment[J]. Language Assessment Quarterly, 2020, 17(1):18-42.
DOI URL |
[22] | QIU Z P, WU X, FAN W. Question difficulty prediction for multiple choice problems in medical exams[C]// Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, Nov 3-7, 2019. New York: ACM, 2019: 139-148. |
[23] | LIN L H, CHANG T H, HSU F Y. Automated prediction of item difficulty in reading comprehension using long short-term memory[C]// Proceedings of the 2019 International Conference on Asian Language Processing, Shanghai, Nov 15-17, 2019. Piscataway: IEEE, 2019: 132-135. |
[24] | YANEVA V, HA L A, BALDWIN P, et al. Predicting item survival for multiple choice questions in a high-stakes me-dical exam[C]// Proceedings of the 2020 Language Resources and Evaluation Conference, Marseille, May 11-16, 2020. Paris: European Language Resources Association, 2020: 6812-6818. |
[25] | BENEDETTO L, CAPPELLI A, TURRIN R, et al. R2DE: a NLP approach to estimating IRT parameters of newly ge-nerated questions[C]// Proceedings of the 10th International Conference on Learning Analytics and Knowledge, Frank-furt, Mar 23-27, 2020. New York: ACM, 2020: 412-421. |
[26] | PERIKOS I, GRIVOKOSTOPOULOU F, KOVAS K, et al. Automatic estimation of exercises’ difficulty levels in a tuto-ring system for teaching the conversion of natural language into first-order logic[J]. Expert Systems-The Journal of Know-ledge Engineering, 2016, 33(6):569-580. |
[27] | GRIVOKOSTOPOULOU F, PERIKOS I, HATZILYGEROUDIS I. Estimating the difficulty of exercises on search algori-thms using a neuro-fuzzy approach[C]// Proceedings of the 27th International Conference on Tools with Artificial Intel-ligence, Vietri sul Mare, Nov 9-11, 2015. Washington: IEEE Computer Society, 2015: 866-872. |
[28] | GRIVOKOSTOPOULOU F, HATZILYGEROUDIS I, PERIKOS I. Teaching assistance and automatic difficulty estimation in converting first order logic to clause form[J]. Artificial Inte-lligence Review, 2014, 42(3):347-367. |
[29] |
RUSCH T, LOWRY P B, MAIR P, et al. Breaking free from the limitations of classical test theory: developing and mea-suring information systems scales using item response theory[J]. Information & Management, 2016, 54(2):189-203.
DOI URL |
[30] |
CONEJO R, GUZMAN E, PEREZ-DE-LA-CRUZ J L, et al. An empirical study on the quantitative notion of task difficulty[J]. Expert Systems with Applications, 2014, 41(2):594-606.
DOI URL |
[31] | HERMI A, ACHOUR W. Difficulty, discrimination and cog-nitive level of microbiology exam questions in the faculty of medicine of tunis[J]. La Tunisie Medicale, 2015, 93(8):1-3. |
[32] | 涂冬波, 蔡艳, 高旭亮, 等. 高级认知诊断[M]. 北京: 北京师范大学出版社, 2019. |
TU D B, CAI Y, GAO X L, et al. Advanced cognitive diagnosis[M]. Beijing: Beijing Normal University Publi-shing Group, 2019. | |
[33] | 黄振亚. 面向个性化学习的数据挖掘方法与应用研究[D]. 合肥: 中国科学技术大学, 2020. |
HUANG Z Y. Data mining techniques and applications for personalized learning[D]. Hefei: University of Science and Technology of China, 2020. | |
[34] |
TORRE D L J. DINA model and parameter estimation: a di-dactic[J]. Journal of Educational and Behavioral Statistics, 2009, 34(1):115-130.
DOI URL |
[35] | FAN X. Item response theory and classical test theory: an empirical comparison of their item/person statistics[J]. Edu-cational & Psychological Measurement, 1998, 58(3):357-381. |
[36] | CHENG S, LIU Q, CHEN E H, et al. DIRT: deep learning enhanced item response theory for cognitive diagnosis[C]// Proceedings of the 2019 International Conference on Infor-mation and Knowledge Management, Beijing, Nov 3-7, 2019. New York: ACM, 2019: 2397-2400. |
[37] | WANG F, LIU Q, CHEN E H, et al. Neural cognitive diagnosis for intelligent education systems[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence, the 32nd Innovative Applications of Artificial Intelligence Con-ference, the 10th AAAI Symposium on Educational Adva-nces in Artificial Intelligence, New York, Feb 7-12, 2020. Menlo Park: AAAI, 2020: 6153-6161. |
[38] | LIU Q, WU R Z, CHEN E H, et al. Fuzzy cognitive dia-gnosis for modelling examinee performance[J]. ACM Tran-sactions on Intelligent Systems and Technology, 2018, 9(4):1-26. |
[39] | YUDELSON M V, KOEDINGER K R, GORDON G J. In-dividualized Bayesian knowledge tracing models[C]// LNCS 7926: Proceedings of the 16th International Conference on Artificial Intelligence in Education, Memphis, Jul 9-13, 2013. Berlin, Heidelberg: Springer, 2013: 171-180. |
[40] | 李晓光, 魏思齐, 张昕, 等. LFKT: 学习与遗忘融合的深度知识追踪模型[J]. 软件学报, 2021, 32(3):818-830. |
LI X GZ, WEI S Q, ZHANG X, et al. LFKT: deep know-ledge tracing model with learning and forgetting behavior merging[J]. Journal of Software, 2021, 32(3):818-830. | |
[41] | 付佩宣. 基于人工神经网络的C.TEST阅读理解题目难度的预测研究[J]. 华文教学与研究, 2014(4):71-78. |
FU P X. A prediction study on the difficulty of C.TEST rea-ding understanding question based on artificial neural network[J]. TCSOL Studies, 2014(4):71-78. | |
[42] | HSU F Y, LEE H M, CHANG T H, et al. Automated esti-mation of item difficulty for multiple-choice tests: an appli-cation of word embedding techniques[J]. Information Pro-cessing & Management, 2018, 54(6):969-984. |
[43] | LIU C C, JIN Z, YANG Z, et al. Automatically difficulty grading method of “instruction system” question bank based on knowledge tree[C]// Proceedings of the 2017 Genetic and Evolutionary Computation Conference Companion, Berlin, Jul 15-19, 2017. New York: ACM, 2017: 283-284. |
[44] | 陈荟慧, 熊杨帆, 蒋滔滔, 等. 基于在线测评系统的编程题目难度研究[J]. 现代计算机, 2018(9):26-30. |
CHEN H H, XIONG Y F, JIANG T T, et al. Research on pro-gramming exercises difficulty based on online measure-ment system[J]. Modern Computer, 2018(9):26-30. | |
[45] |
MASRI Y E, FERRARA S, FOLTZ P W, et al. Predicting item difficulty of science national curriculum tests: the case of key stage 2 assessments[J]. The Curriculum Journal, 2017, 28(1):59-82.
DOI URL |
[46] |
PANDAROVA I, SCHMIDT T, HARTIG J, et al. Predicting the difficulty of exercise items for dynamic difficulty adaptation in adaptive language tutoring[J]. International Journal of Artificial Intelligence in Education, 2019, 29(3):342-367.
DOI URL |
[47] | SANO M. Automated capturing of psycho-linguistic features in reading assessment text[C]// Proceedings of the Annual Meeting of National Council on Measurement in Education, Chicago, Apr 15-19, 2015. |
[48] | SANO M. Improvements in automated capturing of psycho-linguistic features in reading assessment text[C]// Proceedings of the Annual Meeting of National Council on Measurement in Education, Washington, Apr 7-11, 2016. |
[49] | LOUKINA A, YOON S Y, SAKANO J, et al. Textual complexity as a predictor of difficulty of listening items in language proficiency tests[C]// Proceedings of the 2016 Inter-national Conference on Computational Linguistics, Osaka, Dec 11-16, 2016. Stroudsburg: ACL, 2016: 3245-3253. |
[50] | HA L A, YANEVA V, BALDWIN P, et al. Predicting the dif-ficulty of multiple choice questions in a high-stakes medical exam[C]// Proceedings of the 14th Workshop on Innovative Use of NLP for Building Educational Applications, Florence, Aug 2, 2019. Stroudsburg: ACL, 2019: 11-20. |
[51] | DESAI T, MOLDOVAN D I. Towards predicting difficulty of reading comprehension questions[C]// Proceedings of the 32nd International Florida Artificial Intelligence Research Society Conference, Sarasota, May 19-22, 2019. Menlo Park: AAAI, 2019: 8-13. |
[52] | BENEDETTO L, CAPPELLI A, TURRIN R. et al. Intro-ducing a framework to assess newly created questions with natural language processing[C]// LNCS 12163: Proceedings of the 21st International Conference on Artificial Intelli-gence in Education, Ifrane, Jul 6-10, 2020. Cham: Springer, 2020: 43-54. |
[53] | HUTZLER D, DAVID E, AVIGAL M, et al. Learning methods for rating the difficulty of reading comprehension questions[C]// Proceedings of the 2014 International Conference on Software Science, Technology and Engineering, Ramat Gan, Jun 11-12, 2014. Piscataway: IEEE, 2014: 54-62. |
[54] | LUMLEY T, ROUTITSKY A, MENDELOVITS J, et al. A framework for predicting item difficulty in reading tests[R]. Vancouver: American Educational Research Association, 2012. |
[55] | SEYLER D, YAHYA M, BERBERICH K. Knowledge ques-tions from knowledge graphs[C]// Proceedings of the 2017 ACM SIGIR International Conference on Theory of Infor-mation Retrieval, Amsterdam, Oct 1-4, 2017. New York: ACM, 2017: 11-18. |
[56] | SUSANTI Y, NISHIKAWA H, TOKUNAGA T, et al. Item difficulty analysis of English vocabulary questions[C]// Pro-ceedings of the 8th International Conference on Computer Supported Education, Rome, Apr 21-23, 2016. Setubal: Sci-TePress, 2016: 267-274. |
[57] | BEINBORN L, ZESCH T, GURVYCH I. Candidate evaluation strategies for improved difficulty prediction of language tests[C]// Proceedings of the 10th Workshop on Innovative Use of NLP for Building Educational Applications, Denver, Jun 4, 2015. Stroudsburg: ACL, 2015: 1-11. |
[58] | HUANG Z Y, LIU Q, CHEN E H, et al. Question difficulty prediction for READING problems in standard tests[C]// Proceedings of the 31st AAAI Conference on Artificial Inte-lligence, San Francisco, Feb 4-9, 2017. Menlo Park: AAAI, 2017: 1352-1359. |
[59] | PERIKOS I, GRIVOKOSTOPOULOU F, HATZILYGEROUDIS I, et al. Difficulty estimator for converting natural language into first order logic[C]// Proceedings of the 3rd International Con-ference on Intelligent Decision Technologies, Piraeus, Jul 20-22, 2011. Berlin, Heidelberg: Springer, 2011: 135-144. |
[60] |
VENUGOPAL V E, KUMAR P S. Difficulty-level modeling of ontology-based factual questions[J]. Semantic Web, 2020, 11(6):1023-1036.
DOI URL |
[61] | BANERJEE S, RAO N J, RAMANATHAN C. Rubrics for assessment item difficulty in engineering courses[C]// Pro-ceedings of the Frontiers in Education Conference, El Paso, Oct 21-24, 2015. Washington: IEEE Computer Society, 2015: 1-8. |
[62] | ELNAFFAR S. Using software metrics to predict the diffi-culty of code writing questions[C]// Proceedings of the 2016 Global Engineering Education Conference, Abu Dhabi, Apr 10-13, 2016. Piscataway: IEEE, 2016: 513-518. |
[63] | ARYADOUST V. Predicting item difficulty in a language test with an adaptive neuro fuzzy inference system[C]// Procee-dings of the 2013 Workshop on Hybrid Intelligent Models & Applications, Singapore, Apr 16-19, 2013. Piscataway: IEEE, 2013: 43-50. |
[64] | GRIVOKOSTOPOULOU F, PERIKOS I, HATZILYGEROUDIS I. Difficulty estimation of exercises on tree-based search al-gorithms using neuro-fuzzy and neuro-symbolic approaches[C]// Proceedings of the 5th International Workshop CIMA-2015, Vietri sul Mare, Nov 9-11, 2015. Cham: Springer, 2015: 75-91. |
[65] | KOREN Y, BELL R, VOLINSKY C. Matrix factorization techniques for recommender systems[J]. Computer, 2009, 42(8):30-37. |
[66] | CHEN Y Y, LIU Q, HUANG Z Y, et al. Tracking know-ledge proficiency of students with educational priors[C]// Proceedings of the 2017 Conference on Information and Know-ledge Management, Singapore, Nov 6-10, 2017. New York: ACM, 2017: 989-998. |
[67] | SALAKHUTDINOV R, MNIH A. Probabilistic matrix fac-torization[C]// Proceedings of the Annual Conference on Neural Information Processing Systems, Vancouver, Dec 3-6, 2007. New York: Curran Associates, 2007: 1257-1264. |
[68] |
CORBETT A T, ANDERSON J R. Knowledge tracing: modeling the acquisition of procedural knowledge[J]. User Modeling and User-Adapted Interaction, 1995, 4(4):253-278.
DOI URL |
[69] | BHATT S P, ZHAO J J, THILLE C. Evaluating Bayesian knowledge tracing for estimating learner proficiency and guiding learner behavior[C]// Proceedings of the 2020 Con-ference on Learning @Scale, Aug 12-14, 2020. New York: ACM, 2020: 357-360. |
[70] | DOROUDI S, BRUNSKILL E. The misidentified identifia-bility problem of Bayesian knowledge tracing[C]// Procee-dings of the 10th International Conference on Educational Data Mining, Wuhan, Jun 25-28, 2017. Worcester: IEDMS, 2017: 143-149. |
[71] | DEMPSTER A P, LAIRD N M, RUBIN D B. Maximum likelihood from incomplete data via the EM algorithm[J]. Journal of the Royal Statistical Society, 1977, 39(1):1-38. |
[72] | QIU Y M, QI Y M, LU H Y, et al. Does time matter? Mode-ling the effect of time with Bayesian knowledge tracing[C]// Proceedings of the 2011 International Conference on Educa-tional Data Mining, Eindhoven, Jul 6-8, 2011. Worcester: IEDMS, 2011: 139-148. |
[73] | PARDOS Z A, HEFFERNAN N T. KT-IDEM: introducing item difficulty to the knowledge tracing model[C]// LNCS 6787: Proceedings of the 19th Conference on User Modeling, Adaptation, and Personalization, Girona, Jul 11-15, 2011. Berlin, Heidelberg: Springer, 2011: 243-254. |
[74] | HAWKINS W J, HEFFERNAN N T. Using similarity to the previous problem to improve Bayesian knowledge tracing[C]// Proceedings of the 2014 International Conference on Educational Data Mining, London, Jul 4-7, 2014. Worcester: IEDMS, 2014: 1-5. |
[75] | XU Y B, CHANG K M, YUAN Y R, et al. Using EEG in knowledge tracing[C]// Proceedings of the 2014 International Conference on Educational Data Mining, London, Jul 4-7, 2014. Worcester: IEDMS, 2014: 361-362. |
[76] | KHAJAH M, WING R W, LINDSEY R V, et al. Integrating latent-factor and knowledge-tracing models to predict indi-vidual differences in learning[C]// Proceedings of the 2014 International Conference on Educational Data Mining, Lon-don, Jul 4-7, 2014. Worcester: IEDMS, 2014: 99-106. |
[77] | KÄSER T, KLINGLER S, SCHWING A G, et al. Beyond knowledge tracing: modeling skill topologies with Bayesian networks[C]// LNCS 8474: Proceedings of the 12th International Conference on Intelligent Tutoring Systems, Honolulu, Jun 5-9, 2014. Cham: Springer, 2014: 188-198. |
[78] | KHAJAH M, HUANG Y, GONZALEZ-BRENES J P, et al. Integrating knowledge tracing and item response theory: a tale of two frameworks[C]// Proceedings of the 22nd Confe-rence on User Modeling, Adaptation, and Personalization, Aalborg, Jul 7-11, 2014. Cham: Springer, 2014: 7-15. |
[79] | HUANG Y, GONZALEZ-BRENES J P, BRUSILOVSKY P. General features in knowledge tracing to model multiple subskills, temporal item response theory, and expert knowl-edge[C]// Proceedings of the 7th International Conference on Educational Data Mining, London, Jul 4-7, 2014. Worcester: IEDMS, 2014: 84-91. |
[80] | SCHULTZ S E, ARROYO I. Tracing knowledge and enga-gement in parallel in an intelligent tutoring system[C]// Pro-ceedings of the 2014 International Conference on Educational Data Mining, London, Jul 4-7, 2014. Worcester: IEDMS, 2014: 312-315. |
[81] | FALAKMASIR M H, YUDELSON M, RITTER S, et al. Spectral Bayesian knowledge tracing[C]// Proceedings of the 2015 International Conference on Educational Data Mining, Madrid, Jun 26-29, 2015. Worcester: IEDMS, 2015: 360-363. |
[82] | SPAULDING S, BREAZEAL C. Affect and inference in Bayesian knowledge tracing with a robot tutor[C]// Procee-dings of the Annual International Conference on Human-Robot Interaction, Portland, Mar 2-5, 2015. New York: ACM, 2015: 219-220. |
[83] | WANG Z, ZHU J L, LI X, et al. Structured knowledge tracing models for student assessment on coursera[C]// Pro-ceedings of the 2016 Conference on Learning@Scale, Edin-burgh, Apr 25-26, 2016. New York: ACM, 2016: 209-212. |
[84] | KHAJAH M, LINDSEY R V, MOZER M C. How deep is knowledge tracing?[C]// Proceedings of the 2016 International Conference on Educational Data Mining, Raleigh, Jun 29-Jul 2, 2016. Worcester: IEDMS, 2016: 94-101. |
[85] | LIN C, CHI M. Intervention-BKT: incorporating instructional interventions into Bayesian knowledge tracing[C]// LNCS 9684: Proceedings of the 13th International Conference on Intelligent Tutoring Systems, Zagreb, Jun 7-10, 2016. Cham: Springer, 2016: 208-218. |
[86] |
ZHANG K, YAO Y Y. A three learning states Bayesian know-ledge tracing model[J]. Knowledge-Based Systems, 2018, 148:189-201.
DOI URL |
[87] |
ZHU J H, ZANG Y C, QIU H, et al. Integrating temporal information into knowledge tracing: a temporal difference approach[J]. IEEE Access, 2018, 6:27302-27312.
DOI URL |
[88] | AGARWAL D, BAKER R S. Dynamic knowledge tracing through data driven recency weights[C]// Proceedings of the 2020 International Conference on Educational Data Mining, Jul 10-13, 2020. Worcester: IEDMS, 2020: 1-5. |
[89] | PIECH C, SPENCER J, HUANG J, et al. Deep knowledge tracing[C]// Proceedings of the Annual Conference on Neural Information Processing Systems, Montreal, Dec 7-12, 2015. Red Hook: Curran Associates, 2015: 505-513. |
[90] | ZHANG L, XING X L, ZHAO S Y, et al. Incorporating rich features into deep knowledge tracing[C]// Proceedings of the 2017 Conference on Learning@Scale, Cambridge, Apr 20-21, 2017. New York: ACM, 2017: 169-172. |
[91] | CHEUNG L P, YANG H Q. Heterogeneous features integra-tion in deep knowledge tracing[C]// LNCS 10635: Proceedings of the 24th International Conference on Neural Information Processing, Guangzhou, Nov 14-18, 2017. Cham: Springer, 2017: 653-662. |
[92] | SHA L, HONG P Y. Neural knowledge tracing[C]// LNCS 10512: Proceedings of the 1st International Conference on Brain Function Assessment in Learning, Patras, Sep 24-25, 2017. Cham: Springer, 2017: 108-117. |
[93] | YANG H Q, CHEUNG L P. Implicit heterogeneous features embedding in deep knowledge tracing[J]. Cognitive Com-putation, 2018, 10(1):3-14. |
[94] | SU Y, LIU Q W, LIU Q, et al. Exercise-enhanced sequential modeling for student performance prediction[C]// Procee-dings of the 32nd AAAI Conference on Artificial Intelli-gence, the 30th Innovative Applications of Artificial Intelli-gence, and the 8th AAAI Symposium on Educational Adva-nces in Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 2435-2443. |
[95] | MINN S, YU Y, DESMARAIS M C, et al. Deep knowledge tracing and dynamic student classification for knowledge tracing[C]// Proceedings of the 2018 International Confe-rence on Data Mining, Singapore, Nov 17-20, 2018. Pisca-taway: IEEE, 2018: 1182-1187. |
[96] | CHEN P H, YU L, ZHENG V W, et al. Prerequisite-driven deep knowledge tracing[C]// Proceedings of the 2018 Interna-tional Conference on Data Mining, Singapore, Nov 17-20, 2018. Washington: IEEE Computer Society, 2018: 39-48. |
[97] | YEUNG C K, YEUNG D Y. Addressing two problems in deep knowledge tracing via prediction-consistent regulari-zation[C]// Proceedings of the 2018 Conference on Lear-ning@Scale, London, Jun 26-28, 2018. New York: ACM, 2018: 1-10. |
[98] | NAKAGAWA H, IWASAWA Y, MATSUO Y. End-to-end deep knowledge tracing by learning binary question-embedding[C]// Proceedings of the 2018 IEEE International Confe-rence on Data Mining Workshops, Singapore, Nov 17-20, 2018. Piscataway: IEEE, 2018: 334-342. |
[99] | WANG T Q, MA F L, GAO J. Deep hierarchical knowledge tracing[C]// Proceedings of the 12th International Confe-rence on Educational Data Mining, Montreal, Jul 2-5, 2019. Worcester: IEDMS, 2019: 1-4. |
[100] | NAGATANI K, ZHANG Q, SATO M, et al. Augmenting knowledge tracing by considering forgetting behavior[C]// Proceedings of the 2019 International Conference on World Wide Web, San Francisco, May 13-17, 2019. New York: ACM, 2019: 3101-3107. |
[101] | WANG Z W, FENG X Q, TANG J L, et al. Deep know-ledge tracing with side information[C]// LNCS 11626: Pro-ceedings of the 20th International Conference on Artificial Intelligence in Education, Chicago, Jun 25-29, 2019. Cham: Springer, 2019: 303-308. |
[102] | LEE J, YEUNG D Y. Knowledge query network for know-ledge tracing: how knowledge interacts with skills[C]// Pro-ceedings of the 2019 International Conference on Learning Analytics & Knowledge, Tempe, Mar 4-8, 2019. New York: ACM, 2019: 491-500. |
[103] | LI D H, JIA Y M, ZHOU J, et al. Deep knowledge tracing based on Bayesian neural network[C]// Proceedings of the 2019 International Conference on Intelligent and Interac-tive Systems and Applications, Bangkok, Jun 28-30, 2019. Cham: Springer, 2019: 29-37. |
[104] | ZHANG N, DU Y, DENG K, et al. Attention-based know-ledge tracing with heterogeneous information network em-bedding[C]// LNCS 12274: Proceedings of the 13th Inter-national Conference on Knowledge Science, Engineering and Management, Hangzhou, Aug 28-30, 2020. Cham: Springer, 2020: 95-103. |
[105] | YANG Y, SHEN J, QU Y, et al. GIKT: a graph-based inte-raction model for knowledge tracing[C]// LNCS 12457: Pro-ceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, Ghent, Sep 14-18, 2020. Cham: Springer, 2020: 299-315. |
[106] | XU L B, DAVENPORT M A. Dynamic knowledge embed-ding and tracing[C]// Proceedings of the 2020 International Conference on Educational Data Mining, Jul 10-13, 2020. Worcester: IEDMS, 2020: 1-7. |
[107] | SONKAR S, LAN A S, WATERS A E, et al. qDKT: question-centric deep knowledge tracing[C]// Proceedings of the 2020 International Conference on Educational Data Mining, Jul 10-13, 2020. Worcester: IEDMS, 2020: 1-6. |
[108] | LIU D, DAI H H, ZHANG Y P, et al. Deep knowledge trac-king based on attention mechanism for student perfor-mance prediction[C]// Proceedings of the 2020 International Conference on Computer Science and Educational Infor-matization, Xinxiang, Jun 12-14, 2020. Piscataway: IEEE, 2020: 95-98. |
[109] | TONG H S, ZHOU Y, WANG Z. Exercise hierarchical fea-ture enhanced knowledge tracing[C]// LNCS 12164: Procee-dings of the 2020 International Conference on Artificial Intelligence in Education, Ifrane, Jul 6-10, 2020. Cham: Springer, 2020: 324-328. |
[110] |
SU Y, CHENG Z Y, LUO P F, et al. Time-and-concept en-hanced deep multidimensional item response theory for in-terpretable knowledge tracing[J]. Knowledge-Based Systems, 2021, 218:106819.
DOI URL |
[111] | ZHANG J N, SHI X J, KING I, et al. Dynamic key-value memory networks for knowledge tracing[C]// Proceedings of the 2017 International Conference on World Wide Web, Perth, Apr 3-7, 2017. New York: ACM, 2017: 765-774. |
[112] |
GRAVES A, WAYNE G, REYNOLDS M, et al. Hybrid computing using a neural network with dynamic external memory[J]. Nature, 2016, 538(7626):471-476.
DOI URL |
[113] | RITWICK C, HARVINEET S, PRADEEP D, et al. Mode-ling hint-taking behavior and knowledge state of students with multi-task learning[C]// Proceedings of the 2018 Inter-national Conference on Educational Data Mining, Buffal, Jul 15-18, 2018. Worcester: IEDMS, 2018: 1-11. |
[114] |
LIU Q, HUANG Z, YIN Y, et al. EKT: exercise-aware know-ledge tracing for student performance prediction[J]. IEEE Transactions on Knowledge and Data Engineering, 2019, 33(1):100-115.
DOI URL |
[115] | MINN S, DESMARAIS M, ZHU F, et al. Dynamic student classification on memory networks for knowledge tracing[C]// LNCS 11440: Proceedings of the 2019 Pacific-Asia Conference on Knowledge Discovery and Data Mining, Macau, China, Apr 14-17, 2019. Cham: Springer, 2019: 163-174. |
[116] | ABDELRAHMAN G, WANG Q. Knowledge tracing with sequential key-value memory networks[C]// Proceedings of the 2019 International Conference on Research and Deve-lopment in Information Retrieval, Paris, Jul 21-25, 2019. New York: ACM, 2019: 175-184. |
[117] | AI F, CHEN Y, GUO Y. Concept-aware deep knowledge tracing and exercise recommendation in an online learning system[C]// Proceedings of the 12th International Conference on Educational Data Mining, Montréal, Jul 2-5, 2019. Wo-rcester: IEDMS, 2019: 1-6. |
[118] | SUN X, ZHAO X, MA Y, et al. Muti-behavior features based knowledge tracking using decision tree improved DKVMN[C]// Proceedings of the 2019 ACM Turing Celebration Con-ference, Chengdu, May 17-19, 2019. New York: ACM, 2019: 72. |
[119] | YEUNG C K. Deep-IRT: make deep learning based know-ledge tracing explainable using item response theory[C]// Proceedings of the 12th International Conference on Edu-cational Data Mining, Montréal, Jul 2-5, 2019. Worcester: IEDMS, 2019: 1-10. |
[120] | SUN X, ZHAO X, LI B, et al. Dynamic key-value memory networks with rich features for knowledge tracing[J]. IEEE Transactions on Cybernetics, 2021: 1-7. |
[121] | PANDEY S, KARYPIS G. A self-attentive model for know-ledge tracing[C]// Proceedings of the 2019 International Con-ference on Educational Data Mining, Montréal, Jul 2-5, 2019. Worcester: IEDMS, 2019: 1-6. |
[122] | CHOI Y, LEE Y, CHO J, et al. Towards an appropriate query, key, and value computation for knowledge tracing[C]// Pro-ceedings of the 2020 Conference on Learning@Scale, Aug 12-14, 2020. New York: ACM, 2020: 341-344. |
[123] | PU S, YUDELSON M, OU L, et al. Deep knowledge tra-cing with transformers[C]// LNCS 12164: Proceedings of the International Conference on Artificial Intelligence in Educa-tion, Ifrane, Jul 6-10, 2020. Cham: Springer, 2020: 252-256. |
[124] | NAKAGAWA H, IWASAWA Y, MATSUO Y. Graph-based knowledge tracing: modeling student proficiency using graph neural network[C]// Proceedings of the 2019 ACM Interna-tional Conference on Web Intelligence, Thessaloniki, Oct 14-17, 2019. New York: ACM, 2019: 156-163. |
[125] | SHEN S H, LIU Q, CHEN E H, et al. Convolutional know-ledge tracing: modeling individualization in student learning process[C]// Proceedings of the 2020 International Con-ference on Research and Development in Information Ret-rieval, Xi’an, Jul 25-30, 2020. New York: ACM, 2020: 1857-1860. |
[126] | WU J Z, HUANG Z Y, LIU Q, et al. Federated deep know-ledge tracing[C]// Proceedings of the 2021 International Con-ference on Web Search and Data Mining, Israel, Mar 8-12, 2021. New York: ACM, 2021: 662-670. |
[127] | LIU S, ZOU R, SUN J W, et al. A hierarchical memory net-work for knowledge tracing[J]. Expert Systems with Appli-cations, 2021, 177:114935. |
[128] | VIE J J. Deep factorization machines for knowledge tracing[C]// Proceedings of the 2018 Workshop on Innovative Use of NLP for Building Educational Applications@NAACL-HLT, New Orleans, Jun 5, 2018. New York: ACM, 2018: 370-373. |
[129] | GUO H F, TANG R M, YE Y M, et al. DeepFM: a factori-zation-machine based neural network for CTR prediction[C]// Proceedings of the 2017 International Joint Confe-rence on Artificial Intelligence, Melbourne, Aug 19-25, 2017. Palo Alto: AAAI, 2017: 1725-1731. |
[130] | THAI-NGHE N, HORVATH T, SCHMIDT-THIEME L. Fac-torization models for forecasting student performance[C]// Proceedings of the 2011 International Conference on Edu-cational Data Mining, Eindhoven, Jul 6-8, 2011. Worcester: IEDMS, 2011: 11-20. |
[131] | LIU Y F, YANG Y, CHEN X Y, et al. Improving know-ledge tracing via pretraining question embeddings[C]// Proceedings of the 29th International Joint Conference on Artificial Intelligence, Yokohama, Jan 7-15, 2021. Palo Alto: AAAI, 2021: 1577-1583. |
[132] | HUANG Z Y, LIU Q, CHEN E H, et al. Learning or forget-ting? A dynamic approach for tracking the knowledge profi-ciency of students[J]. ACM Transactions on Information Systems, 2020, 38(2):1-33. |
[133] | VIE J J, KASHIMA H. Knowledge tracing machines: facto-rization machines for knowledge tracing[C]// Proceedings of the 2019 AAAI Conference on Artificial Intelligence, Honolulu, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 750-757. |
[1] | 安凤平, 李晓薇, 曹翔. 权重初始化-滑动窗口CNN的医学图像分类[J]. 计算机科学与探索, 2022, 16(8): 1885-1897. |
[2] | 曾凡智, 许露倩, 周燕, 周月霞, 廖俊玮. 面向智慧教育的知识追踪模型研究综述[J]. 计算机科学与探索, 2022, 16(8): 1742-1763. |
[3] | 刘艺, 李蒙蒙, 郑奇斌, 秦伟, 任小广. 视频目标跟踪算法综述[J]. 计算机科学与探索, 2022, 16(7): 1504-1515. |
[4] | 赵小明, 杨轶娇, 张石清. 面向深度学习的多模态情感识别研究进展[J]. 计算机科学与探索, 2022, 16(7): 1479-1503. |
[5] | 夏鸿斌, 肖奕飞, 刘渊. 融合自注意力机制的长文本生成对抗网络模型[J]. 计算机科学与探索, 2022, 16(7): 1603-1610. |
[6] | 谢欣彤, 胡悦阳, 刘譞哲, 赵耀帅, 姜海鸥. 传播用户代表性特征学习的谣言检测方法[J]. 计算机科学与探索, 2022, 16(6): 1334-1342. |
[7] | 孙方伟, 李承阳, 谢永强, 李忠博, 杨才东, 齐锦. 深度学习应用于遮挡目标检测算法综述[J]. 计算机科学与探索, 2022, 16(6): 1243-1259. |
[8] | 刘雅芬, 郑艺峰, 江铃燚, 李国和, 张文杰. 深度半监督学习中伪标签方法综述[J]. 计算机科学与探索, 2022, 16(6): 1279-1290. |
[9] | 程卫月, 张雪琴, 林克正, 李骜. 融合全局与局部特征的深度卷积神经网络算法[J]. 计算机科学与探索, 2022, 16(5): 1146-1154. |
[10] | 钟梦圆, 姜麟. 超分辨率图像重建算法综述[J]. 计算机科学与探索, 2022, 16(5): 972-990. |
[11] | 裴利沈, 赵雪专. 群体行为识别深度学习方法研究综述[J]. 计算机科学与探索, 2022, 16(4): 775-790. |
[12] | 朱伟杰, 陈莹. 双流时间域信息交互的微表情识别卷积网络[J]. 计算机科学与探索, 2022, 16(4): 950-958. |
[13] | 邬开俊, 黄涛, 王迪聪, 白晨帅, 陶小苗. 视频异常检测技术研究进展[J]. 计算机科学与探索, 2022, 16(3): 529-540. |
[14] | 刘利平, 孙建, 高世妍. 单图像盲去模糊方法概述[J]. 计算机科学与探索, 2022, 16(3): 552-564. |
[15] | 刘颖, 郭莹莹, 房杰, 范九伦, 郝羽, 刘继明. 深度学习跨模态图文检索研究综述[J]. 计算机科学与探索, 2022, 16(3): 489-511. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||