Groupwise Learning to Rank Algorithm with Introduction of Activated Weighting

doi:10.3778/j.issn.1673-9418.2101045

Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (7): 1594-1602.DOI: 10.3778/j.issn.1673-9418.2101045

• A.pngicial Intelligence • Previous Articles Next Articles

Groupwise Learning to Rank Algorithm with Introduction of Activated Weighting

LI Yuxuan¹^,²^,⁺(), HONG Xuehai¹^,³, WANG Yang¹, TANG Zhengzheng¹^,², BAN Yan¹

1. Computer Network Information Center, China Academy of Sciences, Beijing 100190, China
2. University of China Academy of Sciences, Beijing 100049, China
3. Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China

Received:2021-01-11 Revised:2021-03-16 Online:2022-07-01 Published:2021-04-09

引入激活加权策略的分组排序学习方法

李玉轩¹^,²^,⁺(), 洪学海¹^,³, 汪洋¹, 唐正正¹^,², 班艳¹

1.中国科学院计算机网络信息中心,北京 100190
2.中国科学院大学,北京 100049
3.中国科学院计算技术研究所,北京 100190

作者简介:李玉轩（1996—）,男,江苏宿迁人,硕士研究生,主要研究方向为机器学习、信息检索等。
LI Yuxuan, born in 1996, M.S. candidate. His research interests include machine learning, information retrieval, etc.
洪学海（1967—）,男,安徽巢湖人,博士,研究员,博士生导师,CCF杰出会员,主要研究方向为高性能计算、人工智能、大数据等。
HONG Xuehai, born in 1967, Ph.D., professor, Ph.D. supervisor, outstanding member of CCF. His research interests include high performance computing, a.pngicial intelligence, big data, etc.
汪洋（1984—）,男,湖北武汉人,博士,高级工程师,硕士生导师,中国科学院计算机网络信息中心信息化发展战略与评估中心主任,主要研究方向为大数据分析、态势感知系统、信息化发展战略研究等。
WANG Yang, born in 1984, Ph.D., senior engineer, M.S. supervisor. His research interests include big data analysis, situational awareness system, strategy research on informatization development, etc.
唐正正（1992—）,男,安徽怀远人,博士研究生,主要研究方向为机器学习、数据挖掘、图表示学习等。
TANG Zhengzheng, born in 1992, Ph.D. candidate. His research interests include machine learning, data mining, graph representation learning, etc.
班艳（1981—）,女,北京人,高级工程师,主要研究方向为信息化发展战略研究。
BAN Yan, born in 1981, senior engineer. Her research interest is strategy research on informatization development.

Abstract

Abstract:

Learning to rank (LtR) applies supervised machine learning (SML) technologies to the ranking problems, aiming at optimizing the relevance of input document list. As regard to previous studies on the deep ranking model, the calculation of the relevance of the documents in the list is independent of each other, which lacks consideration of document interactions. In recent years, some new methods are devoted to mining the interaction between documents, such as groupwise scoring function (GSF), which learns multivariate scoring function to jointly judge the correlation, but most of these methods ignore the differences of the interaction between documents, and bring high calculation cost at the same time. In order to solve this problem, this paper proposes a weighted groupwise deep ranking model (W-GSF). In view of the deep interest network in the field of recommendation, this paper intro-duces the idea of adjusting the weight of historical behavior sequence according to the candidate products. On the basis of multivariate scoring method in learning to rank field, this method uses muti-layer feed forword neural networks as main structure, and adds an activation unit into it before the input module, taking advantage of neural networks to adjust the weight of input multiple variables adaptively, so as to mine the differences of cross document relationship. Experiments on the public benchmark dataset MSLR verify the effectiveness of the method. Compared with baseline ranking models, the introduction of activation strategy brings a significant improvement of ranking metrics, and the computational complexity is greatly reduced compared with the same effect learning to rank methods.

Key words: learning to rank (LtR), groupwise scoring function (GSF), deep neural network, deep interest network

摘要：

排序学习（LtR）将有监督机器学习技术（SML）用于解决排序问题,旨在给出输入文档列表的相关度更优化的排序结果。此前关于深度排序模型的研究,对于列表内文档的相关度计算彼此独立,缺乏考虑文档之间的相互作用。近年来一些新方法致力于挖掘文档之间的相互影响,如分组评分法（GSF）,通过学习多元变量评分函数来联合判断文档相关性,但大多忽略了文档间相互影响的差异性,同时增加了很大的计算代价。针对此问题,提出了一种带权重的分组深度排序模型（W-GSF）。该方法借鉴推荐领域的深度兴趣网络,引入其根据候选商品调整历史行为序列权重的思想,在排序学习中多元评分法基础上,以多层前馈神经网络为主体结构,并在输入端加入激活单元,利用神经网络自适应学习调整输入的多元变量的权重,来挖掘交叉文档关系的差异性。在公共基准数据集MSLR上的实验验证了该方法的有效性,相比基线排序模型,激活策略的引入带来了排序指标上的明显提升,同时相对于同等效果的排序方法计算量大幅降低。

关键词: 排序学习（LtR）, 分组评分法（GSF）, 深度神经网络, 深度兴趣网络

CLC Number:

TP391

LI Yuxuan, HONG Xuehai, WANG Yang, TANG Zhengzheng, BAN Yan. Groupwise Learning to Rank Algorithm with Introduction of Activated Weighting[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1594-1602.

李玉轩, 洪学海, 汪洋, 唐正正, 班艳. 引入激活加权策略的分组排序学习方法[J]. 计算机科学与探索, 2022, 16(7): 1594-1602.

Figures/Tables 5

Fig.1 Structure of basic GSF model, with a sample which only has three documents

Fig.2 Structure of W-GSF and activation unit

Table 1 Baseline learning-to-rank models

方法	具体描述
RankingSVM	经典的基于支持向量机的文档对排序方法
LambdaMART	基于树的方法,善于处理稠密特征向量,多个公开数据集上的SOTA技术
RankNet	经典的文档对方法,基于神经网络模型,可以在损失函数上乘上 $λ$ -权重变成文档列表法
GSF-m	群组大小为 $m$ 的分组文档法

Table 1 Baseline learning-to-rank models

方法	具体描述
RankingSVM	经典的基于支持向量机的文档对排序方法
LambdaMART	基于树的方法,善于处理稠密特征向量,多个公开数据集上的SOTA技术
RankNet	经典的文档对方法,基于神经网络模型,可以在损失函数上乘上 $λ$ -权重变成文档列表法
GSF-m	群组大小为 $m$ 的分组文档法

Table 2 Comparison of models’ test NDCG on MSLR dataset %

Dataset	Model	NDCG@1	NDCG@3	NDCG@5	NDCG@10
Web10K	RankingSVM	36.22(±0.12)	—	34.91(±0.09)	31.07(±0.09)
	RankNet	39.74(±0.05)	—	40.94(±0.07)	43.37(±0.05)
	LambdaMART	46.20(±0.13)	—	46.23(±0.09)	48.33(±0.11)
	GSF-2	41.79(±0.14)	42.13(±0.09)	42.31(±0.11)	44.79(±0.09)
	GSF-64	42.75(±0.14)	42.61(±0.07)	42.65(±0.12)	44.87(±0.13)
	W-GSF	42.52(±0.09)	42.57(±0.09)	42.70(±0.09)	44.91(±0.09)
Web30K	RankingSVM	37.33(±0.09)	—	35.07(±0.13)	32.61(±0.17)
	RankNet	40.74(±0.11)	—	42.12(±0.05)	44.73(±0.06)
	LambdaMART	50.33(±0.07)	—	49.20(±0.09)	51.05(±0.11)
	GSF-2	43.37(±0.11)	43.41(±0.09)	43.69(±0.11)	46.22(±0.09)
	GSF-64	44.43(±0.09)	44.52(±0.13)	44.89(±0.12)	46.84(±0.10)
	W-GSF	44.41(±0.07)	44.54(±0.09)	44.77(±0.09)	46.88(±0.09)

Table 3 Comparison of calculational cost of W-GSF and GSF on MSLR-Web30K

Model	$Δ$ FLOPs	$Δ$ NDCG@10/%
GSF-2	—	—
GSF-64	×28.10	+1.34
W-GSF	×1.33	+1.43

Table 3 Comparison of calculational cost of W-GSF and GSF on MSLR-Web30K

Model	$Δ$ FLOPs	$Δ$ NDCG@10/%
GSF-2	—	—
GSF-64	×28.10	+1.34
W-GSF	×1.33	+1.43

References 22

[1]	LIU T Y. Learning to rank for information retrieval[C]// Pro-ceeding of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval,Geneva, Jul 19-23, 2010. New York: ACM, 2010: 904.
[2]	CHEN W, LIU T Y, LAN Y Y, et al. Ranking measures and loss functions in learning to rank[C]// Advances in Neural Information Processing Systems 22, Vancouver, Dec 7-10,2009. Red Hook: Curran Associates, 2009: 315-323.
[3]	BURGES C J C. From RankNet to LambdaRank to Lambda-MART: an overview: MSR-TR-2010-82[R]. 2010.
[4]	CAO Z, QIN T, LIU T Y, et al. Learning to rank: from pair-wise approach to listwise approach[C]// Proceedings of the 24th International Conference on Machine Learning, Cor-vallis, Jun 20-24, 2007. New York: ACM, 2007: 129-136.
[5]	曹军梅, 马乐荣. 卷积重提取特征的文档列表排序学习方法[J]. 中文信息学报, 2020, 34(8): 86-93.
	CAO J M, MA L R. Listwise reranking via convolutional re-extracted features[J]. Journal of Chinese Information Pro-cessing, 2020, 34(8): 86-93.
[6]	WANG R X, FANG K, ZHOU R K, et al. SERank: optimize sequencewise learning to rank using squeeze-and-excitation network[J]. arXiv: 2006. 04084, 2020.
[7]	AI Q Y, WANG X H, BRUCH S, et al. Learning groupwise multivariate scoring functions using deep neural networks[C]// Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, Santa Clara, Oct 2-5, 2019. New York: ACM, 2019: 85-92.
[8]	YE P, DOERMANN D. Combining preference and absolute judgements in a crowd-sourced setting[C]// Proceedings of the 2013 ICML Workshop on Machine Learning Meets Crowd-sourcing, Atlanta, Jun 16-21, 2013: 1-7.
[9]	JOACHIMS T, GRANKA L A, PAN B, et al. Accurately in-terpreting clickthrough data as implicit feedback[C]// Procee-dings of the 28th Annual International ACM SIGIR Con-ference on Research and Development in Information Re-trieval, Salvador, Aug 15-19, 2005. New York: ACM, 2005:154-161.
[10]	ZHOU G R, ZHU X Q, SONG C R, et al. Deep interest network for click-through rate prediction[C]// Proceedings of the 24th ACM SIGKDD International Conference on Know-ledge Discovery and Data Mining, London, Aug 19-23, 2018. New York: ACM, 2018: 1059-1068.
[11]	JOACHIMS T. Training linear SVMs in linear time[C]// Proceedings of the 12th ACM SIGKDD International Con-ference on Knowledge Discovery and Data Mining, Phila-delphia, Aug 20-23, 2006. New York: ACM, 2006: 217-226.
[12]	FRIEDMAN J H. Greedy function approximation: a gra-dient boosting machine[J]. Annals of Statistics, 2001, 29(5):1189-1232. DOI URL
[13]	JOACHIMS T. Optimizing search engines using clickthrough data[C]// Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Phila-delphia, Aug 20-23, 2006. New York: ACM, 2006: 133-142.
[14]	熊李艳, 陈晓霞, 钟茂生, 等. 基于PairWise排序学习算法研究综述[J]. 科学技术与工程, 2017, 17(21): 184-190.
	XIONG L Y, CHEN X X, ZHONG M S, et al. Survey on PairWise of the learning to rank[J]. Science Technology and Engineering, 2017, 17(21): 184-190.
[15]	TAYLOR M J, GUIVER J, ROBERTSON S, et al. SoftRank: optimizing non-smooth rank metrics[C]// Proceedings of the 2008 International Conference on Web Search and Web Data Mining, Palo Alto, Feb 11-12, 2008. New York: ACM, 2008: 77-86.
[16]	黄震华, 张佳雯, 田春岐, 等. 基于排序学习的推荐算法研究综述[J]. 软件学报, 2016, 27(3): 691-713.
	HUANG Z H, ZHANG J W, TIAN C Q, et al. Survey on learning-to-rank based recommendation algorithms[J]. Journal of Software, 2016, 27(3): 691-713.
[17]	AI Q Y, BI K P, GUO J F, et al. Learning a deep listwise context model for ranking refinement[C]// Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, Ann Arbor, Jul 8-12, 2018. New York: ACM, 2018: 135-144.
[18]	HE K M, ZHANG X Y, REN S Q, et al. Delving deep into re.pngiers: surpassing human-level performance on imagenet classification[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 1026-1034.
[19]	QIN T, LIU T Y. Introducing LETOR 4.0 datasets[J]. arXiv:1306. 2597, 2013.
[20]	JOACHIMS T. A support vector method for multivariate performance measures[C]// Proceedings of the 22nd Inter-national Conference on Machine Learning, Bonn, Aug 7-11, 2005. New York: ACM, 2005: 377-384.
[21]	KE G L, MENG Q, FINLEY T, et al. LightGBM: a highly efficient gradient boosting decision tree[C]// Advances in Neural Information Processing Systems 30, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 3146-3154.
[22]	BRUCH S, WANG X H, BENDERSKY M, et al. An analysis of the softmax cross entropy loss for learning-to-rank with binary relevance[C]// Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, Santa Clara, Oct 2-5, 2019. New York: ACM, 2019: 75-78.

Groupwise Learning to Rank Algorithm with Introduction of Activated Weighting

引入激活加权策略的分组排序学习方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 22

Related Articles 10

Recommended Articles

Metrics

[1]	PEI Lishen, ZHAO Xuezhuan. Survey of Collective Activity Recognition Based on Deep Learning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 775-790.
[2]	LIU Liping, QIAO Lele, JIANG Liucheng. Overview of Image Denoising Methods [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(8): 1418-1431.
[3]	WU Xiaodong, LIU Jinghao, JIN Jie, MAO Siping. DNN Intrusion Detection Model Based on DT and PCA [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(8): 1450-1458.
[4]	SHEN Xueli, QIN Xinyu. KNN Algorithm of Enhanced Clustering Based on Density Canopy and Deep Feature [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(7): 1289-1301.
[5]	XU Hui, ZHU Yuhua, ZHEN Tong, LI Zhihui. Survey of Image Semantic Segmentation Methods Based on Deep Neural Network [J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(1): 47-59.
[6]	LIN Yang, CHU Xu, WANG Yasha, MAO Weijia, ZHAO Junfeng. Cross-Modal Recipe Retrieval with Self-Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(9): 1471-1481.
[7]	LI Junjie, WANG Qian. Perceptually Similar Image Classification Adversarial Example Generation Model [J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(11): 1930-1942.
[8]	ZHANG Tao, REN Xiangying, LIU Yang, GENG Yanzhang. Acoustic Features Extraction of Speech Enhancement Based on Auto-Encoder Feature [J]. Journal of Frontiers of Computer Science and Technology, 2019, 13(8): 1341-1350.
[9]	XU Yi, DONG Qing, DAI Xin, SONG Wei. ELM Optimized Deep Autoencoder Classification Algorithm [J]. Journal of Frontiers of Computer Science and Technology, 2018, 12(5): 820-827.
[10]	ZHANG Wei, ZHOU Zhiping. Image Caption Generation Model with Visual Attention and Dynamic Semantic Information Guiding [J]. Journal of Frontiers of Computer Science and Technology, 2017, 11(12): 2033-2040.