Survey of Sign Language Recognition and Translation

doi:10.3778/j.issn.1673-9418.2205003

Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (11): 2415-2429.DOI: 10.3778/j.issn.1673-9418.2205003

• Surveys and Frontiers • Previous Articles Next Articles

Survey of Sign Language Recognition and Translation

YAN Siyi¹, XUE Wanli¹^,⁺(), YUAN Tiantian²

1. School of Computer Science and Engineering, Tianjin University of Technology, Tianjin 300384, China
2. Technical College for the Deaf, Tianjin University of Technology, Tianjin 300384, China

Received:2022-05-05 Revised:2022-07-20 Online:2022-11-01 Published:2022-11-16
About author:YAN Siyi, born in 1996, M.S. candidate. Her research interests include sign language recognition and sign language translation.
XUE Wanli, born in 1986, Ph.D., member of CCF. His research interests include target tracking and sign language recognition.
YUAN Tiantian, born in 1980, Ph.D., professor, member of CCF. Her research interests include sign language recognition and computer network.
Supported by:
National Natural Science Foundation of China(61906135);National Natural Science Foundation of China(62020106004);National Natural Science Foundation of China(92048301)

手语识别与翻译综述

闫思伊¹, 薛万利¹^,⁺(), 袁甜甜²

1.天津理工大学计算机科学与工程学院，天津 300384
2.天津理工大学聋人工学院，天津 300384

通讯作者: + E-mail: xuewanli@email.tjut.edu.cn
作者简介:闫思伊（1996—），女，河南驻马店人，硕士研究生，主要研究方向为手语识别、手语翻译。
薛万利（1986—），男，江苏南京人，博士，CCF会员，主要研究方向为目标跟踪、手语识别。
袁甜甜（1980—），女，天津人，博士，教授，CCF会员，主要研究方向为手语识别、计算机网络。
基金资助:
国家自然科学基金(61906135);国家自然科学基金(62020106004);国家自然科学基金(92048301)

Abstract

Abstract:

Different from spoken languages, sign language is mainly composed of continuous gestures. Sign langu-age recognition and translation are important means of facilitating barrier-free communication between the hearing-impaired and the hearing person. The sign language recognition and translation research task is a typical multi-domain cross-study by processing and analyzing sign language videos and displaying the recognition results in text form. In recent years, sign language recognition and translation research based on deep learning has made great progress. In order to facilitate researchers to systematically and comprehensively understand the research tasks of sign language recognition and translation, the review work is carried out from the perspectives of sign language recognition and sign language translation. Firstly, the translation research work is classified and summarized and its characteristics are analyzed. Secondly, the common sign language recognition and translation research datasets of different countries are summarized and classified from the perspectives of isolated sign language words and continuous sign language sentences. Based on the difference in research tasks, the corresponding evaluation index system is introduced. Finally, the major challenges of current research on sign language recognition and translation are summarized from the aspects of effective information extraction of sign language visual features, multi-cue weight assignment, relationship between sign language and natural language grammar, and sign language dataset resources.

Key words: sign language recognition, sign language translation, deep learning, neural network

摘要：

不同于有声语言，手语主要由连续的手势动作构成。手语识别与翻译是促成听障人士与健听人士之间无障碍交流的重要手段。手语识别与翻译研究任务通过对手语视频进行处理分析并以文字形式显示识别结果，是一种典型的多领域交叉研究。近年来，基于深度学习的手语识别与翻译研究获得了长足的进步。为了便于研究者们系统、全面地了解手语识别与翻译研究任务，分别以手语识别和手语翻译两大任务为主线，从三方面展开综述工作：首先，对具备代表性的手语识别和手语翻译研究工作进行分类总结并分析其特点；其次，归纳整理当前常用的不同国别手语识别与翻译研究数据集，分别从孤立词和连续手语语句两个角度进行分类，同时根据手语识别和手语翻译研究任务的差异性，介绍了对应的评价指标体系；最后，从手语视觉特征的有效信息提取、多线索权重分配、手语与自然语言语法对应及手语数据集资源等方面总结了手语识别与翻译研究目前存在的主要挑战。

关键词: 手语识别, 手语翻译, 深度学习, 神经网络

CLC Number:

TP391

YAN Siyi, XUE Wanli, YUAN Tiantian. Survey of Sign Language Recognition and Translation[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(11): 2415-2429.

闫思伊, 薛万利, 袁甜甜. 手语识别与翻译综述[J]. 计算机科学与探索, 2022, 16(11): 2415-2429.

Figures/Tables 3

References 145

[1]	吕会华. 中国手语语言学[M]. 北京: 知识产权出版社有限责任公司, 2019.
	LV H H. The linguistics of Chinese sign language[M]. Bei-jing: Intellectual Property Publishing House, 2019.
[2]	HUANG J, ZHOU W G, LI H Q, et al. Attention-based 3D-CNNs for large-vocabulary sign language recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 29(9): 2822-2832. DOI URL
[3]	KOLLER O, ZARGARAN S, NEY H. Re-sign: re-aligned end-to-end sequence modelling with deep recurrent CNN-HMMs[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 3416-3424.
[4]	REN Z, YUAN J S, MENG J J, et al. Robust part-based hand gesture recognition using Kinect sensor[J]. IEEE Transac-tions on Multimedia, 2013, 15(5): 1110-1120.
[5]	HUANG J, ZHOU W G, ZHANG Q L, et al. Video-based sign language recognition without temporal segmentation[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, the 30th Innovative Applications of Artificial Intelligence, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 2257-2264.
[6]	WEN H Y, RAMOS R J, DEY A K. Serendipity: finger ges-ture recognition using an off-the-shelf smartwatch[C]//Pro-ceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, May 7-12, 2016. New York: ACM, 2016: 3847-3851.
[7]	EKIZ D, KAYA G E, BUĞUR S, et al. Sign sentence reco-gnition with smart watches[C]//Proceedings of the 25th Sig-nal Processing and Communications Applications Conference, Antalya, May 15-18, 2017. Piscataway: IEEE, 2017: 1-4.
[8]	HOU J H, LI X Y, ZHU P D, et al. SignSpeaker: a real-time, high-precision smartwatch-based sign language translator[C]//Proceedings of the 25th Annual International Conference on Mobile Computing and Networking, Los Cabos, Oct 21-25, 2019. New York: ACM, 2019: 1-15.
[9]	WANG Z B, ZHAO T D, MA J X, et al. Hear sign langu-age: a real-time end-to-end sign language recognition sys-tem[J]. IEEE Transactions on Mobile Computing, 2020, 21(7): 2398-2410.
[10]	CAMGÖZ N C, KOLLER O, HADFIELD S, et al. Sign lan-guage transformers: joint end-to-end sign language recogni-tion and translation[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 10023-10033.
[11]	KOLLER O. Quantitative survey of the state of the art in sign language recognition[J]. arXiv:2008.09918, 2020.
[12]	FORSTER J, SCHMIDT C, KOLLER O, et al. Extensions of the sign language recognition and translation corpus RWTH-PHOENIX-Weather[C]//Proceedings of the 9th International Conference on Language Resources and Evaluation, Reyk-javik, May 26-31, 2014. Paris: European Language Resour-ces Association, 2014: 1911-1916.
[13]	张淑军, 张群, 李辉. 基于深度学习的手语识别综述[J]. 电子与信息学报, 2020, 42(4): 1021-1032.
	ZHANG S J, ZHANG Q, LI H. Review of sign language recognition based on deep learning[J]. Journal of Electro-nics & Information Technology, 2020, 42(4): 1021-1032.
[14]	米娜瓦尔·阿不拉, 阿里甫·库尔班, 解启娜, 等. 手语识别方法与技术综述[J]. 计算机工程与应用, 2021, 57(18): 1-12. DOI
	ABULA M, KUERBAN A, XIE Q N, et al. Review of sign language recognition methods and techniques[J]. Computer Engineering and Applications, 2021, 57(18): 1-12. DOI
[15]	秦梦现. 手语识别研究综述[J]. 软件导刊, 2021, 20(2):250-252.
	QIN M X. A survey of sign language recognition[J]. Soft-ware Guide, 2021, 20(2): 250-252.
[16]	HU M C, LIU H T, LI J W, et al. Sign language recognition in complex background scene based on adaptive skin color modelling and support vector machine[J]. International Jour-nal of Big Data Intelligence, 2018, 5(1/2): 21.
[17]	LOWE D G. Distinctive image features from scale-invariant key points[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. DOI URL
[18]	DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the 2005 IEEE Con-ference on Computer Vision and Pattern Recognition, San Diego, Jun 20-26, 2005. Washington: IEEE Computer Society, 2005: 886-893.
[19]	LAPTEV I, MARSZALEK M, SCHMID C, et al. Learning realistic human actions from movies[C]//Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recog-nition, Anchorage, Jun 23-28, 2008. Piscataway: IEEE, 2008: 1-8.
[20]	BO L F, LAI K, REN X F, et al. Object recognition with hierarchical kernel descriptors[C]//Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recogni-tion, Colorado Springs, Jun 20-25, 2011. Washington: IEEE Computer Society, 2011: 1729-1736.
[21]	OREIFEJ O, LIU Z C. HON4D: histogram of oriented 4D normal for activity recognition from depth sequences[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, Jun 23-28, 2013. Piscataway: IEEE, 2013: 716-723.
[22]	WANG J, LIU Z C, CHOROWSKI J, et al. Robust 3D ac-tion recognition with random occupancy patterns[C]//LNCS 7573: Proceedings of the 12th European Conference on Com-puter Vision, Florence, Oct 7-13, 2012. Berlin, Heidelberg: Springer, 2012: 872-885.
[23]	MIRANDA L, VIEIRA T, MORERA D M, et al. Real-time gesture recognition from depth data through key poses learning and decision forests[C]//Proceedings of the 2012 25th SIBG-RAPI Conference on Graphics, Patterns and Images, Ouro Preto, Aug 22-25, 2012. Washington: IEEE Computer Society, 2012: 268-275.
[24]	VIEIRA A W, NASCIMENTO E R, OLIVEIRA G L, et al. STOP: space-time occupancy patterns for 3D action recogni-tion from depth map sequences[C]//LNCS 7441: Proceedings of the 17th Iberoamerican Congress on Progress in Pattern Recognition, Image Analysis, Computer Vision, and Appli-cations, Buenos Aires, Sep 3-6, 2012. Berlin, Heidelberg: Springer, 2012: 252-259.
[25]	ZHANG J H, ZHOU W G, XIE C, et al. Chinese sign lan-guage recognition with adaptive HMM[C]//Proceedings of the 2016 IEEE International Conference on Multimedia and Expo, Seattle, Jul 11-15, 2016. Washington: IEEE Computer Society, 2016: 1-6.
[26]	YIN F, CHAI X J, CHEN X L. Iterative reference driven metric learning for signer independent isolated sign langu-age recognition[C]//LNCS 9911: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016: 434-450.
[27]	ZHENG L H, LIANG B. Sign language recognition using depth images[C]//Proceedings of the14th International Con-ference on Control, Automation, Robotics and Vision, Phu-ket, Nov 13-15, 2016. Piscataway: IEEE, 2016: 1-6.
[28]	LI F F, PERONA P. A Bayesian hierarchical model for lear-ning natural scene categories[C]//Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, Jun 20-25, 2005. Piscata-way: IEEE, 2005: 524-531.
[29]	CHEN G, GIULIANI M, CLARKE D, et al. Action recogni-tion using ensemble weighted multi-instance learning[C]//Proceedings of the 2014 IEEE International Conference on Robotics and Automation, Hong Kong, China, May 31-Jun 7, 2014. Piscataway: IEEE, 2014: 4520-4525.
[30]	YANG R D, SARKAR S. Detecting coarticulation in sign language using conditional random fields[C]//Proceedings of the 18th International Conference on Pattern Recognition, Hong Kong, China, Aug 20-24, 2006. Washington: IEEE Com-puter Society, 2006: 108-112.
[31]	BREIMAN L. Random forests[J]. Machine Learning, 2001, 45(1): 5-32. DOI URL
[32]	AGARWAL A, THAKUR M K. Sign language recognition using Microsoft Kinect[C]//Proceedings of the 6th International Conference on Contemporary Computing, Noida, Aug 8-10, 2013. Piscataway: IEEE, 2013: 181-185.
[33]	WANG H J, CHAI X J, ZHOU Y, et al. Fast sign language recognition benefited from low rank approximation[C]//Pro-ceedings of the 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, Ljubljana, May 4-8, 2015. Washington: IEEE Computer So-ciety, 2015: 1-6.
[34]	BURGESCJ C. A tutorial on support vector machines for pattern recognition[J]. Data Mining and Knowledge Disco-very, 1998, 2(2): 121-167.
[35]	LIU F T, WANG Y T, MA H P. Gesture recognition with wearable 9-axis sensors[C]//Proceedings of the 2017 IEEE International Conference on Communications, Paris, May 21-25, 2017. Piscataway: IEEE, 2017: 1-6.
[36]	PU J F, ZHOU W G, LI H Q. Sign language recognition with multi-modal features[C]//LNCS 9917: Proceedings of the 17th Pacific-Rim Conference on Multimedia, Xi’an, Sep 15-16, 2016. Cham: Springer, 2016: 252-261.
[37]	KUMAR N. Sign language recognition for hearing impaired people based on hands symbols classification[C]//Procee-dings of the 2017 International Conference on Computing, Communication and Automation, Greater Noida, May 5-6, 2017. Piscataway: IEEE, 2017: 244-249.
[38]	GUO D, ZHOU W G, LI H Q, et al. Online early-late fu-sion based on adaptive HMM for sign language recognition[J]. ACM Transactions on Multimedia Computing, Commu-nications, and Applications, 2017, 14(1): 1-18.
[39]	LIU T, ZHOU W G, LI H Q. Sign language recognition with long short-term memory[C]//Proceedings of the 2016 IEEE International Conference on Image Processing, Phoenix, Sep 25-28, 2016. Piscataway: IEEE, 2016: 2871-2875.
[40]	HU H Z, ZHOU W G, LI H Q. Hand-model-aware sign language recognition[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, 33rd Conference on Innovative Applications of Artificial Intelligence, the 11th Symposium on Educational Advances in Artificial Intelli-gence. Menlo Park: AAAI, 2021: 1558-1566.
[41]	WANG F, DU Y X, WANG G R, et al. (2+1) D-SLR: an efficient network for video sign language recognition[J]. Neural Computing and Applications, 2022, 34 (3): 2413-2423. DOI URL
[42]	HU H, ZHOU W, LI H. Hand-model-aware sign language recognition[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, 33rd Conference on Innovative Applications of Artificial Intelligence, the 11th Symposium on Educational Advances in Artificial Intelligence. Menlo Park: AAAI, 2021: 1558-1566.
[43]	WU D, PIGOU L, KINDERMANS P J, et al. Deep dynamic neural networks for multimodal gesture segmentation and recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(8): 1583-1597. DOI PMID
[44]	WANG H J, CHAI X J, CHEN X L. A novel sign language recognition framework using hierarchical Grossmann cov-ariance matrix[J]. IEEE Transactions on Multimedia, 2019, 21(11): 2806-2814. DOI URL
[45]	MITTAL A, KUMAR P, ROY P, et al. A modified LSTM model for continuous sign language recognition using leap motion[J]. IEEE Sensors Journal, 2019, 19(16): 7056-7063. DOI URL
[46]	WEI C C, ZHAO J, ZHOU W G, et al. Semantic boundary detection with reinforcement learning for continuous sign language recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(3): 1138-1149. DOI URL
[47]	KOLLER O, NEY H, BOWDEN R. Deep hand: how to train a CNN on 1 million hand images when your data is continuous and weakly labelled[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recogni-tion, Las Vegas, Jun 27-30, 2016. Washington: IEEE Com-puter Society, 2016: 3793-3802.
[48]	KOLLER O, ZARGARAN O, NEY H, et al. Deep sign: hybrid CNN-HMM for continuous sign language recogni-tion[C]//Proceedings of the British Machine Vision Confer-ence 2016, York, Sep 19-22, 2016. Durham: BMVA Press, 2016: 1-12.
[49]	LI H B, GAO L Q, HAN R Z, et al. Key action and joint CTC-Attention based sign language recognition[C]//Procee-dings of the 2020 IEEE International Conference on Acous-tics, Speech and Signal Processing, Barcelona, May 4-8, 2020. Piscataway: IEEE, 2020: 2348-2352.
[50]	CHENG K L, YANG Z Y, CHEN Q F, et al. Fully convolu-tional networks for continuous sign language recognition[C]//LNCS 12369: Proceedings of the 16th European Con-ference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 697-714.
[51]	TRAN D, BOURDEV L D, FERGUS R, et al. Learning spatio-temporal features with 3D convolutional networks[C]//Pro-ceedings of the 2015 IEEE International Conference on Com-puter Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 4489-4497.
[52]	JI S W, XU W, YANG M, et al. 3D convolutional neural networks for human action recognition[J]. IEEE Transac-tions on Pattern Analysis and Machine Intelligence, 2012, 35(1): 221-231.
[53]	CARREIRA J, ZISSERMAN A. Quo Vadis, action recogni-tion? A new model and the kinetics dataset[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pat-tern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 4724-4733.
[54]	HARA K, KATAOKA H, SATOH Y. Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Washington: IEEE Computer Society, 2018: 6546-6555.
[55]	ZHAO K, ZHANG K J, ZHAI Y, et al. Real-time sign lan-guage recognition based on video stream[C]//Proceedings of the 2020 39th Chinese Control Conference, Shenyang, Jul 27-29, 2020. Piscataway: IEEE, 2021: 158-174.
[56]	LIAO Y Q, XIONG P W, MIN W D, et al. Dynamic sign language recognition based on video sequence with BLSTM-3D residual networks[J]. IEEE Access, 2019, 7: 38044-38054. DOI URL
[57]	YANG Z Y, SHI Z M, SHEN X Y, et al. SF-Net: structured-feature network for continuous sign language recognition[J]. arXiv:1908.01341, 2019.
[58]	PU J F, ZHOU W G, LI H Q. Iterative alignment network for continuous sign language recognition[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 15-20, 2019. Piscata-way: IEEE, 2019: 4165-4174.
[59]	HOCHREITER S, SCHMIDHUBER J. Long short-term me-mory[J]. Neural Computation, 1997, 9(8): 1735-1780. DOI URL
[60]	GRAVES A, FERNÁNDEZ S, SCHMIDHUBER J. Bidirec-tional LSTM networks for improved phoneme classifica-tion and recognition[C]//LNCS 3697: Proceedings of the 15th International Conference on Artificial Neural Networks, Warsaw, Sep 11-15, 2005. Berlin, Heidelberg: Springer, 2005: 799-804.
[61]	CHO K, VAN MERRIËNBOER B, GÜLÇEHRE Ç, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proc-essing, Oct 25-29, 2014. Stroudsburg: ACL, 2014: 1724-1734.
[62]	PU J F, ZHOU W G, HU H Z, et al. Boosting continuous sign language recognition via cross modality augmentation[C]//Proceedings of the 28th ACM International Conference on Multimedia, Seattle, Oct 12-16, 2020. New York: ACM, 2020: 1497-1505.
[63]	CAMGÖZ N C, HADFIELD S, KOLLER O, et al. SubUNets: end-to-end hand shape and continuous sign language recog-nition[C]//Proceedings of the 2017 IEEE International Con-ference on Computer Vision, Venice, Oct 22-29, 2017. Wa-shington: IEEE Computer Society, 2017: 3075-3084.
[64]	PU J F, ZHOU W G, LI H Q. Dilated convolutional network with iterative optimization for continuous sign language recog-nition[C]//Proceedings of the 27th International Joint Con-ference on Artificial Intelligence, Stockholm, Jul 13-19, 2018. San Francisco: Morgan Kaufmann, 2018: 885-891.
[65]	GRAVES A, FERNÁNDEZ S, GOMEZ F, et al. Connec-tionist temporal classification: labelling unsegmented sequence data with recurrent neural networks[C]//Proceedings of the 23rd International Conference on Machine learning, Pitts-burgh, Jun 25-29, 2006. New York: ACM, 2006: 369-376.
[66]	WANG S, GUO D, ZHOU W G, et al. Connectionist tem-poral fusion for sign language translation[C]//Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Oct 22-26, 2018. New York: ACM, 2018: 1483-1491.
[67]	XIAO Q K, CHANG X, ZHANG X, et al. Multi-informa-tion spatial-temporal LSTM fusion continuous sign language neural machine translation[J]. IEEE Access, 2020, 8: 216718-216728.
[68]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Proces-sing Systems 30, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 5998-6008.
[69]	TUNGA A, NUTHALAPATI S V, WACHS J P. Pose-based sign language recognition using GCN and BERT[C]//Procee-dings of the 2021 IEEE Winter Conference on Applications of Computer Vision, Waikoloa, Jan 5-9, 2021. Piscataway: IEEE, 2021: 31-40.
[70]	NIU Z, MAK B. article-title>Stochastic fine-grained labeling of multi-state sign glosses for continuous sign language recognition[C]//LNCS 12361: Proceedings of the 16th European Con-ference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 172-186.
[71]	VAROL G, MOMENI L, ALBANIE S, et al. Read and attend: temporal localisation in sign language videos[C]//Procee-dings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition, Nashville, Jun 20-25, 2021. Pisca-taway: IEEE, 2021: 16857-16866.
[72]	ZHANG Z H, PU J F, ZHUANG L S, et al. Continuous sign language recognition via reinforcement learning[C]//Procee-dings of the 2019 IEEE International Conference on Image Processing, Taiwan, China, Sep 22-25, 2019. Piscataway: IEEE, 2019: 285-289.
[73]	YIN Q F, TAO W Q, LIU X L, et al. Neural sign language translation with SF-transformer[C]//Proceedings of the 6th International Conference on Innovation in Artificial Intelli-gence, Guangzhou, Mar 4-6, 2022. New York: ACM, 2022: 64-68.
[74]	ZHOU H, ZHOU W G, ZHOU Y, et al. Spatial-temporal multi-cue network for sign language recognition and translation[J]. IEEE Transactions on Multimedia, 2022, 24: 768-779. DOI URL
[75]	PAPASTRATIS I, DIMITROPOULOS K, KONSTANTINIDIS D, et al. Continuous sign language recognition through cross-modal alignment of video and text embeddings in a joint-latent space[J]. IEEE Access, 2020, 8: 91170-91180. DOI URL
[76]	GAO L Q, LI H B, LIU Z, et al. RNN-transducer based Chinese sign language recognition[J]. Neurocomputing, 2021, 434: 45-54. DOI URL
[77]	CAMGÖZ N C, HADFIELD S, KOLLER O, et al. Neural sign language translation[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Washington: IEEE Computer Society, 2018: 7784-7793.
[78]	GUO D, ZHOU W G, LI A Y, et al. Hierarchical recurrent deep fusion using adaptive clip summarization for sign lan-guage translation[J]. IEEE Transactions on Image Processing, 2019, 29: 1575-1590. DOI URL
[79]	LI D X, XU C C, YU X, et al. TSPNet: hierarchical feature learning via temporal semantic pyramid for sign language translation[C]//Advances in Neural Information Processing Systems 33, 2020: 12034-12045.
[80]	CHEN Y T, WEI F Y, SUN X, et al. A simple multi-modality transfer learning baseline for sign language translation[J]. arXiv:2203.04287, 2022.
[81]	CAMGÖZ N C, KOLLER O, HADFIELD S, et al. Multi-channel transformers for multi-articulatory sign language translation[C]//LNCS 12538: Proceedings of the 2020 Euro-pean Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 301-319.
[82]	FANG B Y, CO J, ZHANG M. DeepASL: enabling ubiqui-tous and non-intrusive word and sentence-level sign langu-age translation[C]//Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems, Delft, Nov 6-8, 2017. New York: ACM, 2017: 1-13.
[83]	YIN K, READ J. Better sign language translation with STMC-transformer[C]//Proceedings of the 28th International Con-ference on Computational Linguistics, Barcelona, Dec 8-13, 2020: 5975-5989.
[84]	ZHOU H, ZHOU W G, QI W Z, et al. Improving sign lan-guage translation with monolingual data by sign back-translation[C]// Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition, Nashville, Jun 20-25, 2021. Piscataway: IEEE, 2021: 1316-1325.
[85]	YIN A X, ZHAO Z, LIU J L, et al. SimulSLT: end-to-end simultaneous sign language translation[C]// Proceedings of the 29th ACM International Conference on Multimedia, Cheng-du, Oct 20-24, 2021. New York: ACM, 2021: 4118-4127.
[86]	GAO W, MA J Y, WU J Q, et al. Sign language recognition based on HMM/ANN/DP[J]. International Journal of Pattern Recognition and Artificial Intelligence, 2000, 14(5): 587-602. DOI URL
[87]	IWAI Y, WATANABE K, YAGI Y, et al. Gesture recognition by using colored gloves[C]// Proceedings of the 1996 Inter-national Conference on Pattern Recognition, Beijing, Oct 14-17, 1996. Piscataway: IEEE, 1996: 76-81.
[88]	MISTRY J, INDEN B. An approach to sign language trans-lation using the Intel real sense camera[C]// Proceedings of the 2018 10th Computer Science and Electronic Enginee-ring Conference, Colchester, Sep 19-21, 2018. Piscataway: IEEE, 2018: 219-224.
[89]	MEJÍA-PERÉZ K, CÓRDOVA-ESPARZA D M, TERVEN J, et al. Automatic recognition of Mexican sign language using a depth camera and recurrent neural networks[J]. Applied Sciences, 2022, 12(11): 5523. DOI URL
[90]	ATHITSOS V, NEIDLE C, SCLAROFF S, et al. The Ame-rican sign language lexicon video dataset[C]// Proceedings of the 2008 IEEE Computer Society Conference on Com-puter Vision and Pattern Recognition, Anchorage, Jun 23-28, 2008. Washington: IEEE Computer Society, 2008: 1-8.
[91]	JOHNSTON T. From archive to corpus: transcription and annotation in the creation of signed language corpora[J]. International Journal of Corpus Linguistics, 2010, 15(1): 106-131. DOI URL
[92]	SCHEMBRI A, FENLON J, RENTELIS R, et al. Building the British sign language corpus[J]. Language Documenta-tion & Conservation, 2013, 7: 136-154.
[93]	OSZUST M, MARIAN W. Polish sign language words recog-nition with Kinect[C]// Proceedings of the 6th International Conference on Human System Interactions, Sopot, Jun 6-8, 2013. Piscataway: IEEE, 2013: 219-226.
[94]	CHAI X J, WANG H J, CHEN X L. The DEVISIGN large vocabulary of Chinese sign language database and baseline evaluations: VIPL-TR-14-SLR-001[R]. 2014.
[95]	GUTIERREZ-SIGUT E, COSTELLO B, BAUS C, et al. LSE-sign: a lexical database for Spanish sign language[J]. Beha-vior Research Methods, 2016, 48(1): 123-137.
[96]	RONCHETTI F, QUIROGA F, ESTREBOU C A, et al. LSA-64: an Argentinian sign language dataset[C]// Proceedings of the XXII Congreso Argentino de Ciencias de la Compu-tación, San Luis, Jun 2, 2016. Argentina: Nueva Editorial Universitario, 2016: 794-803.
[97]	MAVADATI S M, SANGER P, MAHOOR M H. Extended DISFA dataset: investigating posed and spontaneous facial expressions[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 26-Jul 1, 2016. Washington: IEEE Computer Society, 2016: 1452-1459.
[98]	EBLING S, CAMGÖZ N C, BRAEM P B, et al. SMILE Swiss German sign language dataset[C]// Proceedings of the 11th International Conference on Language Resources and Evaluation, Miyazaki, May 7-12, 2018: 1-9.
[99]	JOZE H R V, KOLLER O. MS-ASL: a large-scale data set and benchmark for understanding American sign language[C]// Proceedings of the 30th British Machine Vision Con-ference 2019, Cardiff, Sep 9-12, 2019. Durham: BMVA Press, 2019: 100.
[100]	LI D X, OPAZO C R, YU X, et al. Word-level deep sign language recognition from video: a new large-scale dataset and methods comparison[C]// Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision,Snowmass Village, Mar 1-5, 2020. Piscataway: IEEE, 2020: 1448-1458.
[101]	ÖZDEMIR O, KINDIROĞLU A A, CAMGÖZ N C, et al. BosphorusSign22k sign language recognition dataset[C]// Proceedings of the 9th Workshop on the Representation and Processing of Sign Languages, Marseille, May 11-16, 2020: 181-188.
[102]	SINCAN O M, KELES H Y. AUTSL: a large scale multi-modal Turkish sign language dataset and baseline methods[J]. IEEE Access, 2020, 8: 181340-181355. DOI URL
[103]	ALBANIE S, VAROL G, MOMENI L, et al. BSL-1K: sca-ling up co-articulated sign language recognition using mou-thing cues[C]// LNCS 12356:Proceedings of the 16th Euro-pean Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 35-53.
[104]	SRIDHAR A, GANESAN R G, KUMAR P, et al. INCLUDE: a large scale dataset for Indian sign language recognition[C]// Proceedings of the 28th ACM International Confer-ence on Multimedia, New York, Oct 12-16, 2020. New York: ACM, 2020: 1366-1375.
[105]	TAVELLA F, SCHLEGEL V, ROMEO M, et al. WLASL-LEX: a dataset for recognising phonological properties in american sign language[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Dublin, May 22-27, 2022. Stroudsburg: ACL, 2022: 453-463.
[106]	DREUW P, FORSTER J, DESELAERS T, et al. Efficient approximations to model-based joint tracking and recogni-tion of continuous sign language[C]// Proceedings of the 8th IEEE International Conference on Automatic Face & Gesture Recognition, Amsterdam, Sep 17-19, 2008. Washing-ton: IEEE Computer Society, 2008: 1-6.
[107]	VON A U, KRAISS K F. Towards a video corpus for signer-independent continuous sign language recognition[C]// Pro-ceedings of the 2007 International Conference on Gesture in Human-Computer Interaction and Simulation, Lisbon, 2007.
[108]	VIITANIEMI V, JANTUNEN T, SAVOLAINEN L, et al. S-pot—a benchmark in spotting signs within continuous signing[C]// Proceedings of the 9th International Conference on Language Resources and Evaluation, Reykjavik, May 26-31, 2014. European Language Resources Association, 2014.
[109]	KO S K, KIM C J, JUNG H, et al. Neural sign language translation based on human key point estimation[J]. App-lied Sciences, 2019, 9(13): 2683.
[110]	ADALOGLOU N M, CHATZIS T, PAPASTRATIS I, et al. A comprehensive study on deep learning-based methods for sign language recognition[J]. IEEE Transactions on Multi-media, 2022, 24: 1750-1762.
[111]	BULL H, BRAFFORT A, GOUIFFÈS M. MEDIAPI-SKEL—a 2D-skeleton video database of french sign language with aligned French subtitles[C]// Proceedings of the 12th Confer-ence on Language Resources and Evaluation, Marseille, May 11-16, 2020. Stroudsburg: ACL, 2020: 6063-6068.
[112]	DUARTE A C, PALASKAR S, VENTURA L, et al. How-2Sign: a large-scale multimodal dataset for continuous American sign language[C]// Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition, Piscataway: IEEE, 2021: 2735-2744.
[113]	中国残疾人联合会教育就业部, 中国聋人协会. 中国手语[M]. 北京: 华夏出版社, 2003.
	China Disable Persons’ Federation Employment Service and Administration Center, China Association of Persons with Hearing Disabilities. Chinese sign language[M]. Beijing: Huaxia Publishing House, 2003.
[114]	ZHANGZ Y. Microsoft Kinect sensor and its effect[J]. IEEE Multimedia, 2012, 19(2): 4-10.
[115]	WANG H J, CHAI X J, HONG X P, et al. Isolated sign language recognition with grassmann covariance matrices[J]. ACM Transactions on Accessible Computing, 2016, 8 (4): 1-21.
[116]	EIDE E, GISH H, JEANRENAUD P, et al. Understanding and improving speech recognition performance through the use of diagnostic tools[C]// Proceedings of the 1995 In-ternational Conference on Acoustics, Speech, and Signal Processing, Detroit, May 9-12, 1995. Washington: IEEE Computer Society, 1995: 221-224.
[117]	PAPINENI K, ROUKOS S, WARD T, et al. BLEU: a method for automatic evaluation of machine translation[C]// Procee-dings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Jul 6-12, 2002. Stroudsburg: ACL, 2002: 311-318.
[118]	VEDANTAM R, ZITNICK C L, PARIKH D. CIDEr-R: consensus-based image description evaluation[C]// Procee-dings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washing-ton: IEEE Computer Society, 2015: 4566-4575.
[119]	LIN C Y. ROUGE: a package for automatic evaluation of summaries[C]// Proceedings of the 2004 Workshop on Text Summarization Branches Out, Barcelona, Jul 25, 2004. Stroudsburg: ACL, 2004: 74-81.
[120]	BANERJEE S, LAVIE A. METEOR: an automatic metric for MT evaluation with improved correlation with human judgments[C]// Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization@ACL 2005, Ann Arbor, Jun 29, 2005. Stroudsburg: ACL, 2005: 65-72.
[121]	MARTINEAU J, FININ T. Delta TFIDF: an improved fea-ture space for sentiment analysis[C]// Proceedings of the 3rd International Conference on Weblogs and Social Media, San Jose, May 17-20, 2009. Menlo Park: AAAI, 2009: 258-261.
[122]	WEI C C, ZHOU W G, PU J F, et al. Deep grammatical multi-classifier for continuous sign language recognition[C]// Proceedings of the 5th IEEE International Conference on Multimedia Big Data, Singapore, Sep 11-13, 2019. Pisca-taway: IEEE, 2019: 435-442.
[123]	KOLLER O, NEY H, BOWDEN R. Automatic alignment of HamnoSys subunits for continuous sign language recog-nition[C]// Proceedings of the 10th Edition of Its Language Resources and Evaluation Conference 7th Workshop on the Representation and Processing of Sign Languages, Por-torož, May 23-28, 2016. New York: McGraw-Hill, 2016: 121-128.
[124]	SARHAN N, LAURI M, FRINTROP S. Multi-phase fine-tuning: a new fine-tuning approach for sign language recog-nition[J]. KI-Künstliche Intelligent, 2022, 36(1): 91-98.
[125]	CUI R P, HU L, ZHANG C S. Recurrent convolutional neural networks for continuous sign language recognition by staged optimization[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 1610-1618.
[126]	KONSTANTINIDIS D, DIMITROPOULOS K, DARAS P. A deep learning approach for analyzing video and skele-tal features in sign language recognition[C]// Proceedings of the 2018 IEEE International Conference on Imaging Sys-tems and Techniques, Krakow, Oct 16-18, 2018. Piscataway: IEEE, 2018: 1-6.
[127]	GUO D, ZHOU W G, LI H Q, et al. Hierarchical LSTM for sign language translation[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence, the 30th In-novative Applications of Artificial Intelligence, and the 8th AAAI Symposium on Educational Advances in Artifi-cial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 6845-6852.
[128]	KOLLER O, ZARGARAN S, NEY H, et al. Deep sign: enabling robust statistical continuous sign language recog-nition via hybrid CNN-HMMs[J]. International Journal of Computer Vision, 2018, 126(12): 1311-1325. DOI URL
[129]	CUI R P, LIU H, ZHANG C S. A deep neural framework for continuous sign language recognition by iterative trai-ning[J]. IEEE Transactions on Multimedia, 2019, 21(7): 1880-1891. DOI URL
[130]	KOLLER O, CAMGOZ N C, NEY H, et al. Weakly super-vised learning with multi-stream CNN-LSTM-HMMs to discover sequential parallelism in sign language videos[J]. IEEE Transactions on Pattern Analysis and Machine Intel-ligence, 2020, 42(9): 2306-2320.
[131]	BORG M, CAMILLERI K P. Phonologically-meaningful subunits for deep learning-based sign language recognition[C]// LNCS 12536: Proceedings of the 16th European Con-ference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 199-217.
[132]	BULL H, GOUIFFÈS M, BRAFFORT A. Automatic segmen-tation of sign language into subtitle-units[C]// LNCS 12536: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 186-198.
[133]	ZHOU M J, NG M, CAI Z X, et al. Self-attention-based fully-inception networks for continuous sign language recogni-tion[C]// Proceedings of the 24th European Conference on Artificial Intelligence, Santiago de Compostela, Aug 29-Sep 8, 2020. Amsterdam: IOS Press, 2020: 2832-2839.
[134]	MIN Y C, HAO A M, CHAI X J, et al. Visual alignment constraint for continuous sign language recognition[C]// Proceedings of the 2021 IEEE/CVF Conference on Com-puter Vision and Pattern Recognition, Montreal, Oct 10-17, 2021. Piscataway: IEEE, 2021: 11542-11551.
[135]	PEI X K, GUO D, ZHAO Y. Continuous sign language recognition based on pseudo-supervised learning[C]// Pro-ceedings of the 2nd Workshop on Multimedia for Acces-sible Human Computer Interfaces, Nice, Oct 25, 2019. New York: ACM, 2019: 33-39.
[136]	RENZ K, STACHE N C, ALBANIE S, et al. Sign language segmentation with temporal convolutional networks[C]// Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, Jun 6-11, 2021. Piscataway: IEEE, 2021: 2135-2139.
[137]	ZHOU H, ZHOU W G, LI H Q. Dynamic pseudo label de-coding for continuous sign language recognition[C]// Procee-dings of the 2019 IEEE International Conference on Multi-media and Expo, Shanghai, Jul 8-12, 2019. Piscataway: IEEE, 2019: 1282-1287.
[138]	SONG P P, GUO D, XIN H R, et al. Parallel temporal en-coder for sign language translation[C]// Proceedings of the 2019 IEEE International Conference on Image Processing,Taiwan, China, Sep 22-25, 2019. Piscataway: IEEE, 2019: 1915-1919.
[139]	GUO D, WANG S, TIAN Q, et al. Dense temporal convolu-tion network for sign language translation[C]// Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 744-750.
[140]	GÖKÇE Ç, ÖZDEMIR O, KINDIROĞLU A A, et al. Score-level multi cue fusion for sign language recognition[C]// LNCS 12536: Proceedings of the 2020 European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 294-309.
[141]	LI D X, YU X, XU C C, et al. Transferring cross-domain knowledge for video sign language recognition[C]// Procee-dings of the 2020 IEEE/CVF Conference on Computer Vi-sion and Pattern Recognition, Seattle, Jun 13-19, 2020. Pis-cataway: IEEE, 2020: 6204-6213.
[142]	VAROL G, MOMENI L, ALBANIE S, et al. Read and at-tend: temporal localisation in sign language videos[C]// Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021:16857-16866.
[143]	MORYOSSEF A, TSOCHANTARIDIS I, DINN J, et al. Evaluating the immediate applicability of pose estimation for sign language recognition[C]// Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recog-nition. Piscataway: IEEE, 2021: 3434-3440.
[144]	JIANG S Y, SUN B, WANG L C, et al. Skeleton aware multi-modal sign language recognition[C]// Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 3413-3423.
[145]	LI R M, MENG L. Multi-view spatial-temporal network for continuous sign language recognition[J]. arXiv:2204.08747, 2022.

数据集	语言	类型	属性
数据集	语言	类型	包含手语词个数	视频数	手语演示者个数
Boston ASLLVD^[90]	美国手语	孤立词	2 742	9 794	6
AUSLAN^[91]	澳大利亚手语	孤立词	—	1 100	100
BSL Corpus^[92]	英国手语	孤立词	5 000	—	249
PSL Kinect30^[93]	波兰手语	孤立词	30	300	1
DEVISIGN-G/D/L^[94]	中国手语	孤立词	36/500/2 000	24 000	8
LSE-sign^[95]	西班牙手语	孤立词	2 400	2 400	2
LSA64^[96]	阿根廷手语	孤立词	64	3 200	10
USTC-ICSL^[27]	中国手语	孤立词	500	125 000	50
DISFA^[97]	英国手语	孤立词	—	130 000	27
SMILE^[98]	德国手语	孤立词	—	—	30
MS-ASL^[99]	美国手语	孤立词	1 000	25 513	222
WLASL^[100]	美国手语	孤立词	2 000	21 083	119
BosphorusSign22k^[101]	土耳其手语	孤立词	744	22 542	6
AUTSL^[102]	土耳其手语	孤立词	226	38 336	43
CSSL5000^[50]	中国手语	孤立词	1 000	100 000	10
BSL-1K^[103]	英国手语	孤立词	1 064	1 412	40
INCLUDE^[104]	印度手语	孤立词	263	4 287	7
NMFs-CSL^[39]	中国手语	孤立词	1 067	32 010	10
WLASL-LEX^[105]	美国手语	孤立词	800	10 017	3
Boston-104^[106]	美国手语	连续手语语句	128	214	3
SIGNUM^[107]	德国手语	孤立词+连续手语语句	455	1 230	25
S-pot^[108]	芬兰手语	连续手语语句	—	5 539	5
RWTH-PHOENIX-WEATHER-2014^[12]	德国手语	连续手语语句	1 081	6 841	9
USTC-CCSL^[5]	中国手语	连续手语语句	178	25 000	50
RWTH-PHOENIX-WEATHER-2014-T^[77]	德国手语	连续手语语句	1 066	8 257	9
RCSD^[44]	中国手语	连续手语语句	242	—	10
KETI^[109]	韩国手语	连续手语语句	524	14 672	14
GSL^[110]	希腊手语	孤立词+连续手语语句	310	10 290	7
MEDIAPI-SKEL corpus^[111]	法国手语	连续手语语句	14 383	368	>100
How2Sign^[112]	美国手语	连续手语语句	16 000	2 500	11
CSL-Daily^[83]	中国手语	连续手语语句	2 000	20 654	10

数据集	语言	类型	属性
数据集	语言	类型	包含手语词个数	视频数	手语演示者个数
Boston ASLLVD^[90]	美国手语	孤立词	2 742	9 794	6
AUSLAN^[91]	澳大利亚手语	孤立词	—	1 100	100
BSL Corpus^[92]	英国手语	孤立词	5 000	—	249
PSL Kinect30^[93]	波兰手语	孤立词	30	300	1
DEVISIGN-G/D/L^[94]	中国手语	孤立词	36/500/2 000	24 000	8
LSE-sign^[95]	西班牙手语	孤立词	2 400	2 400	2
LSA64^[96]	阿根廷手语	孤立词	64	3 200	10
USTC-ICSL^[27]	中国手语	孤立词	500	125 000	50
DISFA^[97]	英国手语	孤立词	—	130 000	27
SMILE^[98]	德国手语	孤立词	—	—	30
MS-ASL^[99]	美国手语	孤立词	1 000	25 513	222
WLASL^[100]	美国手语	孤立词	2 000	21 083	119
BosphorusSign22k^[101]	土耳其手语	孤立词	744	22 542	6
AUTSL^[102]	土耳其手语	孤立词	226	38 336	43
CSSL5000^[50]	中国手语	孤立词	1 000	100 000	10
BSL-1K^[103]	英国手语	孤立词	1 064	1 412	40
INCLUDE^[104]	印度手语	孤立词	263	4 287	7
NMFs-CSL^[39]	中国手语	孤立词	1 067	32 010	10
WLASL-LEX^[105]	美国手语	孤立词	800	10 017	3
Boston-104^[106]	美国手语	连续手语语句	128	214	3
SIGNUM^[107]	德国手语	孤立词+连续手语语句	455	1 230	25
S-pot^[108]	芬兰手语	连续手语语句	—	5 539	5
RWTH-PHOENIX-WEATHER-2014^[12]	德国手语	连续手语语句	1 081	6 841	9
USTC-CCSL^[5]	中国手语	连续手语语句	178	25 000	50
RWTH-PHOENIX-WEATHER-2014-T^[77]	德国手语	连续手语语句	1 066	8 257	9
RCSD^[44]	中国手语	连续手语语句	242	—	10
KETI^[109]	韩国手语	连续手语语句	524	14 672	14
GSL^[110]	希腊手语	孤立词+连续手语语句	310	10 290	7
MEDIAPI-SKEL corpus^[111]	法国手语	连续手语语句	14 383	368	>100
How2Sign^[112]	美国手语	连续手语语句	16 000	2 500	11
CSL-Daily^[83]	中国手语	连续手语语句	2 000	20 654	10

特征提取网络	参考文献号
GCN	[132][144][145]
GooleNet	[3][48][62][81][122][124][125][128][129][130][131][133]
VGG-Net	[46][72][83][125][126][129][130]
ResNet	[39][48][57][69]
CaffeNet	[63]
AlexNet	[128]
其他2D-CNN	[10][49][76][123][134]
C3D	[5][65][127][138]
B3D	[55]
I3D	[39][79][103][136][137] [141][142][143]
S3D	[80]
其他3D-CNN	[46][58][64][73][135][139][140][144]

特征提取网络	参考文献号
GCN	[132][144][145]
GooleNet	[3][48][62][81][122][124][125][128][129][130][131][133]
VGG-Net	[46][72][83][125][126][129][130]
ResNet	[39][48][57][69]
CaffeNet	[63]
AlexNet	[128]
其他2D-CNN	[10][49][76][123][134]
C3D	[5][65][127][138]
B3D	[55]
I3D	[39][79][103][136][137] [141][142][143]
S3D	[80]
其他3D-CNN	[46][58][64][73][135][139][140][144]

Survey of Sign Language Recognition and Translation

手语识别与翻译综述

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 3

References 145

Related Articles 15

Recommended Articles

Metrics

[1]	LYU Xiaoqi, JI Ke, CHEN Zhenxiang, SUN Runyuan, MA Kun, WU Jun, LI Yidong. Expert Recommendation Algorithm Combining Attention and Recurrent Neural Network [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2068-2077.
[2]	ZHU Bingyu, LIU Zhen, ZHANG Jingxiang. COVID-19 Detection Algorithm Combining Grad-CAM and Convolutional Neural Network [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2108-2120.
[3]	ZHANG Xiangping, LIU Jianxun. Overview of Deep Learning-Based Code Representation and Its Applications [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2011-2029.
[4]	LI Dongmei, LUO Sisi, ZHANG Xiaoping, XU Fu. Review on Named Entity Recognition [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1954-1968.
[5]	REN Ning, FU Yan, WU Yanxia, LIANG Pengju, HAN Xi. Review of Research on Imbalance Problem in Deep Learning Applied to Object Detection [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1933-1953.
[6]	YANG Caidong, LI Chengyang, LI Zhongbo, XIE Yongqiang, SUN Fangwei, QI Jin. Review of Image Super-resolution Reconstruction Algorithms Based on Deep Learning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1990-2010.
[7]	ZENG Fanzhi, XU Luqian, ZHOU Yan, ZHOU Yuexia, LIAO Junwei. Review of Knowledge Tracing Model for Intelligent Education [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1742-1763.
[8]	AN Fengping, LI Xiaowei, CAO Xiang. Medical Image Classification Algorithm Based on Weight Initialization-Sliding Window CNN [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1885-1897.
[9]	HUANG Hao, GE Hongwei. Deep Residual Expression Recognition Network to Enhance Inter-class Discrimination [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1842-1849.
[10]	YU Huilin, CHEN Wei, WANG Qi, GAO Jianwei, WAN Huaiyu. Knowledge Graph Link Prediction Based on Subgraph Reasoning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1800-1808.
[11]	LI Yuxuan, HONG Xuehai, WANG Yang, TANG Zhengzheng, BAN Yan. Groupwise Learning to Rank Algorithm with Introduction of Activated Weighting [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1594-1602.
[12]	LIU Yi, LI Mengmeng, ZHENG Qibin, QIN Wei, REN Xiaoguang. Survey on Video Object Tracking Algorithms [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1504-1515.
[13]	ZHAO Xiaoming, YANG Yijiao, ZHANG Shiqing. Survey of Deep Learning Based Multimodal Emotion Recognition [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1479-1503.
[14]	XIA Hongbin, XIAO Yifei, LIU Yuan. Long Text Generation Adversarial Network Model with Self-Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1603-1610.
[15]	SUN Fangwei, LI Chengyang, XIE Yongqiang, LI Zhongbo, YANG Caidong, QI Jin. Review of Deep Learning Applied to Occluded Object Detection [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1243-1259.