Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (2): 305-322.DOI: 10.3778/j.issn.1673-9418.2106055

• Surveys and Frontiers • Previous Articles     Next Articles

Review of Human Behavior Recognition Research

PEI Lishen1, LIU Shaobo1,+(), ZHAO Xuezhuan2   

  1. 1. School of Computer and Information Engineering, Henan University of Economics and Law, Zhengzhou 450046, China
    2. School of Intelligent Engineering, Zhengzhou University of Aeronautics, Zhengzhou 450046, China
  • Received:2021-05-19 Revised:2021-07-26 Online:2022-02-01 Published:2021-08-04
  • About author:PEI Lishen, born in 1988, Ph.D., lecturer, M.S. supervisor, member of CCF. Her research interests include action recognition, image processing, computer vision, machine learning, etc.
    LIU Shaobo, born in 1999. His research interest is human action recognition.
    ZHAO Xuezhuan, born in 1986, Ph.D., lecturer, member of CCF. His research interests include object detection, object tracking, saliency detection, abnormal behavior detection, computer vision, etc.
  • Supported by:
    National Natural Science Foundation of China(61806073);Key Research & Development and Promotion Project of Henan Province(192102210097);Key Research & Development and Promotion Project of Henan Province(192102210126);Key Research & Development and Promotion Project of Henan Province(212102210160)

人体行为识别研究综述

裴利沈1, 刘少博1,+(), 赵雪专2   

  1. 1.河南财经政法大学 计算机与信息工程学院,郑州 450046
    2.郑州航空工业管理学院 智能工程学院,郑州 450046
  • 通讯作者: + E-mail: 2559113707@qq.com
  • 作者简介:裴利沈(1988—),女,河南郑州人,博士,讲师,硕士生导师,CCF会员,主要研究方向为行为识别、图像处理、计算机视觉、机器学习等。
    刘少博(1999—),男,河南郑州人,主要研究方向为人体行为识别。
    赵雪专(1986—),男,河南郑州人,博士,讲师,CCF会员,主要研究方向为目标检测、目标跟踪、显著性检测、异常行为检测、计算机视觉等。
  • 基金资助:
    国家自然科学基金(61806073);河南省重点研发与推广专项(科技攻关)基金(192102210097);河南省重点研发与推广专项(科技攻关)基金(192102210126);河南省重点研发与推广专项(科技攻关)基金(212102210160)

Abstract:

Behavior recognition is a hot topic in the field of computer vision. It has experienced the development process from manual design feature representation to deep learning feature expression. This paper classifies the mainstream algorithms in the development of behavior recognition from two aspects of traditional behavior recognition models and deep learning models. The traditional behavior recognition models mainly include feature description methods based on silhouette, space-time interest points, human joint point and trajectories. Among them, the improved dense trajectory method has good robustness and reliability. Deep learning network architecture mainly includes two-stream network, 3D convolution network and hybrid network. Firstly, this paper focuses on the main research ideas and innovations of each behavior recognition algorithm, and introducees the model architecture, algorithm features, application scenarios of each kind of algorithm. Then, the widely used public behavior databases are classified, and the HMDB51 and UCF101 datasets are introduced in detail. The recognition effects of traditional methods and deep learning algorithms on each dataset are compared and analyzed. Through comparative analysis, the traditional methods are not suitable for high-precision behavior recognition, and it is not easy to achieve cross database or cross scene promotion. In depth architecture, two-stream network and 3D convolution network have achieved good behavior recognition effect and are widely used. Finally, the future development of behavior recognition is prospected, and some feasible research directions in the future are pointed out.

Key words: human behavior recognition, deep learning, neural network, behavior dataset

摘要:

行为识别是计算机视觉领域意义重大的热点研究问题,它经历了从手工设计特征表征到深度学习特征表达的发展过程。从传统行为识别模型和深度学习模型两方面,对行为识别发展历程中产生的主流算法进行了归类梳理。传统行为识别模型主要包括基于轮廓剪影、时空兴趣点、人体关节点、运动轨迹的特征描述方法。其中改进的密集轨迹方式拥有良好的鲁棒性和可靠性;深度学习网络架构主要有双流网络、3D卷积网络和混合网络。首先,重点阐述了各行为识别算法的主要研究思路与创新点,并介绍了每类算法的模型架构、算法特色、适用情境等。然后,对广泛使用的公共行为数据库进行了分类阐述,着重对HMDB51和UCF101数据集进行了详细介绍,比较分析了传统方法和深度学习算法在各数据集上的识别效果。通过对比分析发现,传统方法不适用于高精细行为的识别,且不易实现跨数据库或跨场景的推广;深度架构中,双流网络和3D卷积网络获得了比较好的行为识别效果且被广泛使用。最后,对行为识别的未来发展进行了展望,指出了若干将来可行的研究方向。

关键词: 人体行为识别, 深度学习, 神经网络, 行为数据集

CLC Number: