计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (4): 898-908.DOI: 10.3778/j.issn.1673-9418.2010070

• 人工智能 • 上一篇    下一篇

应用区域关联自适应图卷积的动作识别方法

马利, 郑诗雨, 牛斌+()   

  1. 辽宁大学 信息学院,沈阳 110036
  • 收稿日期:2020-10-26 修回日期:2021-01-06 出版日期:2022-04-01 发布日期:2021-03-12
  • 通讯作者: + E-mail: niub@lnu.edu.cn
  • 作者简介:马利(1978—),女,辽宁锦州人,博士,讲师,主要研究方向为计算机视觉、片上系统。
    郑诗雨(1995—),女,浙江台州人,硕士研究生,主要研究方向为行为识别。
    牛斌(1963—),男,辽宁大连人,硕士,教授,主要研究方向为嵌入式系统、计算机视觉。
  • 基金资助:
    辽宁省自然科学基金博士启动项目(20170520276)

Action Recognition Method on Regional Association Adaptive Graph Convolution

MA Li, ZHENG Shiyu, NIU Bin+()   

  1. College of Information, Liaoning University, Shenyang 110036, China
  • Received:2020-10-26 Revised:2021-01-06 Online:2022-04-01 Published:2021-03-12
  • About author:MA Li, born in 1978, Ph.D., lecturer. Her research interests include computer vision and system on a chip.
    ZHENG Shiyu, born in 1995, M.S. candidate. Her research interest is action recognition.
    NIU Bin, born in 1963, M.S., professor. His research interests include embedded systems and computer vision.
  • Supported by:
    Ph.D. Start-up Fund of Natural Science Foundation of Liaoning Province(20170520276)

摘要:

基于骨架数据的动作识别方法由于其对动态环境和复杂背景的强适应性而受到广泛的关注和研究,应用图卷积网络描述人体骨架实现人体动作识别可以取得很好的识别效果,但实现过程中图的拓扑结构通常是手动设置的,且在所有层和输入样本上的结构固定,只能捕获关节之间的局部物理关系,会遗漏非物理连接的关节相关性。提出了一种新的基于区域关联自适应图卷积网络的骨架动作识别,通过自适应图卷积使参数化的全局图和单个数据图的结构与模型卷积参数在不同的层中分别进行训练和更新,增加了模型中图形构造的灵活性与模型对于各种数据样本的通用性。同时引入区域关联图卷积,通过在关节特征与连接特征之间交替信息传递来捕获数据帧间各关节的非物理连接相关性。并加入骨骼的二阶数据对原有关节数据进行信息补充,融合两者构成双流网络提升识别网络的性能。在NTU-RGBD大规模数据集上的实验表明,该模型在动作识别的准确率上有了一定的提升。

关键词: 自适应, 区域关联, 双流网络, 图卷积

Abstract:

Action recognition methods based on skeleton data have received extensive attention and research due to their strong adaptability to dynamic environments and complex backgrounds. The application of graph convo-lutional networks to describe human skeleton to realize human action recognition can achieve good recognition results, but the topological structure of the graph is often manually set, and the structure on all layers and input samples is fixed. Also the graph convolutional networks can only capture the local physical relationship between joints, and miss the correlation of non-physical joints. This paper proposes a new skeleton action recognition based on regional association adaptive graph convolutional network. Through adaptive graph convolution, the structure of the parameterized global graph and the single data graph and model convolution parameters are trained and updated in different layers, increasing the flexibility of the graph structure in the model and the versatility of the model for various data samples. This paper introduces the regional association graph convolution, and the non-physical conne-ction correlation of each joint between data frames is captured by alternating information transfer between joint features and connection features. And it adds the second-order data of the skeleton to supplement the original joint data, merges this two to form a two-stream network to improve the performance of the recognition network. Exper-iments on the NTU-RGBD large-scale dataset show that the model has a certain improvement in the accuracy of action recognition.

Key words: adaptation, regional association, two-stream network, graph convolution

中图分类号: