Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (8): 1842-1849.DOI: 10.3778/j.issn.1673-9418.2011042

• Artificial Intelligence • Previous Articles     Next Articles

Deep Residual Expression Recognition Network to Enhance Inter-class Discrimination

HUANG Hao1,2, GE Hongwei1   

  1. 1. Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence (Jiangnan Uni-versity), Wuxi, Jiangsu 214122, China
    2. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu 214122, China
  • Received:2020-11-12 Revised:2021-01-29 Online:2022-08-01 Published:2021-03-28
  • About author:HUANG Hao, born in 1996, M.S. candidate, student member of CCF. His research interests include pattern recognition and machine learning.
    GE Hongwei, born in 1967, Ph.D., professor, Ph.D. supervisor. His research interests include artificial intelligence, pattern recognition, machine learning, image processing and analysis, etc.
  • Supported by:
    the Innovation Program for Graduate of Jiangsu Province(KYLX16_0781);and the Superior Discipline Construction Project of Jiangsu Higher Education Institutions.

强化类间区分的深度残差表情识别网络

黄浩1,2, 葛洪伟1   

  1. 1. 江苏省模式识别与计算智能工程实验室(江南大学),江苏 无锡 214122
    2. 江南大学 人工智能与计算机学院,江苏 无锡 214122
  • 作者简介:黄浩(1996—),男,湖北黄冈人,硕士研究生,CCF学生会员,主要研究方向为模式识别、机器学习。
    葛洪伟(1967—),男,江苏无锡人,博士,教授,博士生导师,主要研究方向为人工智能、模式识别、机器学习、图像处理与分析等。
  • 基金资助:
    江苏省研究生创新计划项目(KYLX16_0781);江苏高校优势学科建设工程资助项目。

Abstract:

Deep facial expression recognition is a challenging task for neural networks to apply in pattern reco-gnition. Compared with face recognition tasks such as identity authentication and feature point recognition, there are a lot of redundant information in facial expression recognition task. To achieve good results, more accurate class-ification is needed. Most researches focus on the generalization and network structure of data and ignore the inter-class relationship of data. In this paper, a deep residual expression recognition network RMRnet (recall matrix distinguished residual net) based on class analysis is proposed. First, the data are fed to the backbone network(Resnet18) to obtain the confusion matrix, after that, the recall matrix is used to analyze the relationship between classes. Then, the branches of network structure are designed based on the inter-class relationship to distinguish the strong-related classes, while the rest classes are designed as the supplementary branch to balance the weak-related class. In the end, by adding those branches to the corresponding position of backbone network, RMRnet is constituted. Compared with baseline method and advanced methods in recent years, experimental results in the popular large-scale database show that the proposed method is more effective than the baseline method and has str-ong competitiveness among a group of advanced methods.

Key words: facial expression recognition, neural network, confusion matrix, inter-class association

摘要:

深度人脸表情识别是神经网络应用于模式识别上一项极具挑战性的任务。相对于身份认证和特征点识别等人脸识别任务,表情识别任务中存在着大量的冗余信息,要得到好的效果,需要更精确的分类。多数研究关注点在数据的泛化性和网络结构上,而忽视了数据的类间关系。提出了一种基于类间分析的深度残差表情识别网络RMRnet。首先,将数据通过骨干网络Resnet18得到混淆矩阵,进一步得到召回率矩阵分析类间关系;然后,凭借类间关系设计网络结构分支,进一步区分强联系类,设计补充支路平衡弱联系类;最后,将分支添加到骨干网络的相应位置,得到RMRnet网络模型。在流行的大型数据库上,与基准方法和近年来的先进方法的对比实验结果表明,提出的方法相较于基准方法效果良好,在一众先进方法中也有很强的竞争力。

关键词: 表情识别, 神经网络, 混淆矩阵, 类间关联

CLC Number: