Abstract: To extract the regional features which are conducive to expression classification and achieve the decision-level fusion of multi-regional features, this paper proposes an expression recognition frame based on neighbor binary pattern relation (NBPR) feature descriptor and credibility modification evidence fusion (CMEF). Firstly, for the traditional local binary pattern (LBP) operator only takes the fixed center pixel as threshold, NBPR is proposed, which can encode the XOR relationship of binary pattern between adjacent pixels from multi-orientations in a local region. Then, the extracted NBPR texture features of three salient regions such as eyebrows, eyes and mouth are used to construct the initial probability assignments of evidences. Finally, in view of the deficiency of conflicting evidences combination with Dempster-Shafer (D-S) theory of evidence, a new combination method based on the evidence credibility is used to achieve the decision fusion of the three regional evidences. Experiments of the proposed method are performed on Cohn-Kanade (CK), it achieves an average expression recognition rate of 94.67% and an average time of recognition of 752 ms. The experimental results show that the NBPR is conducive to texture descriptor and CMEF strategy is beneficial to decision-level fusion, so the proposed method has higher expression recognition rate.

Key words: neighbor binary pattern relation (NBPR), multi-regional feature fusion, Dempster-Shafer theory of evidence, credibility modification

摘要： 为了提取有利于表情分类的区域特征以及实现多区域特征的决策级融合，提出了一种基于邻近二值模式关系（neighbor binary pattern relation，NBPR）特征描述子和可信度修正证据融合（credibility modification evidence fusion，CMEF）的表情识别框架。首先针对传统局部二值模式（local binary pattern，LBP）算子仅以中心像素为编码阈值的局限，提出一种NBPR描述子，它对局部区域多方向相邻像素之间的二值模式异或关系进行编码；然后根据提取的眉毛、眼睛和嘴巴区域的NBPR纹理特征进行证据的初始基本概率分配；最后针对D-S（Dempster-Shafer）证据理论在合成冲突证据时的不足，通过一种新的基于证据可信度的合成方法实现3个区域证据的决策融合。该方法在CK（Cohn-Kanade）库上分别取得了94.67%的平均表情识别率以及752 ms的平均识别时间。实验结果表明，提出的NBPR描述子和CMEF策略有利于表情区域的纹理描述和决策级融合，从而具有较高的表情识别率。

关键词: 邻近二值模式关系, 多区域特征融合, D-S证据理论, 可信度修正