Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (5): 1146-1154.DOI: 10.3778/j.issn.1673-9418.2104106
• Graphics and Image • Previous Articles Next Articles
CHENG Weiyue1,+(), ZHANG Xueqin2, LIN Kezheng2, LI Ao2
Received:
2021-04-29
Revised:
2021-08-05
Online:
2022-05-01
Published:
2022-05-19
About author:
CHENG Weiyue, born in 1988, M.S., lecturer. Her research interests include sparse representation, image processing, pattern recognition, etc.Supported by:
通讯作者:
+ E-mail: cheng_weiyue@sina.cn作者简介:
程卫月(1988—),女,黑龙江哈尔滨人,硕士,讲师,主要研究方向为稀疏表示、图像处理、模式识别等。基金资助:
CLC Number:
CHENG Weiyue, ZHANG Xueqin, LIN Kezheng, LI Ao. Deep Convolutional Neural Network Algorithm Fusing Global and Local Features[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 1146-1154.
程卫月, 张雪琴, 林克正, 李骜. 融合全局与局部特征的深度卷积神经网络算法[J]. 计算机科学与探索, 2022, 16(5): 1146-1154.
Add to citation manager EndNote|Ris|BibTeX
URL: http://fcst.ceaj.org/EN/10.3778/j.issn.1673-9418.2104106
网络层 | 输入大小 | 参数 | 输出大小 |
---|---|---|---|
Conv-1 | 64×64×3 | 7×7×64-1-3 | 64×64×64 |
MaxPool-1 | 64×64×64 | 2×2-2-0 | 32×32×64 |
Dropout1 | 32×32×64 | — | 32×32×64 |
Conv-2 | 32×32×64 | 3×3×256-1-0 | 32×32×256 |
MaxPool-2 | 32×32×256 | 2×2-2-0 | 16×16×256 |
Dropout2 | 16×16×256 | — | 16×16×256 |
FC | 1×500 | — | 1×500 |
Table 1 Configuration of CNN
网络层 | 输入大小 | 参数 | 输出大小 |
---|---|---|---|
Conv-1 | 64×64×3 | 7×7×64-1-3 | 64×64×64 |
MaxPool-1 | 64×64×64 | 2×2-2-0 | 32×32×64 |
Dropout1 | 32×32×64 | — | 32×32×64 |
Conv-2 | 32×32×64 | 3×3×256-1-0 | 32×32×256 |
MaxPool-2 | 32×32×256 | 2×2-2-0 | 16×16×256 |
Dropout2 | 16×16×256 | — | 16×16×256 |
FC | 1×500 | — | 1×500 |
网络层 | 输入大小 | 参数 | 输出大小 |
---|---|---|---|
Conv-1 | 64×64×3 | 3×3×64-1-1 | 64×64×64 |
Conv-2 | 64×64×64 | 3×3×64-1-1 | 64×64×64 |
MaxPool-1 | 64×64×64 | 2×2-2-0 | 32×32×64 |
Dropout1 | 32×32×64 | — | 32×32×64 |
Conv-3 | 32×32×64 | 3×3×128-1-1 | 32×32×128 |
Conv-4 | 32×32×128 | 3×3×128-1-1 | 32×32×128 |
MaxPool-2 | 32×32×128 | 2×2-2-0 | 16×16×128 |
Dropout2 | 16×16×128 | — | 16×16×128 |
Conv-5 | 16×16×128 | 3×3×256-1-1 | 16×16×256 |
Conv-6 | 16×16×256 | 3×3×256-1-1 | 16×16×256 |
Conv-7 | 16×16×256 | 3×3×256-1-1 | 16×16×256 |
Conv-8 | 16×16×256 | 3×3×256-1-1 | 16×16×256 |
MaxPool-3 | 16×16×256 | 2×2-2-0 | 8×8×256 |
Dropout3 | 8×8×256 | — | 8×8×256 |
Conv-9 | 8×8×256 | 3×3×512-1-1 | 8×8×512 |
Conv-10 | 8×8×512 | 3×3×512-1-1 | 8×8×512 |
Conv-11 | 8×8×512 | 3×3×512-1-1 | 8×8×512 |
Conv-12 | 8×8×512 | 3×3×512-1-1 | 8×8×512 |
MaxPool-4 | 8×8×512 | 2×2-2-0 | 4×4×512 |
Dropout4 | 4×4×512 | — | 4×4×512 |
Conv-13 | 4×4×512 | 3×3×512-1-1 | 4×4×512 |
Conv-14 | 4×4×512 | 3×3×512-1-1 | 4×4×512 |
Conv-15 | 4×4×512 | 3×3×512-1-1 | 4×4×512 |
Conv-16 | 4×4×512 | 3×3×512-1-1 | 4×4×512 |
MaxPool-5 | 4×4×512 | 2×2-2-0 | 2×2×512 |
Dropout5 | 2×2×512 | — | 2×2×512 |
FC-1 | 2×2×512 | — | 500 |
Dropout6 | 1×500 | — | 1×500 |
Table 2 Configuration of improved VGG19
网络层 | 输入大小 | 参数 | 输出大小 |
---|---|---|---|
Conv-1 | 64×64×3 | 3×3×64-1-1 | 64×64×64 |
Conv-2 | 64×64×64 | 3×3×64-1-1 | 64×64×64 |
MaxPool-1 | 64×64×64 | 2×2-2-0 | 32×32×64 |
Dropout1 | 32×32×64 | — | 32×32×64 |
Conv-3 | 32×32×64 | 3×3×128-1-1 | 32×32×128 |
Conv-4 | 32×32×128 | 3×3×128-1-1 | 32×32×128 |
MaxPool-2 | 32×32×128 | 2×2-2-0 | 16×16×128 |
Dropout2 | 16×16×128 | — | 16×16×128 |
Conv-5 | 16×16×128 | 3×3×256-1-1 | 16×16×256 |
Conv-6 | 16×16×256 | 3×3×256-1-1 | 16×16×256 |
Conv-7 | 16×16×256 | 3×3×256-1-1 | 16×16×256 |
Conv-8 | 16×16×256 | 3×3×256-1-1 | 16×16×256 |
MaxPool-3 | 16×16×256 | 2×2-2-0 | 8×8×256 |
Dropout3 | 8×8×256 | — | 8×8×256 |
Conv-9 | 8×8×256 | 3×3×512-1-1 | 8×8×512 |
Conv-10 | 8×8×512 | 3×3×512-1-1 | 8×8×512 |
Conv-11 | 8×8×512 | 3×3×512-1-1 | 8×8×512 |
Conv-12 | 8×8×512 | 3×3×512-1-1 | 8×8×512 |
MaxPool-4 | 8×8×512 | 2×2-2-0 | 4×4×512 |
Dropout4 | 4×4×512 | — | 4×4×512 |
Conv-13 | 4×4×512 | 3×3×512-1-1 | 4×4×512 |
Conv-14 | 4×4×512 | 3×3×512-1-1 | 4×4×512 |
Conv-15 | 4×4×512 | 3×3×512-1-1 | 4×4×512 |
Conv-16 | 4×4×512 | 3×3×512-1-1 | 4×4×512 |
MaxPool-5 | 4×4×512 | 2×2-2-0 | 2×2×512 |
Dropout5 | 2×2×512 | — | 2×2×512 |
FC-1 | 2×2×512 | — | 500 |
Dropout6 | 1×500 | — | 1×500 |
表情分类 | CK+ | JAFFE |
---|---|---|
Angry(生气) | 45 | 31 |
Disgust(厌恶) | 59 | 29 |
Fear(恐惧) | 25 | 30 |
Happy(高兴) | 69 | 32 |
Sad(悲伤) | 28 | 30 |
Surprise(惊讶) | 83 | 31 |
Table 3 Number distribution of 6 types of expressions in CK+ and JAFFE datasets
表情分类 | CK+ | JAFFE |
---|---|---|
Angry(生气) | 45 | 31 |
Disgust(厌恶) | 59 | 29 |
Fear(恐惧) | 25 | 30 |
Happy(高兴) | 69 | 32 |
Sad(悲伤) | 28 | 30 |
Surprise(惊讶) | 83 | 31 |
算法 | 识别率/% | |
---|---|---|
CK+ | JAFFE | |
人工特征提取 | 88.14 | 87.32 |
CNN+LBP | 92.08 | 88.33 |
文献[ | 93.68 | 88.73 |
VGG16 | 95.00 | 90.86 |
文献[ | 95.40 | 92.10 |
GL-DCNN | 95.51 | 93.01 |
Table 4 Comparison of recognition performance of different algorithms on CK+ and JAFFE datasets
算法 | 识别率/% | |
---|---|---|
CK+ | JAFFE | |
人工特征提取 | 88.14 | 87.32 |
CNN+LBP | 92.08 | 88.33 |
文献[ | 93.68 | 88.73 |
VGG16 | 95.00 | 90.86 |
文献[ | 95.40 | 92.10 |
GL-DCNN | 95.51 | 93.01 |
处理器 | 训练时间/s | 识别率/% |
---|---|---|
CPU | 130.00 | 91.69 |
GPU | 2.00 | 92.71 |
Table 5 Comparison of training time and recognition rate using CPU and GPU
处理器 | 训练时间/s | 识别率/% |
---|---|---|
CPU | 130.00 | 91.69 |
GPU | 2.00 | 92.71 |
[1] |
YANG B, CAO J, NI R, et al. Facial expression recognition using weighted mixture deep neural network based on double-channel facial images[J]. IEEE Access, 2018, 6: 4630-4640.
DOI URL |
[2] |
WU B F, LIN C H. Adaptive feature mapping for customizing deep learning based facial expression recognition model[J]. IEEE Access, 2018, 6: 12451-12461.
DOI URL |
[3] | ZHANG T, ZHENG W M, CUI Z, et al. A deep neural network-driven feature learning method for multi-view facial expression recognition[J]. IEEE Transactions on Multime-dia, 2018, 18(12): 2528-2536. |
[4] | ZENG G, ZHOU J, JIA X, et al. Hand-crafted feature guided deep learning for facial expression recognition[C]// Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition, Xi’an, May 15-19, 2018. Was-hington: IEEE Computer Society, 2018: 423-430. |
[5] | WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// LNCS 11211: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 564-574. |
[6] |
孙晓, 潘汀. 基于兴趣区域深度神经网络的静态面部表情识别[J]. 电子学报, 2017, 45(5): 1189-1197.
DOI |
SUN X, PAN T. Static facial expression recognition system using ROI deep neural networks[J]. Acta Electronica Sinica, 2017, 45(5): 1189-1197. | |
[7] | WANG X JIN, LIU W, et al. Feature fusion of HOG and WLD for facial expression recognition[C]// Proceedings of the 2013 IEEE/SICE International Symposium on System Inte-gration, Kobe, Dec 15-17, 2013. Piscataway: IEEE, 2013: 227-232. |
[8] |
JIA Q, GAO X, GUO H, et al. Multi-layer sparse represen-tation for weighted LBP-patches based facial expression reco-gnition[J]. Sensors, 2015, 15(3): 6719.
DOI URL |
[9] | WANG L, LI R F, WANG K, et al. Feature representation for facial expression recognition based on FACS and LBP[J]. International Journal of Automation & Computing, 2014, 11(5): 459-468. |
[10] |
ZHOU J, ZHANG S, MEI H, et al. A method of facial exp-ression recognition based on Gabor and NMF[J]. Pattern Recognition and Image Analysis, 2016, 26(1): 119-124.
DOI URL |
[11] |
SILWAL R, ALSADOON A, PRASAD P W C, et al. A novel deep learning system for facial feature extraction by fusing CNN and MB-LBP and using enhanced loss function[J]. Multimedia Tools and Applications, 2020, 79(1): 1-21.
DOI URL |
[12] | SHIN M, KIM M, KWON D S. Baseline CNN structure analysis for facial expression recognition[C]// Proceedings of the 25th IEEE International Symposium on Robot and Human Interactive Communication, New York, Aug 26-31, 2016. Piscataway: IEEE, 2016: 724-729. |
[13] | CONNIE T, AL-SHABI M, CHEAH W P, et al. Facial ex-pression recognition using a hybrid CNN-SIFT aggregator[C]// LNCS 10607: Proceedings of the 11th International Work-shop on Multi-Disciplinary Trends in Artificial Intelligence, Gadong, Nov 20-22, 2017. Cham: Springer, 2017: 139-149. |
[14] |
ZOU G F, FU G X, GAO M L, et al. A new approach for small sample face recognition with pose variation by fusing gabor encoding features and deep features[J]. Multimedia Tools and Applications, 2020, 79(31): 23571-23598.
DOI URL |
[15] | SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409.1556, 2014. |
[16] | 杨恢先, 张翡, 陈永, 等. 结合改进CSLBP和位平面分解的单样本人脸识别[J]. 激光与光电子学进展, 2018, 55(7): 221-228. |
YANG H X, ZHANG F, CHEN Y, et al. Single-sample face recognition combining improved CSLBP and bit-plane de-composition[J]. Progress in Laser and Optoelectronics, 2018, 55(7): 221-228. | |
[17] | PATIL V, SARODE T. Modified CSLBP[J]. International Journal of Electrical and Computer Engineering, 2019, 9(4):2950-2959. |
[18] | 胡渝苹. 基于HOG-CSLBP与深度学习的跨年龄人脸识别算法[J]. 西南师范大学学报(自然科学版), 2020, 45(3): 115-120. |
HU Y P. A cross-age face recognition algorithm based on HOG-CSLBP and deep learning[J]. Journal of Southwest China Normal University (Natural Science Edition), 2020, 45(3): 115-120. | |
[19] | 董夙慧, 徐永刚. 基于HOG-CSLBP的快速行人检测算法[J]. 计算机工程与设计, 2018, 39(4): 1125-1129. |
DONG S H, XU Y G. Fast pedestrian detection algorithm based on HOG-CSLBP[J]. Computer Engineering and Design, 2018, 39(4): 1125-1129. | |
[20] | 翟海庆, 刘丹, 刘晙. 利用双流卷积神经网络的人脸表情识别方法[J]. 光学技术, 2020, 46(6): 712-720. |
ZHAI H Q, LIU D, LIU J. A facial expression recognition method based on dual-stream convolutional neural network[J]. Optical Technique, 2020, 46(6): 712-720. | |
[21] | LUCEY P, COHN J F, KANADE T J, et al. The extended Cohn-Kanade dataset (CK+) a complete dataset for action unit and emotion-specified expression[C]// Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Re-cognition, San Francisco, Jun 13-18, 2010. Washington: IEEE Computer Society, 2010: 94-101. |
[22] | 林克正, 白婧轩, 李昊天, 等. 深度学习下融合不同模型的小样本表情识别[J]. 计算机科学与探索, 2020, 14(3): 482-492. |
LIN K Z, BAI J X, LI H T, et al. Facial expression recognition with small samples fused with different models under deep learning[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(3): 482-492. | |
[23] |
LOPES A T, DEAGUIAR E, DESOUZA A F, et al. Facial expression recognition with convolutional neural networks: coping with few data and the training sample order[J]. Pattern Recognition, 2017, 61(1): 610-628.
DOI URL |
[24] | 张俞晴, 何宁, 魏润辰. 基于卷积神经网络融合SIFT特征的人脸表情识别[J]. 计算机应用与软件, 2019, 36(11): 161-167. |
ZHANG Y Q, HE N, WEI R C. Face expression recog-nition based on convolutional neural networks fusing SIFT features[J]. Computer Applications and Software, 2019, 36(11): 161-167. |
[1] | AN Fengping, LI Xiaowei, CAO Xiang. Medical Image Classification Algorithm Based on Weight Initialization-Sliding Window CNN [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1885-1897. |
[2] | HUANG Hao, GE Hongwei. Deep Residual Expression Recognition Network to Enhance Inter-class Discrimination [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1842-1849. |
[3] | ZENG Fanzhi, XU Luqian, ZHOU Yan, ZHOU Yuexia, LIAO Junwei. Review of Knowledge Tracing Model for Intelligent Education [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1742-1763. |
[4] | HONG Huiqun, SHEN Guiping, HUANG Fenghua. Summary of Expression Recognition Technology [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1764-1778. |
[5] | LIU Yi, LI Mengmeng, ZHENG Qibin, QIN Wei, REN Xiaoguang. Survey on Video Object Tracking Algorithms [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1504-1515. |
[6] | ZHAO Xiaoming, YANG Yijiao, ZHANG Shiqing. Survey of Deep Learning Based Multimodal Emotion Recognition [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1479-1503. |
[7] | XIA Hongbin, XIAO Yifei, LIU Yuan. Long Text Generation Adversarial Network Model with Self-Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1603-1610. |
[8] | PENG Hao, LI Xiaoming. Multi-scale Selection Pyramid Networks for Small-Sample Target Detection Algorithms [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1649-1660. |
[9] | ZHAO Yunji, FAN Cunliang, ZHANG Xinliang. Object Tracking Algorithm with Fusion of Multi-feature and Channel Awareness [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1417-1428. |
[10] | SUN Fangwei, LI Chengyang, XIE Yongqiang, LI Zhongbo, YANG Caidong, QI Jin. Review of Deep Learning Applied to Occluded Object Detection [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1243-1259. |
[11] | LIU Yafen, ZHENG Yifeng, JIANG Lingyi, LI Guohe, ZHANG Wenjie. Survey on Pseudo-Labeling Methods in Deep Semi-supervised Learning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1279-1290. |
[12] | ZHONG Mengyuan, JIANG Lin. Review of Super-Resolution Image Reconstruction Algorithms [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 972-990. |
[13] | XU Jia, WEI Tingting, YU Ge, HUANG Xinyue, LYU Pin. Review of Question Difficulty Evaluation Approaches [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 734-759. |
[14] | PEI Lishen, ZHAO Xuezhuan. Survey of Collective Activity Recognition Based on Deep Learning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 775-790. |
[15] | ZHAO Pengfei, XIE Linbo, PENG Li. Deep Small Object Detection Algorithm Integrating Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 927-937. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||
/D:/magtech/JO/Jwk3_kxyts/WEB-INF/classes/