Review of Bare Footprint Recognition

doi:10.3778/j.issn.1673-9418.2307048

Abstract

Abstract: Bare footprint recognition technology is a branch of image recognition technology, which plays an important role in criminal investigation, medical treatment and security fields, and is expected to become a new means of personal identification. However, this technology has not yet formed a relatively unified framework, nor has it established a standardized procedure. In order to provide guidance for future researchers, it is necessary to standardize the recognition process of different bare footprint images and summarize the relevant research of bare footprint recognition technology. Firstly, the background and significance of bare footprint recognition research are expounded. Then, the development history of this technology is reviewed, and the bare footprint images are divided into four categories according to different acquisition methods: ink stamped bare footprint images, plantar scanning images, footprint images acquired by optical footprint acquisition equipment and foot pressure images acquired by footprint pressure acquisition system. It is pointed out that the latter two images are the hot spots of bare footprint recognition research at present. Then, the research status of bare footprint recognition technology is analyzed from three aspects: dataset, image preprocessing and recognition methods. Among them, the recognition methods are divided into traditional methods and deep learning-based methods, and the latter is further divided into network structure innovation methods and loss function optimization methods. The evaluation indices of identification methods are given, and various methods are compared from many aspects. Finally, the problems faced by this technology are pointed out, and its future development direction is prospected.

Key words: image recognition, bare footprint, personal identification, footprint images, foot pressure images, deep learning

摘要： 赤足足迹识别技术是图像识别技术的一个分支，在刑侦、医疗以及安全领域发挥着重要作用，有望成为一种新的进行人身识别的手段。但是该技术尚未形成较为统一的框架，也没有一个规范化的流程。为了给今后的研究人员提供指导，需要规范不同足迹图像的识别流程，并对赤足足迹识别技术相关研究进行归纳与总结。首先对赤足足迹识别研究的背景和意义进行阐述，然后回顾该技术的发展脉络，并根据采集方式的不同将赤足足迹图像分为油墨捺印足迹图像、足底扫描图像、光学足迹采集设备采集的足迹图像以及足迹压力采集系统采集的足压图像四类，并指出后两种图像是目前赤足足迹识别研究的热点。之后分别从赤足足迹数据集、图像预处理、识别方法三个方面分析赤足足迹识别技术的研究现状。其中，识别方法分为传统方法和基于深度学习的方法，后者又进一步划分为网络结构创新方法和损失函数优化方法。在给出识别方法的评价指标后，从多个方面对各种方法进行对比。最后指出该技术目前面临的问题，并对其今后的发展方向进行展望。

关键词: 图像识别, 赤足足迹, 人身识别, 足迹图像, 足压图像, 深度学习

WANG Kun, GUO Wei, WANG Zunyan, HAN Wenqiang. Review of Bare Footprint Recognition[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(1): 44-57.

王昆, 郭威, 王尊严, 韩文强. 赤足足迹识别研究综述[J]. 计算机科学与探索, 2024, 18(1): 44-57.

References

[1] KENNEDY R B, PRESSMAN I S, CHEN S, et al. Statis-tical analysis of barefoot impressions[J]. Journal of Forensic Science, 2003, 48(1): 1-9.
[2] 许爱东, 糜忠良. 足迹鉴定技术理论与实务研究[M]. 北京: 法律出版社, 2018: 1-14.
XU A D, MI Z L. Research on the theory and practice of footprint identification technology[M]. Beijing: Law Press, 2018: 1-14.
[3] 金益锋, 于霄雪, 王丽, 等. 基于多尺度特征的赤足足迹图像人身识别算法[J]. 刑事技术, 2022, 47(6): 587-592.
JIN Y F, YU X X, WANG L, et al. Algorithm of personal recognition based on multi-scale features from barefoot foot-print image[J]. Forensic Science and Technology, 2022, 47(6): 587-592.
[4] IBRAHIM Y I, ALHAMDANI I M. A hyprid technique for human footprint recognition[J]. International Journal of Electrical and Computer Engineering, 2019, 9(5): 4060-4068.
[5] 方敏. 基于小样本学习的赤足足迹分类方法研究[D]. 合肥: 安徽大学, 2020.
FANG M. Research on bare footprint classification method based on small sample learning[D]. Hefei: Anhui Univer-sity, 2020.
[6] 汪桐生. 赤足足迹压力图像的检索算法研究[D]. 合肥: 安徽大学, 2021.
WANG T S. Research on retrieval algorithm of barefoot footprint pressure image[D]. Hefei: Anhui University, 2021.
[7] 王申. 足迹分析检验系统的设计与实验[D]. 扬州: 扬州大学, 2019.
WANG S. Design and experiment of footprint analysis and inspection system[D]. Yangzhou: Yangzhou University, 2019.
[8] 鲍文霞, 茅丽丽, 王年, 等. 基于注意力双分支网络的跨模态足迹检索[J]. 东南大学学报(自然科学版), 2021, 51(5): 914-922.
BAO W X, MAO L L, WANG N, et al. Cross-modal foot-print retrieval based on the two-branch CNN with attention[J]. Journal of Southeast University (Natural Science Edition), 2021, 51(5): 914-922.
[9] WANG X N, WANG H Y, CHENG Q, et al. Single 2D pre-ssure footprint based person identification[C]//Proceedings of the 2017 IEEE International Joint Conference on Bio-metrics, Denver, Oct 1-4, 2017. Piscataway: IEEE, 2017: 413-419.
[10] 鲍文霞, 王云飞, 王年, 等. 基于度量学习核函数的光学足迹图像识别算法[J]. 华中科技大学学报(自然科学版), 2020, 48(11): 11-16.
BAO W X, WANG Y F, WANG N, et al. Identification algorithm of optical footprint image based on metric learn-ing kernel function[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2020, 48(11): 11-16.
[11] 史力民. 足迹检验技术规范[M]. 北京: 中国人民公安大学出版社, 2012: 1-57.
SHI L M. Technical specifications for footprints examin-ation[M]. Beijing: People??s Public Security University of China Press, 2012: 1-57.
[12] 雷航, 童莉, 平西建. 平面赤足迹特征分析与身份识别方法[J]. 计算机辅助设计与图形学学报,?2008, 20(5): 659-664.
LEI H, TONG L, PING X J. Feature analysis and ident-ification method of plane bare footprint[J]. Journal of Computer-Aided Design & Computer Graphics, 2008, 20(5): 659-664.
[13] 童莉. 平面赤足迹形状分析与身份鉴别研究[D]. 郑州: 解放军信息工程大学, 2007.
TONG L. Study on shape analysis and identity identifi-cation of plane bare footprint[D]. Zhengzhou: The PLA Information Engineering University, 2007.
[14] 李磊. 平面赤足迹形态特征提取与分析[D]. 郑州: 解放军信息工程大学, 2006.
LI L. Extraction and analysis of morphological features of plane bare footprint[D]. Zhengzhou: The PLA Information Engineering University, 2006.
[15] 王永栋, 顾士清, 党素琴, 等. 基于重压面形状分析的足迹识别系统的研究[C]//第十四届全国图象图形学学术会议论文集. 北京: 清华大学出版社, 2008: 794-797.
WANG Y D, GU S Q, DANG S Q, et al. Research on foot-print recognition system based on shape analysis of heavy pressure surface[C]//Proceedings of the 14th National Conference on Image Graphics. Beijing: Tsinghua Univer-sity Press, 2008: 794-797.
[16] 王鹏鹏. 基于相似度分析的2D光学赤足足迹识别方法研究[D]. 合肥: 安徽大学, 2021.
WANG P P. Research on 2D optical barefoot footprint recognition method based on similarity analysis[D]. Hefei: Anhui University, 2021.
[17] 鲍文霞, 胡伟, 王年, 等. 基于深度中心匹配哈希网络的足迹压力图像检索[J]. 华中科技大学学报(自然科学版), 2023(9): 81-87.
BAO W X, HU W, WANG N, et al. Footprint pressure image retrieval based on deep center matching hash network[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2023(9): 81-87.
[18] 王鹏鹏, 吴洛天, 汪曙光, 等. 基于关系网络的赤足足迹识别[J]. 传感器与微系统, 2021, 40(4): 126-130.
WANG P P, WU L T, WANG S G, et al. Bare footprint recognition based on relation network[J]. Transducer and Microsystem Technologies, 2021, 40(4): 126-130.
[19] 梁栋, 高玮玮, 张艳, 等. 基于足底压力图像的静态触觉步态识别[J]. 华中科技大学学报(自然科学版), 2013, 41(10): 25-29.
LIANG D, GAO W W, ZHANG Y, et al. Static tactile gait recognition based on plantar pressure image[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2013, 41(10): 25-29.
[20] 鲍文霞, 茅丽丽, 王年, 等. 非局部注意力双分支网络的跨模态赤足足迹检索[J]. 中国图象图形学报, 2022, 27(7): 2199-2213.
BAO W X, MAO L L, WANG N, et al. Non-local attention dual-branch network based cross-modal barefoot footprint retrieval[J]. Journal of Image and Graphics, 2022, 27(7): 2199-2213.
[21] PAWARA P, OKAFOR E, SCHOMAKER L, et al. Data augmentation for plant classification[C]//Proceedings of the 2017 International Conference on Advanced Concepts for Intelligent Vision Systems, Brussels, Sep 18-21, 2017. Cham: Springer, 2017: 615-626.
[22] ZHONG Z, ZHENG L, KANG G, et al. Random erasing data augmentation[C]//Proceedings of the 2020 AAAI Conference on Artificial Intelligence, New York, Feb 7-12, 2020. Menlo Park: AAAI, 2020: 13001-13008.
[23] HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786): 504-507.
[24] 郑远攀, 李广阳, 李晔. 深度学习在图像识别中的应用研究综述[J]. 计算机工程与应用, 2019, 55(12): 20-36.
ZHENG Y P, LI G Y, LI Y. Survey of application of deep learning in image recognition[J]. Computer Engineering and Applications, 2019, 55(12): 20-36.
[25] 王颢. 深度学习在图像识别中的研究与应用[J]. 科技视界, 2020(24): 37-38.
WANG H. Research and application of deep learning in image recognition[J]. Science & Technology Vision, 2020(24): 37-38.
[26] BAO W X, WANG Y F, WANG N, et al. Optical footprint image recognition algorithm based on metric learning and SVM[C]//Proceedings of the 2020 International Conference on Computer Engineering and Application, Sanya, Oct 20-22, 2020: 864-868.
[27] SELESNICK I W, BARANIUK R G, KINGSBURY N C. The dual-tree complex wavelet transform[J]. IEEE Signal Processing Magazine, 2005, 22(6): 123-151.
[28] ZHANG L, LI D, ZHONG C. Collaborative optimization of clustering by fuzzy c-means and weight determination by ReliefF[C]//Proceedings of the 2009 6th International Conf-erence on Fuzzy Systems and Knowledge Discovery, Tianjin, Aug 14-16, 2009: 454-459.
[29] AHONEN T, RAHTU E, OJANSIVU V, et al. Recognition of blurred faces using local phase quantization[C]//Proceedings of the 2008 19th International Conference on Pattern Recognition, Tampa, Dec 8-11, 2008: 1-4.
[30] XIAO Y, CAO Z G, WANG L. Local phase quantization plus: a principled method for embedding local phase quantization into Fisher vector for blurred image recog-nition[J]. Information Sciences, 2017, 420: 77-95.
[31] HART P E, STORK D G, DUDA R O. Pattern classification[M]. Hoboken: Wiley, 2000: 1-30.
[32] BLANCO-DELGADO N, DE HAAG M U. Multipath analysis using code-minus-carrier for dynamic testing of GNSS rece-ivers[C]//Proceedings of the 2011 International Conference on Localization and GNSS, Tampere, Jun 29-30, 2011: 25-30.
[33] 汪飞跃, 姚志明, 许胜强, 等. 基于柔性力敏传感器的左右脚动态识别方法[J]. 传感技术学报, 2015, 28(7): 964-971.
WANG F Y, YAO Z M, XU S Q, et al. Dynamic footprint recognition method based on flexible force-sensitive sensor[J]. Chinese Journal of Sensors and Actuators, 2015, 28(7): 964-971.
[34] HEYDARZADEH M, BIRJANDTALAB J, POUYAN M B, et al. Gaits analysis using pressure image for subject identification[C]//Proceedings of the 2017 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI), Orlando, Feb 16-19, 2017: 333-336.
[35] 张艳, 王乔, 王年, 等. 基于多模特征的足迹识别算法[J]. 华中科技大学学报(自然科学版), 2019, 47(5): 73-78.
ZHANG Y, WANG Q, WANG N, et al. Footprint recog-nition algorithm based on multi-modal features[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2019, 47(5): 73-78.
[36] 孙永胜. 基于多尺度重排填充机制的足迹识别方法研究[D]. 合肥: 安徽大学, 2022.
SUN Y S. Research on footprint recognition method based on multi-scale rearrangement and filling mechanism[D]. Hefei: Anhui University, 2022.
[37] ZHANG J. Seesawfacenets: sparse and robust face verific-ation model for mobile platform[J]. arXiv:1908.09124, 2019.
[38] 郑治. 基于卷积神经网络的赤足足迹生物信息挖掘技术研究[D]. 长春: 吉林大学, 2022.
ZHENG Z. Research on bare footprint bioinformatics mining technology based on convolutional neural network[D]. Changchun: Jilin University, 2022.
[39] MA N, ZHANG X, ZHENG H T, et al. Shufflenet v2: practical guidelines for efficient CNN architecture design[C]//Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018: 116-131.
[40] 朱明, 江畅, 于小勇, 等. 基于深度度量学习的足迹图像检索算法[J]. 刑事技术, 2023, 48(3): 283-291.
ZHU M, JIANG C, YU X Y, et al. Footprint image retrieval algorithm based on depth metric learning[J]. Forensic Science and Technology, 2023, 48(3): 283-291.
[41] DAI Z, CHEN M, GU X, et al. Batch dropblock network for person re-identification and beyond[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 3, 2019: 3691-3701.
[42] KEATSAMARN T, PINTAVIROOJ C. Footprint identifi-cation using deep learning[C]//Proceedings of the 2018 11th Biomedical Engineering International Conference, Aachen, May 23-25, 2018: 1-4.
[43] 许秋菊. 自然行走和负重条件下的赤足压力图像识别算法研究[D]. 合肥: 安徽大学, 2022.
XU Q J. Research on barefoot pressure image recognition algorithm under natural walking and load-bearing condi-tions[D]. Hefei: Anhui University, 2022.
[44] 朱明, 汪桐生, 王年, 等. 基于多尺度自注意卷积的足迹压力图像检索算法[J]. 模式识别与人工智能, 2020, 33(12): 1097-1103.
ZHU M, WANG T S, WANG N, et al. Footprint pressure image retrieval algorithm based on multi-scale self-attention convolution[J]. Pattern Recognition and Artificial Intellig-ence, 2020, 33(12): 1097-1103.
[45] 瞿金杰. 基于卷积神经网络的压力赤足足迹识别算法研究[D]. 合肥: 安徽大学, 2021.
QU J J. Research on recognition algorithm of pressure barefoot footprint based on convolutional neural network[D]. Hefei: Anhui University, 2021.
[46] CHANG C H, YU C H, CHEN S Y, et al. KG-GAN: knowl-edge-guided generative adversarial networks[J]. arXiv:1905. 12261, 2019.
[47] 高梓健. 基于多级融合分布图网络的赤足压力足迹分类方法研究[D]. 合肥: 安徽大学, 2022.
GAO Z J. Research on barefoot pressure footprint classif-ication method based on multi-level fusion distribution net-work[D]. Hefei: Anhui University, 2022.
[48] 鲍文霞, 瞿金杰, 王年, 等. 基于空间聚合加权卷积神经网络的力触觉足迹识别[J]. 东南大学学报(自然科学版), 2020, 50(5): 959-964.
BAO W X, QU J J, WANG N, et al. Force-tactile footprint recognition based on spatial aggregation weighted convo-lutional neural network[J]. Journal of Southeast University (Natural Science Edition), 2020, 50(5): 959-964.
[49] 吴正建. 串行足底压力图像的检索算法研究[D]. 合肥: 安徽大学, 2022.
WU Z J. Research on retrieval algorithm of serial plantar pressure image[D]. Hefei: Anhui University, 2022.
[50] WENXIA B, WEI H, Dong L, et al. Deep supervised binary hash codes for footprint image retrieval[C]//Proceedings of the 2020 International Conference on Intelligent Compu-ting and Human-Computer Interaction, Sanya, Dec 4-6, 2020: 138-141.
[51] 胡伟. 基于卷积神经网络的赤足足迹检索算法研究[D]. 合肥: 安徽大学, 2022.
HU W. Research on bare footprint retrieval algorithm based on convolutional neural network[D]. Hefei: Anhui Unive-rsity, 2022.
[52] 丁汉, 唐云祁, 郭威. 自然行走状态下的足底压力稳定性研究[J]. 计算机技术与发展, 2017, 27(4): 153-156.
DING H, TANG Y Q, GUO W. Research on stability of plantar pressure in normal human walking condition[J]. Computer Technology and Development, 2017, 27(4): 153-156.