Research Progress in Application of Deep Learning in Animal Behavior Analysis

doi:10.3778/j.issn.1673-9418.2306033

Abstract

Abstract: In recent years, animal behavior analysis has become one of the most important methods in the fields of neuroscience and artificial intelligence. Taking advantage of the powerful deep-learning-based image analysis technology, researchers have developed state-of-the-art automatic animal behavior analysis methods with complex functions. Compared with traditional methods of animal behavior analysis, special labeling is not required in these methods, animal pose can be efficiently estimated and tracked. These methods like in a natural environment, which hold the potential for complex animal behavior experiments. Therefore, the application of deep learning in animal behavior analysis is reviewed. Firstly, this paper analyzes the tasks and current status of animal behavior analysis. Then, it highlights and compares existing deep learning-based animal behavior analysis tools. According to the dimension of experimental analysis, the deep learning-based animal behavior analysis tools are divided into two-dimensional animal behavior analysis tools and three-dimensional animal behavior analysis tools, and the functions, performance and scope of application of tools are discussed. Furthermore, the existing animal datasets and evaluation metrics are introduced, and the algorithm mechanism used in the existing animal behavior analysis tool is summarized from the advantages, limitations and applicable scenarios. Finally, the deep learning-based animal behavior analysis tools are prospected from the aspects of dataset, experimental paradigm and low latency.

Key words: animal behavior analysis methods, deep learning, animal pose estimation

摘要： 近年来动物行为分析已成为脑科学与人工智能等领域的重要研究手段之一，研究者基于深度学习的图像分析技术，构建了自动化、智能化的动物行为分析方法。相较于传统的动物行为分析方法，该类方法无需对动物进行特殊标记，可高效地对动物的姿态进行估计和跟踪，实验情景贴近自然情况，为复杂的动物行为实验提供了潜在可能性。对深度学习在动物行为分析中的应用研究进行综述，首先简要分析动物行为分析的任务及现状；然后重点介绍并比较现有的基于深度学习的动物行为分析工具，根据实验分析的行为维度，将该类工具分为二维的动物行为分析工具和三维的动物行为分析工具，并对工具的功能、性能以及适用范围进行论述；进而介绍了现有的动物数据集和评价指标，对现有的动物行为分析工具所利用的算法机制从优势、局限性、适用场景做出总结；最后，从数据集、实验范式和低延时性等方面对基于深度学习的动物行为分析工具做出展望。

关键词: 动物行为分析方法, 深度学习, 动物姿态估计

SHEN Tong, WANG Shuo, LI Meng, QIN Lunming. Research Progress in Application of Deep Learning in Animal Behavior Analysis[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(3): 612-626.

申通, 王硕, 李孟, 秦伦明. 深度学习在动物行为分析中的应用研究进展[J]. 计算机科学与探索, 2024, 18(3): 612-626.

References

[1] PEREIRA T D, SHAEVITZ J W, MURTHY M. Quantifying behavior to understand the brain[J]. Nature Neuroscience, 2020, 23(12): 1537-1549.
[2] FORKOSH O. Animal behavior and animal personality from a non-human perspective: getting help from the machine[J]. Patterns, 2021, 2(3): 100194.
[3] WEI K, KORDING K P. Behavioral tracking gets real[J]. Nature Neuroscience, 2018, 21(9): 1146-1147.
[4] XIA F, KHEIRBEK M A. Circuit-based biomarkers for mood and anxiety disorders[J]. Trends in Neurosciences, 2020, 43(11): 902-915.
[5] WEBER R Z, MULDERS G, KAISER J, et al. Deep learning-based behavioral profiling of rodent stroke recovery[J]. BMC Biology, 2022, 20(1): 1-19.
[6] ANDERSON D J, PERONA P. Toward a science of computational ethology[J]. Neuron, 2014, 84(1): 18-31.
[7] 孙秀萍, 王琼, 石哲, 等. 动物行为实验方法学研究的回顾与展望[J]. 中国比较医学杂志, 2018, 28(3): 1-7.
SUN X P, WANG Q, SHI Z，et al. Review and prospect of experiment methodology on animal behavior[J]. Chinese Journal of Comparative Medicine, 2018, 28(3): 1-7.
[8] TINBERGEN N. On aims and methods of ethology[J]. Zeitschrift für Tierpsychologie, 1963, 20(4): 410-433.
[9] BALA M M, VASUNDHARA D N, HARITHA A, et al. Design, development and performance analysis of cognitive assisting aid with multi sensor fused navigation for visually impaired people[J]. Journal of Big Data, 2023, 10(1): 21.
[10] TASGAONKAR P P, GARG R D, GARG P K. Vehicle detection and traffic estimation with sensors technologies for intelligent transportation systems[J]. Sensing and Imaging, 2020, 21: 1-28.
[11] KOYDEMIR H C, OZCAN A. Wearable and implantable sensors for biomedical applications[J]. Annual Review of Analytical Chemistry, 2018, 11: 127-146.
[12] PATEL S, PARK H, BONATO P, et al. A review of wearable sensors and systems with application in rehabilitation[J]. Journal of Neuroengineering and Rehabilitation, 2012, 9(1): 1-17.
[13] INTILLE S S, LESTER J, SALLIS J F, et al. New horizons in sensor development[J]. Medicine and Science in Sports and Exercise, 2012, 44: 24-31.
[14] PAPANDREOU G, ZHU T, KANAZAWA N, et al. Towards accurate multi-person pose estimation in the wild[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2017: 4903-4911.
[15] LI W, WANG Z, YIN B, et al. Rethinking on multi-stage networks for human pose estimation[J]. arXiv:1901.00148, 2019.
[16] WEI S E, RAMAKRISHNA V, KANADE T, et al. Convolutional pose machines[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 4724-4732.
[17] CHEN Y, WANG Z, PENG Y, et al. Cascaded pyramid network for multi-person pose estimation[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 7103-7112.
[18] XIAO B, WU H, WEI Y. Simple baselines for human pose estimation and tracking[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 466-481.
[19] SUN K, XIAO B, LIU D, et al. Deep high-resolution representation learning for human pose estimation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 5693-5703.
[20] XU Y, ZHANG J, ZHANG Q, et al. ViTPose: simple vision transformer baselines for human pose estimation[C]//Advances in Neural Information Processing Systems 35, New Orleans, Nov 28-Dec 9, 2022: 38571-38584.
[21] NEWELL A, YANG K, DENG J. Stacked hourglass networks for human pose estimation[C]//Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 483-499.
[22] CHENG B, XIAO B, WANG J, et al. HigherHRNet: scale-aware representation learning for bottom-up human pose estimation[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2020: 5386-5395.
[23] TOSHEV A, SZEGEDY C. DeepPose: human pose estimation via deep neural networks[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2014: 1653-1660.
[24] CAO Z, SIMON T, WEI S E, et al. Realtime multi-person 2D pose estimation using part affinity fields[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society,2017: 7291-7299.
[25] ZHENG C, ZHU S, MENDIETA M, et al. 3D human pose estimation with spatial and temporal transformers[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 11656-11665.
[26] LI W, LIU H, TANG H, et al. MHFormer: multi-hypothesis transformer for 3D human pose estimation[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 13147-13156.
[27] MOON G, CHANG J Y, LEE K M. Camera distance-aware top-down approach for 3D multi-person pose estimation from a single RGB image[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE, 2019: 10133-10142.
[28] CHENG Y, WANG B, YANG B, et al. Graph and temporal convolutional networks for 3D multi-person pose estimation in monocular videos[C]//Proceedings of the 2021 AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2021: 1157-1165.
[29] CHENG Y, WANG B, YANG B, et al. Monocular 3D multi-person pose estimation by integrating top-down and bottom-up networks[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 7649-7659.
[30] ZHANG Z, WANG C, QIU W, et al. Adafuse: adaptive multiview fusion for accurate human pose estimation in the wild[J]. International Journal of Computer Vision, 2021, 129: 703-718.
[31] MA H, CHEN L, KONG D, et al. Transfusion: cross-view fusion with transformer for 3D human pose estimation[J]. arXiv:2110.09554, 2021.
[32] ZHANG Y, SUN P, JIANG Y, et al. Bytetrack: multi-object tracking by associating every detection box[C]//Proceedings of the 17th European Conference on Computer Vision. Cham: Springer, 2022: 1-21.
[33] QIU X, SUN X, CHEN Y, et al. Pedestrian detection and counting method based on YOLOv5+DeepSORT[C]//Proceedings of the 4th International Symposium on Power Electronics and Control Engineering, Nanchang, Sep 16-19, 2021: 177-181.
[34] AHARON N, ORFAIG R, BOBROVSKY B Z. BoT-SORT: robust associations multi-pedestrian tracking[J]. arXiv:2206. 14651, 2022.
[35] DU Y, ZHAO Z, SONG Y, et al. StrongSORT: make deepsort great again[J]. IEEE Transactions on Multimedia, 2023,25: 8725-8737.
[36] WENG S K, KUO C M, TU S K. Video object tracking using adaptive Kalman filter[J]. Journal of Visual Communication & Image Representation, 2006(6): 17.
[37] ZHANG L, LI Y, NEVATIA R. Global data association for multi-object tracking using network flows[C]//Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2008: 1-8.
[38] HE S, LUO H, WANG P, et al. TransReID: transformer-based object re-identification[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 15013-15022.
[39] DICLE C, CAMPS O I, SZNAIER M. The way they move: tracking multiple targets with similar appearance[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision. Washington: IEEE Computer Society, 2013: 2304-2311.
[40] ZHENG L, SHEN L, LU T， et al. Scalable person re-identification: a benchmark[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Washington:IEEE Computer Society, 2015: 1116-1124.
[41] 袁姮, 赵肖祎. 流形背景感知的相关滤波目标跟踪[J]. 计算机科学与探索, 2023, 17(6): 1373-1386.
YUAN H, ZHAO X Y. Manifold background-aware correlation filter target tracking[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(6): 1373-1386.
[42] XU T, ZHU X F, WU X J. Learning spatio-temporal discriminative model for affine subspace based visual object tracking[J]. Visual Intelligence, 2023, 1(1): 4.
[43] 牛宇辉, 奚峥皓, 薛亚静, 等. 基于光流运动约束马尔可夫随机场改进模型的多目标跟踪算法[J]. 激光与光电子学进展, 2022, 59(2): 8.
NIU Y H, XI Z H, XUE Y J, et al. Multi-objective tracking algorithm based on improved model of optical flow motion constrained Markov random field[J]. Laser & Optoelectronics Progress, 2022, 59(2): 8.
[44] LEONIDA K L, SEVILLA K V, MANLISES C O. A motion-based tracking system using the Lucas-Kanade optical flow method[C]//Proceedings of the 2022 14th International Conference on Computer and Automation Engineering. Piscataway: IEEE, 2022: 86-90.
[45] LIU X L, YU S, FLIERMAN N A, et al. OptiFlex: multi-frame animal pose estimation combining deep learning with optical flow[J]. Frontiers in Cellular Neuroscience, 2021, 15: 621252.
[46] DOORNWEERD J E, KOOTSTRA G, VEERKAMP R F, et al. Across-species pose estimation in poultry based on images using deep learning[J]. Frontiers in Animal Science, 2021, 2: 791290.
[47] JIANG L, LEE C, TEOTIA D, et al. Animal pose estimation: a closer look at the state-of-the-art, existing gaps and opportunities[J]. Computer Vision and Image Understanding, 2022, 222: 103483.
[48] ARNK?RN B, SCHOELER S, ULLAH M, et al. Deep learning based multiple animal pose estimation[J]. Electronic Imaging, 2022, 34(6): 1-6.
[49] 何泉, 吴磊, 柳珑. 机器学习在动物行为分析中的应用研究进展[J]. 生命科学研究, 2022, 26(5): 433-440.
HE Q, WU L, LIU L. Research progress on the application of machine learning in animal behavior analysis[J]. Chinese Journal of Life Science Research, 2022, 26(5): 433-440.
[50] MATHIS M W, MATHIS A. Deep learning tools for the measurement of animal behavior in neuroscience[J]. Current Opinion in Neurobiology, 2020, 60: 1-11.
[51] 张红民, 庄旭, 郑敬添, 等. 优化 YOLO 网络的人体异常行为检测方法[J]. 计算机工程与应用, 2023, 59(7): 242-249.
HANG H M, ZHUANG X, ZHENG J T, et al. Optimizing human abnormal behavior detection method of YOLO network[J]. Computer Engineering and Applications, 2023, 59(7): 242-249.
[52] 何儒汉, 熊捷繁, 熊明福. 基于背景自适应学习的行人重识别算法研究[J]. 计算机工程与应用, 2023, 59(7): 126-133.
HE R H, XIONG J F, XIONG M F. Research on person re-identification based on background adaptive learning[J]. Computer Engineering and Applications, 2023, 59(7): 126-133.
[53] 何坚, 郭泽龙, 刘乐园, 等. 基于滑动窗口和卷积神经网络的可穿戴人体活动识别技术[J]. 电子与信息学报, 2022, 44(1): 168-177.
HE J, GUO Z L, LIU L Y, et al. Wearable human activity recognition technology based on sliding window and convolutional neural network[J]. Journal of Electronics & Information Technology, 2022, 44(1): 168-177.
[54] NAIK H, CHAN A H H, YANG J, et al. 3D-POP—an automated annotation approach to facilitate markerless 2D-3D tracking of freely moving birds with marker-based motion capture[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2023: 21274-21284.
[55] LAUER J, ZHOU M, YE S, et al. Multi-animal pose estimation, identification and tracking with DeepLabCut[J]. Nature Methods, 2022, 19(4): 496-504.
[56] GRAVING J M, CHAE D, NAIK H, et al. DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning[J]. eLife, 2018: e47994.
[57] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 4700-4708.
[58] ZENG F, DONG B, ZHANG Y, et al. MOTR: end-to-end multiple-object tracking with transformer[C]//Proceedings of the 17th European Conference on Computer Vision. Cham: Springer, 2022: 659-675.
[59] PEREIRA T D, TABRIS N, MATSLIAH A, et al. SLEAP: a deep learning system for multi-animal pose tracking[J]. Nature Methods, 2022, 19(4): 486-495.
[60] MIHAJLOVIC M, ZHANG Y, BLACK M J, et al. LEAP: learning articulated occupancy of people[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 10461-10471.
[61] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Oct 5-9, 2015. Cham: Springer, 2015: 234-241.
[62] HU Y, FERRARIO C R, MAITLAND A D, et al. LabGym: quantification of user-defined animal behaviors using learning-based holistic assessment[J]. Cell Reports Methods, 2023, 3(3).
[63] 田皓宇, 马昕, 李贻斌. 基于骨架信息的异常步态识别方法[J]. 吉林大学学报(工学版), 2022, 52(4): 725-737.
TIAN H Y, MA X, LI Y B. Abnormal gait recognition method based on skeleton information[J]. Journal of Jilin University (Engineering Science), 2022, 52(4): 725-737.
[64] SHEPPARD K, GARDIN J, SABNIS G S, et al. Gait-level analysis of mouse open field behavior using deep learning-based pose estimation[J]. bioRxiv, 2020. DOI: 10.1101/2020. 12.29.424780.
[65] SEGALIN C, WILLIAMS J, KARIGO T, et al. The mouse action recognition system (MARS) software pipeline for automated analysis of social behaviors in mice[J]. eLife, 2021, 10: e63720.
[66] EIGEN D, PUHRSCH C, FERGUS R. Depth map prediction from a single image using a multi-scale deep network[C]//Advances in Neural Information Processing Systems 27, Montreal, Dec 8-13, 2014: 2366-2374.
[67] PéREZ-ESCUDERO A, VICENTE-PAGE J, HINZ R C, et al. idTracker: tracking individuals in a group by automatic identification of unmarked animals[J]. Nature Methods, 2014, 11(7): 743-748.
[68] RODRIGUEZ A, ZHANG H, KLAMINDER J, et al. ToxTrac: a fast and robust software for tracking organisms[J]. Methods in Ecology and Evolution, 2018, 9(3): 460-464.
[69] ROMERO-FERRERO F, BERGOMI M G, HINZ R C, et al. Idtracker.ai: tracking all individuals in small or large collectives of unmarked animals[J]. Nature Methods, 2019, 16(2): 179-182.
[70] XU Z, CHENG X E. Zebrafish tracking using convolutional neural networks[J]. Scientific Reports, 2017, 7(1): 42815.
[71] BORGMEIER C, LOMAN S L, STRICKLAND-COHEN M K. ABC tracker: increasing teacher capacity for assessing student behavior[J]. Beyond Behavior, 2017, 26(3): 113-123.
[72] SCHWEIHOFF J F, LOSHAKOV M, PAVLOVA I, et al. DeepLabStream enables closed-loop behavioral experiments using deep learning-based markerless, real-time posture detection[J]. Communications Biology, 2021, 4(1): 130.
[73] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 770-778.
[74] 王仕宸, 黄凯, 陈志刚, 等. 深度学习的三维人体姿态估计综述[J]. 计算机科学与探索, 2023, 17(1): 74-87.
WANG S C, HUANG K, CHEN Z G, et al. Survey on 3D human pose estimation of deep learning[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(1): 74-87.
[75] MARSHALL J D, ALDARONDO D E, DUNN T W, et al. Continuous whole-body 3D kinematic recordings across the rodent behavioral repertoire[J]. Neuron, 2021, 109(3): 420-437.
[76] SCHNEIDER A, ZIMMERMANN C, ALYAHYAY M, et al. 3D pose estimation enables virtual head fixation in freely moving rats[J]. Neuron, 2022, 110(13): 2080-2093.
[77] NATH T, MATHIS A, CHEN A C, et al. Using DeepLabCut for 3D markerless pose estimation across species and behaviors[J]. Nature Protocols, 2019, 14(7): 2152-2176.
[78] GüNEL S, RHODIN H, MORALES D, et al. DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult drosophila[J]. eLife, 2019, 8: e48571.
[79] HUANG K, HAN Y, CHEN K, et al. A hierarchical 3D-motion learning framework for animal spontaneous behavior mapping[J]. Nature Communications, 2021, 12(1): 1-14.
[80] HAN Y, HUANG K, CHEN K, et al. MouseVenue3D: a markerless three-dimension behavioral tracking system for matching two-photon brain imaging in free-moving mice[J]. Neuroscience Bulletin, 2022, 38(3): 303-317.
[81] LI T, SEVERSON K S, WANG F, et al. Improved 3D markerless mouse pose estimation using temporal semi-supervision[J]. International Journal of Computer Vision, 2023, 131(6): 1389-1405.
[82] QIU H, WANG C, WANG J, et al. Cross view fusion for 3D human pose estimation[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 4342-4351.
[83] SHUKLA S, ARAC A. A step-by-step implementation of DeepBehavior, deep learning toolbox for automated behavior analysis[J]. Journal of Visualized Experiments, 2020, 156: e60763.
[84] BALA P C, EISENREICH B R, YOO S B M, et al. Automated markerless pose estimation in freely moving macaques with OpenMonkeyStudio[J]. Nature Communications, 2020, 11(1): 4560.
[85] KARASHCHUK P, RUPP K L, DICKINSON E S, et al. Anipose: a toolkit for robust markerless 3D pose estimation[J]. Cell Reports, 2021, 36(13): 109730.
[86] FU H, GONG M, WANG C, et al. Deep ordinal regression network for monocular depth estimation[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 2002-2011.
[87] DUNN T W, MARSHALL J D, SEVERSON K S, et al. Geometric deep learning enables 3D kinematic profiling across species and environments[J]. Nature Methods, 2021, 18(5): 564-573.
[88] GOSZTOLAI A, GüNEL S, LOBATO-RíOS V, et al. LiftPose3D, a deep learning-based approach for transforming two-dimensional to three-dimensional poses in laboratory animals[J]. Nature Methods, 2021, 18(8): 975-981.
[89] ZIMMERMANN C, SCHNEIDER A, ALYAHYAY M, et al. FreiPose: a deep learning framework for precise animal motion capture in 3D spaces[J]. bioRxiv, 2020. DOI: 10.1101/2020.02.27.967620.
[90] ZUFFI S, KANAZAWA A, JACOBS D W, et al. 3D menagerie: modeling the 3D shape and pose of animals[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2017: 6365-6373.
[91] LOPER M, MAHMOOD N, ROMERO J, et al. SMPL: a skinned multi-person linear model[J]. ACM Transactions on Graphics, 2015, 34(6): 1-16.
[92] YU H, XU Y, ZHANG J, et al. Ap-10k: a benchmark for animal pose estimation in the wild[J]. arXiv:2108.12617, 2021.
[93] LI S, LI J, TANG H, et al. ATRW: a benchmark for Amur tiger re-identification in the wild[J]. arXiv:1906.05586, 2019.
[94] CAO J, TANG H, FANG H S, et al. Cross-domain adaptation for animal pose estimation[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 9498-9507.
[95] KHOSLA A, JAYADEVAPRAKASH N, YAO B, et al. Novel dataset for fine-grained image categorization: Stanford dogs[C]//Proceedings of the 2011 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop on Fine-Grained Visual Categorization. Piscataway: IEEE, 2011: 1646-1653.
[96] BIGGS B, BOYNE O, CHARLES J, et al. Who left the dogs out? 3D animal reconstruction with expectation maximization in the loop[C]//Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 195-211.
[97] NG X L, ONG K E, ZHENG Q, et al. Animal kingdom: a large and diverse dataset for animal behavior understanding[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 19023-19034.
[98] DEL PERO L, RICCO S, SUKTHANKAR R, et al. Articulated motion discovery using pairs of trajectories[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2015: 2151-2160.
[99] DEL PERO L, RICCO S, SUKTHANKAR R, et al. Behavior discovery and alignment of articulated object classes from unstructured video[J]. International Journal of Computer Vision, 2017, 121: 303-325.
[100] MATHIS A, BIASI T, SCHNEIDER S, et al. Pretraining boosts out-of-domain robustness for pose estimation[C]//Proceedings of the 2021 IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2021: 1859-1868.
[101] SUN J J, KARIGO T, CHAKRABORTY D, et al. The multi-agent behavior dataset: mouse dyadic social interactions[J]. arXiv:2104.02710, 2021.
[102] BIGGS B, RODDICK T, FITZGIBBON A, et al. Creatures great and SMAL: recovering the shape and motion of animals from video[C]//Proceedings of the 14th Asian Conference on Computer Vision, Perth, Dec 2-6, 2018. Cham:Springer, 2019: 3-19.
[103] PEDERSEN M, HAURUM J B, BENGTSON S H, et al. 3D-ZEF: a 3D zebrafish tracking benchmark dataset[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 2426-2436.
[104] ZUFFI S, KANAZAWA A, BERGER-WOLF T, et al. Three-D Safari: learning to estimate zebra pose, shape, and texture from images “in the wild”[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 5359-5368.
[105] MU J, QIU W, HAGER G D, et al. Learning from synthetic animals[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 12386-12395.
[106] YANG Y, RAMANAN D. Articulated human detection with flexible mixtures of parts[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 35(12): 2878-2890.
[107] CHEN C H, RAMANAN D. 3D human pose estimation= 2D pose estimation+matching[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2017: 7035-7043.
[108] EVERINGHAM M, VAN GOOL L, WILLIAMS C K I, et al. The Pascal visual object classes (VOC) challenge[J]. International Journal of Computer Vision, 2010, 88: 303-338.
[109] LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]//Proceedings of the 13th European Conference on Computer Vision, Zurich, Sep 6-12, 2014. Cham: Springer, 2014: 740-755.
[110] ANDRILUKA M, PISHCHULIN L, GEHLER P, et al. 2D human pose estimation: new benchmark and state of the art analysis[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2014: 3686-3693.
[111] ANDRILUKA M, IQBAL U, INSAFUTDINOV E, et al. Posetrack: a benchmark for human pose estimation and tracking[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 5167-5176.
[112] IONESCU C, PAPAVA D, OLARU V, et al. Human3.6m: large scale datasets and predictive methods for 3D human sensing in natural environments[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 36(7): 1325-1339.
[113] MA N, ZHANG X, ZHENG H T, et al. ShuffleNET V2: practical guidelines for efficient CNN architecture design[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 116-131.
[114] YU C, XIAO B, GAO C, et al. Lite-HRNet: a lightweight high-resolution network[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 10440-10450.
[115] WANG Y, LI M, CAI H, et al. Lite pose: efficient architecture design for 2D human pose estimation[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 13126-13136.
[116] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 2818-2826.