Auroral Substorm Event Recognition Method Combining Eye Movement Infor-mation and Sequence Fingerprint

doi:10.3778/j.issn.1673-9418.2105044

Abstract

Abstract: The occurrence process, effects, and models of the auroral substorm phenomenon has been one of the most important frontier topics in solar-terrestrial physics in past 30 years. The ultraviolet imager (UVI) carried by the polar satellite collects a very large number of ultraviolet auroral images every year. Automatically and accurately identifying substorm events from massive UVI images is an urgent problem in this field. At present, there are some studies on the detection and recognition of auroral substorm events. These methods effectively use the physical characteristics of substorm events, but none of them use expert visual cognition information as a priori for substorm events recognition task. Therefore, in this paper, eye movement information is used as a visual and intuitive repre-sentation of expert knowledge, and combined with the physical characteristics of the auroral substorm sequences, an auroral substorm event recognition method combining eye movement information and sequence fingerprint is proposed. Firstly, this paper uses the eye tracker to obtain the eye movement information of space physics experts on the auroral substorm sequence. Secondly, substorm events are marked according to their physical characteristics to obtain the sequence fingerprint of each auroral substorm event. Finally, the recognition of substorm events is identified based on expert eye movement information and sequence fingerprint identification strategy. At the same time, experiments show the effectiveness of the proposed method.

Key words: auroral substorms, visual cognitive, eye movement information, sequence fingerprint, sequence recognition

摘要： 极光亚暴的发生过程、效应和模型的研究一直是近30年来日地物理学中最受重视的核心前沿课题之一。Polar卫星携带的紫外成像仪（UVI）每年采集到的紫外极光图像数量非常庞大，自动且精确地从海量的紫外极光图像中识别亚暴事件是该领域亟需解决的问题。目前已经有针对极光亚暴事件检测、识别等方面的研究，这些方法均有效利用了亚暴事件的物理特性，但都没有利用专家视觉认知信息作为先验。因此，以眼动信息作为专家知识的一种视觉直观表示，结合亚暴序列本身的物理特性，提出了结合眼动信息和序列指纹的亚暴事件识别方法。首先，利用眼动仪获取空间物理专家对极光亚暴序列认知的眼动信息；其次，根据亚暴事件的物理特征对其进行标记得到每个极光亚暴事件的序列指纹；最后，利用专家的眼动信息与序列指纹识别策略实现对亚暴事件的自动识别。同时，实验也表明了所提出的方法的有效性。

关键词: 极光亚暴, 视觉认知, 眼动信息, 序列指纹, 事件识别

HAN Yiyuan, HAN Bing, GAO Xinbo. Auroral Substorm Event Recognition Method Combining Eye Movement Infor-mation and Sequence Fingerprint[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(1): 187-197.

韩怡园, 韩冰, 高新波. 结合眼动信息和序列指纹的亚暴事件识别方法[J]. 计算机科学与探索, 2023, 17(1): 187-197.

References

[1] AKASOFU S I. The development of the auroral substorm[J]. Planetary and Space Science, 1964, 12(4): 273-282.
[2] LI L Y, CAO J B, ZHOU G C, et al. Statistical roles of storms and substorms in changing the entire outer zone relativistic electron population[J]. Journal of Geophysical Research: Space Physics, 2009, 114: A12214.
[3] FREY H U, MENDE S B, VO H B, et al. Conjugate obser-vation of optical aurora with polar satellite and ground-based cameras[J]. Advances in Space Research, 1999, 23(10): 1647-1652.
[4] BRITTNACHER M, SPANN J, PARKS G, et al. Auroral observations by the polar ultraviolet imager (UVI)[J]. Ad-vances in Space Research, 1997, 20(4/5): 1037-1042.
[5] MENDE S B, HEETDERKS H, FREY H U, et al. Far ultra-violet imaging from the IMAGE spacecraft. 1. System design[J]. Space Science Reviews, 2000, 91(1): 243-270.
[6] LIOU K, MENG C I, LUI T Y, et al. On relative timing in substorm onset signatures[J]. Journal of Geophysical Research: Space Physics, 1999, 104(A10): 22807-22817.
[7] FREY H U, MENDE S B. Substorm onsets as observed by IMAGE-FUV[C]//Proceedings of the 8th International Con-ference on Substorms, Calgary, Mar 27-31, 2006: 71-75.
[8] LIOU K. Polar ultraviolet imager observation of auroral brea-kup[J]. Journal of Geophysical Research: Space Physics, 2010, 115(A12): A12219.
[9] VENNERSTR?M S, FRIIS-CHRISTENSEN E, TROSHICHEV O A, et al. Comparison between the polar cap index, PC, and the auroral electrojet indices AE, AL, and AU[J]. Journal of Geophysical Research: Space Physics, 1991, 96(A1): 101-113.
[10] WEYGAND J M, MCPHERRON R L, KAURIISTIE K. et al. Relation of auroral substorm onset to local AL index and dispersion less particle injections[J]. Journal of Atmosph-eric and Solar-Terrestrial Physics, 2008, 70(18): 2336-2345.
[11] 杨秋菊, 梁继民, 刘俊明, 等. 一种基于紫外极光图像的亚暴膨胀期起始时刻的自动检测方法[J]. 地球物理学报, 2013, 56(5): 1435-1447.
YANG Q Q, LIANG J M, LIU J M, et al. A method for automatic identification of substorm expansion phase onset from UVI images[J]. Chinese Journal of Geophysics, 2013, 56(5): 1435-1447.
[12] YANG X, GAO X B, TAO D, et al. Shape-constrained sparse and low-rank decomposition for auroral substorm detection[J]. IEEE Transactions on Neural Networks and Learning Systems, 2015, 27(1): 32-46.
[13] 王向军, 蔡方方, 刘峰, 等. 非接触动态实时视线跟踪技术[J]. 计算机科学与探索, 2015, 9(3): 266-278.
WANG X J, CAI F F, LIU F, et al. Non-contact dynamic real-time eye tracking technology[J]. Journal of Frontiers of Computer Science and Technology, 2015, 9(3): 266-278.
[14] PASHIN A B, BAUMJOHANN W, RASPOPOV O M, et al. Pi2 magnetic pulsations, auroral break-ups, and the subs-torm current wedge: a case study[J]. Journal of Geophysics, 1982, 51(1): 223-233.
[15] LESTER M, HUGHES W J, SINGER H J. Longitudinal structure in Pi2 pulsations and the substorm current wedge[J]. Journal of Geophysical Research: Space Physics, 1984, 89(A7): 5489-5494.
[16] NEWELL P T, GJERLOEY J W. Evaluation of SuperMAG auroral electrojet indices as indicators of substorms and auroral power[J]. Journal of Geophysical Research: Space Physics, 2011, 116(A12): A12211.
[17] ?STGAARDS N, VONDRAK R R, GJERLOEV J W, et al. A relation between the energy deposition by electron precipi-tation and geomagnetic indices during substorms[J]. Journal of Geophysical Research: Space Physics, 2002, 107(A9): 161-167.
[18] SUTCLIFFE P R. Substorm onset identification using neural networks and Pi2 pulsations[J]. Annales Geophysicae, 1997, 15(10): 1257-1264.
[19] 连慧芳. 紫外极光图像边界和强度建模及亚暴预测研究[D]. 西安: 西安电子科技大学, 2019.
LIAN H F. Study on modeling of the auroral oval boundary and intensity and prediction of auroral substrom[D]. Xi’an: Xidian University, 2019.
[20] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Image-Net classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems 25, Lake Tahoe, Dec 3-6, 2012: 1097-1105.
[21] SIMONYAN K, ZISSERMAN A. Very deep convolutional net-works for large-scale image recognition[J]. arXiv:1409.1556, 2014.
[22] SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 1-9.
[23] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Con-ference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[24] ZAGORUYKO S, KOMODAKIS N. Wide residual networks[J]. arXiv:1605.07146, 2016.
[25] HUANG G, LIU Z, VANDER M L, et al. Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition,Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 2261-2269.
[26] XIE S N, GIRSHICK R, DOLLáR P, et al. Aggregated resi-dual transformations for deep neural networks[C]//Procee-dings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washing-ton: IEEE Computer Society, 2017: 5987-5995.
[27] GAO S H, CHEN M M, ZHAO K, et al. Res2Net: a new multi-scale backbone architecture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(2): 652-662.
[28] ZHANG H, WU C R, ZHANG Z Y, et al. ResNeSt: split-attention networks[J]. arXiv:2004.08955, 2020.
[29] LANDOLA F N, HAN S, MOSKEWICZ W M, et al. Squeeze-Net: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size[J]. arXiv:1602.07360, 2016.
[30] CHOLLET F. Xception: deep learning with depthwise se-parable convolutions[C]//Proceedings of the 2017 IEEE Con-ference on Computer Vision and Pattern Recognition, Ho-nolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 1800-1807.
[31] HOWARD A G, ZHU M, CHEN B, et al. MobileNets: effi-cient convolutional neural networks for mobile vision app-lications[J]. arXiv:1704.04861, 2017.
[32] ZHANG X Y, ZHOU X Y, LIN M X, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 19-21, 2018. Washington: IEEE Computer Society, 2018: 6848-6856.
[33] TAN M X, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[C]//Proceedings of the 36th International Conference on Machine Learning, Long Beach, Jun 9-15, 2019: 6105-6114.
[34] HAN B, HAN Y Y, GAO X B, et al. Boundary constraint factor embedded localizing active contour model for medi-cal image segmentation[J]. Journal of Ambient Intelligence and Humanized Computing, 2019, 10(10): 3853-3862.
[35] HAWKINS D M. The problem of overfitting[J]. Journal of Chemical Information and Computer Sciences, 2004, 44(1): 1-12.