计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (9): 2050-2060.DOI: 10.3778/j.issn.1673-9418.2103079
李珍琦1,2, 王晶1,2,+(), 贾子钰1,2, 林友芳1,2
收稿日期:
2021-03-23
修回日期:
2021-06-03
出版日期:
2022-09-01
发布日期:
2021-06-23
通讯作者:
+ E-mail: wj@bjtu.edu.cn作者简介:
李珍琦(1996—),女,河北唐山人,硕士研究生,主要研究方向为深度学习、时间序列分析与挖掘、脑机接口等。基金资助:
LI Zhenqi1,2, WANG Jing1,2,+(), JIA Ziyu1,2, LIN Youfang1,2
Received:
2021-03-23
Revised:
2021-06-03
Online:
2022-09-01
Published:
2021-06-23
About author:
LI Zhenqi, born in 1996, M.S. candidate. Her research interests include deep learning, time series analysis and mining, brain-computer interface, etc.Supported by:
摘要:
运动想象(MI)作为脑机接口(BCI)的重要应用,是运动康复训练的重要支撑。由于脑电的电极分布并非天然的欧式空间,对运动想象进行准确分类具有很大的挑战。而且现有方法仅仅考虑了脑电信号(EEG)中某一维度或者某两维度的信息,无法全面捕获脑电信号在时、频、空三个维度存在的内在特征。同时,脑电信号各维度上的动态关联强度影响了分类的鲁棒性。针对上述问题,提出了一种新颖的融合注意力的多维特征图卷积网络(AMFGCN)。首先,根据电极节点分布的非欧空间特性设计出图结构,充分表示电极间的空间相关性。其次,提出时-空、频-空的双分支框架,同时表示脑电信号在时域、频域和空间域上的信息。最后,通过融合注意力机制、图卷积和时间/频谱卷积从图表示中学习脑电信号的空间表示、时间依赖性和频率依赖性,并自适应捕获各维度上的动态关联强度。在四个公开脑机接口数据集上进行了实验,结果表明AMFGCN模型提高了分类性能,优于其他现有的运动想象分类方法。
中图分类号:
李珍琦, 王晶, 贾子钰, 林友芳. 融合注意力的多维特征图卷积运动想象分类[J]. 计算机科学与探索, 2022, 16(9): 2050-2060.
LI Zhenqi, WANG Jing, JIA Ziyu, LIN Youfang. Attention-Based Multi-dimensional Feature Graph Convolutional Network for Motor Imagery Classification[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2050-2060.
Layer | KernelSize/Stride | Kernel | Activation |
---|---|---|---|
时/频卷积 | (1,5)/(1,1) | 64 | ReLU |
表1 时间/频谱卷积结构
Table 1 Convolution structure of temporal/spectral
Layer | KernelSize/Stride | Kernel | Activation |
---|---|---|---|
时/频卷积 | (1,5)/(1,1) | 64 | ReLU |
Layer | KernelSize/Stride | Kernel | Activation |
---|---|---|---|
全局空间聚合 | | 64 | ReLU |
全局时/频聚合 | | 64 | ReLU |
表2 全局特征聚合的结构
Table 2 Structure of global feature aggregation
Layer | KernelSize/Stride | Kernel | Activation |
---|---|---|---|
全局空间聚合 | | 64 | ReLU |
全局时/频聚合 | | 64 | ReLU |
名称 | 版本说明 |
---|---|
操作系统 | Ubuntu 16.04.2 LTS |
内存 | 128 GB |
CPU型号 | Intel® Xeon® CPU E5-2683 v3 @2.00 GHz |
GPU型号 | Tesla K80 |
CUDA版本 | 10.1 |
cuDNN版本 | 7.5.1 |
Python版本 | 3.6.8 |
TensorFlow版本 | 1.13.1 |
Keras版本 | 2.1.6 |
表3 实验环境
Table 3 Experimental environment
名称 | 版本说明 |
---|---|
操作系统 | Ubuntu 16.04.2 LTS |
内存 | 128 GB |
CPU型号 | Intel® Xeon® CPU E5-2683 v3 @2.00 GHz |
GPU型号 | Tesla K80 |
CUDA版本 | 10.1 |
cuDNN版本 | 7.5.1 |
Python版本 | 3.6.8 |
TensorFlow版本 | 1.13.1 |
Keras版本 | 2.1.6 |
Subject | Metric | FBCSP | Shallow ConvNet | EEGNet | Multi-branch-3D | MSFBCNN | CNN-LSTM | AMFGCN |
---|---|---|---|---|---|---|---|---|
A01 | Acc | 0.734 4 | 0.833 3 | 0.833 3 | 0.812 2 | 0.816 0 | 0.874 1 | 0.889 8 |
Kappa | 0.645 7 | 0.777 4 | 0.777 3 | 0.749 6 | 0.754 1 | 0.831 9 | 0.852 9 | |
A02 | Acc | 0.560 8 | 0.677 5 | 0.638 0 | 0.679 5 | 0.641 5 | 0.773 9 | 0.823 3 |
Kappa | 0.414 1 | 0.569 2 | 0.516 8 | 0.572 7 | 0.522 0 | 0.697 7 | 0.764 4 | |
A03 | Acc | 0.804 2 | 0.874 1 | 0.887 6 | 0.840 9 | 0.869 8 | 0.907 3 | 0.909 3 |
Kappa | 0.738 8 | 0.832 0 | 0.849 8 | 0.787 8 | 0.826 0 | 0.843 0 | 0.879 0 | |
A04 | Acc | 0.576 8 | 0.740 0 | 0.624 1 | 0.656 9 | 0.681 4 | 0.827 7 | 0.835 9 |
Kappa | 0.435 9 | 0.653 3 | 0.499 2 | 0.542 5 | 0.574 2 | 0.770 0 | 0.781 0 | |
A05 | Acc | 0.573 8 | 0.634 1 | 0.587 2 | 0.705 8 | 0.712 7 | 0.728 9 | 0.733 5 |
Kappa | 0.431 3 | 0.511 9 | 0.449 7 | 0.607 8 | 0.615 8 | 0.630 1 | 0.644 5 | |
A06 | Acc | 0.494 8 | 0.768 7 | 0.585 1 | 0.675 7 | 0.633 7 | 0.825 1 | 0.835 5 |
Kappa | 0.326 3 | 0.691 7 | 0.446 2 | 0.567 6 | 0.511 1 | 0.766 5 | 0.780 5 | |
A07 | Acc | 0.812 5 | 0.864 6 | 0.848 1 | 0.858 7 | 0.905 4 | 0.895 8 | 0.917 1 |
Kappa | 0.749 7 | 0.819 3 | 0.797 2 | 0.811 6 | 0.873 6 | 0.794 0 | 0.889 4 | |
A08 | Acc | 0.735 2 | 0.859 8 | 0.821 2 | 0.849 4 | 0.778 7 | 0.851 7 | 0.878 0 |
Kappa | 0.646 3 | 0.812 8 | 0.760 8 | 0.799 1 | 0.704 7 | 0.768 4 | 0.837 3 | |
A09 | Acc | 0.663 6 | 0.823 8 | 0.783 0 | 0.781 0 | 0.703 1 | 0.889 1 | 0.894 5 |
Kappa | 0.551 4 | 0.764 7 | 0.710 6 | 0.708 0 | 0.603 5 | 0.817 6 | 0.859 3 | |
Mean | Acc | 0.661 8 | 0.786 2 | 0.734 2 | 0.762 2 | 0.749 1 | 0.841 5 | 0.857 4 |
Kappa | 0.548 8 | 0.714 7 | 0.645 3 | 0.683 0 | 0.665 0 | 0.768 8 | 0.809 8 |
表4 不同方法在数据集BCICIV-2a中的分类性能比较
Table 4 Performance comparison of different methods on BCICIV-2a dataset
Subject | Metric | FBCSP | Shallow ConvNet | EEGNet | Multi-branch-3D | MSFBCNN | CNN-LSTM | AMFGCN |
---|---|---|---|---|---|---|---|---|
A01 | Acc | 0.734 4 | 0.833 3 | 0.833 3 | 0.812 2 | 0.816 0 | 0.874 1 | 0.889 8 |
Kappa | 0.645 7 | 0.777 4 | 0.777 3 | 0.749 6 | 0.754 1 | 0.831 9 | 0.852 9 | |
A02 | Acc | 0.560 8 | 0.677 5 | 0.638 0 | 0.679 5 | 0.641 5 | 0.773 9 | 0.823 3 |
Kappa | 0.414 1 | 0.569 2 | 0.516 8 | 0.572 7 | 0.522 0 | 0.697 7 | 0.764 4 | |
A03 | Acc | 0.804 2 | 0.874 1 | 0.887 6 | 0.840 9 | 0.869 8 | 0.907 3 | 0.909 3 |
Kappa | 0.738 8 | 0.832 0 | 0.849 8 | 0.787 8 | 0.826 0 | 0.843 0 | 0.879 0 | |
A04 | Acc | 0.576 8 | 0.740 0 | 0.624 1 | 0.656 9 | 0.681 4 | 0.827 7 | 0.835 9 |
Kappa | 0.435 9 | 0.653 3 | 0.499 2 | 0.542 5 | 0.574 2 | 0.770 0 | 0.781 0 | |
A05 | Acc | 0.573 8 | 0.634 1 | 0.587 2 | 0.705 8 | 0.712 7 | 0.728 9 | 0.733 5 |
Kappa | 0.431 3 | 0.511 9 | 0.449 7 | 0.607 8 | 0.615 8 | 0.630 1 | 0.644 5 | |
A06 | Acc | 0.494 8 | 0.768 7 | 0.585 1 | 0.675 7 | 0.633 7 | 0.825 1 | 0.835 5 |
Kappa | 0.326 3 | 0.691 7 | 0.446 2 | 0.567 6 | 0.511 1 | 0.766 5 | 0.780 5 | |
A07 | Acc | 0.812 5 | 0.864 6 | 0.848 1 | 0.858 7 | 0.905 4 | 0.895 8 | 0.917 1 |
Kappa | 0.749 7 | 0.819 3 | 0.797 2 | 0.811 6 | 0.873 6 | 0.794 0 | 0.889 4 | |
A08 | Acc | 0.735 2 | 0.859 8 | 0.821 2 | 0.849 4 | 0.778 7 | 0.851 7 | 0.878 0 |
Kappa | 0.646 3 | 0.812 8 | 0.760 8 | 0.799 1 | 0.704 7 | 0.768 4 | 0.837 3 | |
A09 | Acc | 0.663 6 | 0.823 8 | 0.783 0 | 0.781 0 | 0.703 1 | 0.889 1 | 0.894 5 |
Kappa | 0.551 4 | 0.764 7 | 0.710 6 | 0.708 0 | 0.603 5 | 0.817 6 | 0.859 3 | |
Mean | Acc | 0.661 8 | 0.786 2 | 0.734 2 | 0.762 2 | 0.749 1 | 0.841 5 | 0.857 4 |
Kappa | 0.548 8 | 0.714 7 | 0.645 3 | 0.683 0 | 0.665 0 | 0.768 8 | 0.809 8 |
Subject | Metric | FBCSP | Shallow ConvNet | EEGNet | Multi-branch-3D | MSFBCNN | CNN-LSTM | AMFGCN |
---|---|---|---|---|---|---|---|---|
K3b | Acc | 0.913 9 | 0.967 4 | 0.962 5 | 0.948 1 | 0.963 9 | 0.911 8 | 0.984 0 |
Kappa | 0.884 8 | 0.956 4 | 0.949 9 | 0.930 9 | 0.951 7 | 0.882 1 | 0.978 7 | |
K6b | Acc | 0.679 2 | 0.769 8 | 0.705 2 | 0.752 8 | 0.785 4 | 0.883 3 | 0.922 9 |
Kappa | 0.572 1 | 0.691 1 | 0.607 3 | 0.670 4 | 0.713 5 | 0.840 6 | 0.896 7 | |
L1b | Acc | 0.803 1 | 0.829 2 | 0.802 1 | 0.807 8 | 0.822 9 | 0.916 7 | 0.957 3 |
Kappa | 0.736 9 | 0.771 0 | 0.735 4 | 0.743 7 | 0.763 1 | 0.886 8 | 0.943 0 | |
Mean | Acc | 0.798 7 | 0.855 4 | 0.823 3 | 0.836 3 | 0.857 4 | 0.903 9 | 0.954 7 |
Kappa | 0.731 3 | 0.806 2 | 0.764 2 | 0.781 7 | 0.809 4 | 0.869 9 | 0.939 4 |
表5 不同方法在数据集BCICIII-3a中的分类性能比较
Table 5 Performance comparison of different methods on BCICIII-3a dataset
Subject | Metric | FBCSP | Shallow ConvNet | EEGNet | Multi-branch-3D | MSFBCNN | CNN-LSTM | AMFGCN |
---|---|---|---|---|---|---|---|---|
K3b | Acc | 0.913 9 | 0.967 4 | 0.962 5 | 0.948 1 | 0.963 9 | 0.911 8 | 0.984 0 |
Kappa | 0.884 8 | 0.956 4 | 0.949 9 | 0.930 9 | 0.951 7 | 0.882 1 | 0.978 7 | |
K6b | Acc | 0.679 2 | 0.769 8 | 0.705 2 | 0.752 8 | 0.785 4 | 0.883 3 | 0.922 9 |
Kappa | 0.572 1 | 0.691 1 | 0.607 3 | 0.670 4 | 0.713 5 | 0.840 6 | 0.896 7 | |
L1b | Acc | 0.803 1 | 0.829 2 | 0.802 1 | 0.807 8 | 0.822 9 | 0.916 7 | 0.957 3 |
Kappa | 0.736 9 | 0.771 0 | 0.735 4 | 0.743 7 | 0.763 1 | 0.886 8 | 0.943 0 | |
Mean | Acc | 0.798 7 | 0.855 4 | 0.823 3 | 0.836 3 | 0.857 4 | 0.903 9 | 0.954 7 |
Kappa | 0.731 3 | 0.806 2 | 0.764 2 | 0.781 7 | 0.809 4 | 0.869 9 | 0.939 4 |
Subject | Metric | FBCSP | Shallow ConvNet | EEGNet | Multi-branch-3D | MSFBCNN | CNN-LSTM | AMFGCN |
---|---|---|---|---|---|---|---|---|
1 | Acc | 0.476 5 | 0.847 6 | 0.874 0 | 0.867 2 | 0.899 9 | 0.707 3 | 0.911 4 |
Kappa | 0.372 0 | 0.816 9 | 0.848 6 | 0.840 6 | 0.879 8 | 0.648 7 | 0.893 5 | |
2 | Acc | 0.286 3 | 0.675 2 | 0.672 2 | 0.733 7 | 0.783 7 | 0.697 0 | 0.831 3 |
Kappa | 0.143 8 | 0.610 0 | 0.606 5 | 0.680 4 | 0.736 3 | 0.596 4 | 0.797 4 | |
3 | Acc | 0.368 2 | 0.793 3 | 0.823 6 | 0.798 8 | 0.820 9 | 0.689 8 | 0.835 3 |
Kappa | 0.241 7 | 0.751 6 | 0.788 1 | 0.758 5 | 0.780 7 | 0.607 4 | 0.802 3 | |
4 | Acc | 0.359 9 | 0.748 3 | 0.769 4 | 0.769 9 | 0.814 9 | 0.771 6 | 0.827 1 |
Kappa | 0.231 5 | 0.697 8 | 0.723 1 | 0.723 7 | 0.777 7 | 0.705 3 | 0.792 4 | |
5 | Acc | 0.299 8 | 0.703 7 | 0.703 2 | 0.741 2 | 0.790 4 | 0.758 6 | 0.796 1 |
Kappa | 0.159 0 | 0.644 4 | 0.643 7 | 0.689 3 | 0.748 2 | 0.710 3 | 0.755 2 | |
6 | Acc | 0.383 0 | 0.845 8 | 0.893 3 | 0.854 9 | 0.891 4 | 0.878 7 | 0.878 1 |
Kappa | 0.259 5 | 0.814 9 | 0.871 9 | 0.825 8 | 0.869 5 | 0.834 1 | 0.853 6 | |
7 | Acc | 0.221 0 | 0.384 4 | 0.434 6 | 0.399 9 | 0.468 1 | 0.508 0 | 0.515 9 |
Kappa | 0.064 6 | 0.261 0 | 0.321 3 | 0.279 7 | 0.361 4 | 0.412 1 | 0.419 0 | |
8 | Acc | 0.213 5 | 0.391 6 | 0.442 5 | 0.388 6 | 0.454 1 | 0.491 8 | 0.493 7 |
Kappa | 0.056 9 | 0.269 5 | 0.331 5 | 0.266 0 | 0.340 9 | 0.390 2 | 0.392 0 | |
9 | Acc | 0.727 2 | 0.989 4 | 0.988 4 | 0.988 4 | 0.988 4 | 0.692 0 | 0.990 0 |
Kappa | 0.671 7 | 0.987 3 | 0.986 0 | 0.986 0 | 0.986 0 | 0.583 7 | 0.987 9 | |
10 | Acc | 0.418 0 | 0.796 8 | 0.810 3 | 0.817 4 | 0.853 4 | 0.586 2 | 0.853 7 |
Kappa | 0.301 2 | 0.755 9 | 0.772 0 | 0.780 8 | 0.824 0 | 0.482 2 | 0.824 3 | |
11 | Acc | 0.596 1 | 0.955 9 | 0.953 5 | 0.928 3 | 0.948 5 | 0.729 4 | 0.956 1 |
Kappa | 0.515 0 | 0.947 0 | 0.944 1 | 0.913 9 | 0.938 2 | 0.635 1 | 0.947 3 | |
12 | Acc | 0.444 3 | 0.829 6 | 0.849 3 | 0.859 0 | 0.894 5 | 0.852 1 | 0.904 8 |
Kappa | 0.332 9 | 0.795 4 | 0.819 0 | 0.830 7 | 0.873 3 | 0.782 1 | 0.885 6 | |
Mean | Acc | 0.399 5 | 0.746 8 | 0.767 9 | 0.762 3 | 0.800 7 | 0.696 9 | 0.816 1 |
Kappa | 0.279 2 | 0.696 0 | 0.721 3 | 0.714 6 | 0.759 7 | 0.615 6 | 0.779 2 |
表6 不同方法在数据集HaLT中的分类性能比较
Table 6 Performance comparison of different methods on HaLT dataset
Subject | Metric | FBCSP | Shallow ConvNet | EEGNet | Multi-branch-3D | MSFBCNN | CNN-LSTM | AMFGCN |
---|---|---|---|---|---|---|---|---|
1 | Acc | 0.476 5 | 0.847 6 | 0.874 0 | 0.867 2 | 0.899 9 | 0.707 3 | 0.911 4 |
Kappa | 0.372 0 | 0.816 9 | 0.848 6 | 0.840 6 | 0.879 8 | 0.648 7 | 0.893 5 | |
2 | Acc | 0.286 3 | 0.675 2 | 0.672 2 | 0.733 7 | 0.783 7 | 0.697 0 | 0.831 3 |
Kappa | 0.143 8 | 0.610 0 | 0.606 5 | 0.680 4 | 0.736 3 | 0.596 4 | 0.797 4 | |
3 | Acc | 0.368 2 | 0.793 3 | 0.823 6 | 0.798 8 | 0.820 9 | 0.689 8 | 0.835 3 |
Kappa | 0.241 7 | 0.751 6 | 0.788 1 | 0.758 5 | 0.780 7 | 0.607 4 | 0.802 3 | |
4 | Acc | 0.359 9 | 0.748 3 | 0.769 4 | 0.769 9 | 0.814 9 | 0.771 6 | 0.827 1 |
Kappa | 0.231 5 | 0.697 8 | 0.723 1 | 0.723 7 | 0.777 7 | 0.705 3 | 0.792 4 | |
5 | Acc | 0.299 8 | 0.703 7 | 0.703 2 | 0.741 2 | 0.790 4 | 0.758 6 | 0.796 1 |
Kappa | 0.159 0 | 0.644 4 | 0.643 7 | 0.689 3 | 0.748 2 | 0.710 3 | 0.755 2 | |
6 | Acc | 0.383 0 | 0.845 8 | 0.893 3 | 0.854 9 | 0.891 4 | 0.878 7 | 0.878 1 |
Kappa | 0.259 5 | 0.814 9 | 0.871 9 | 0.825 8 | 0.869 5 | 0.834 1 | 0.853 6 | |
7 | Acc | 0.221 0 | 0.384 4 | 0.434 6 | 0.399 9 | 0.468 1 | 0.508 0 | 0.515 9 |
Kappa | 0.064 6 | 0.261 0 | 0.321 3 | 0.279 7 | 0.361 4 | 0.412 1 | 0.419 0 | |
8 | Acc | 0.213 5 | 0.391 6 | 0.442 5 | 0.388 6 | 0.454 1 | 0.491 8 | 0.493 7 |
Kappa | 0.056 9 | 0.269 5 | 0.331 5 | 0.266 0 | 0.340 9 | 0.390 2 | 0.392 0 | |
9 | Acc | 0.727 2 | 0.989 4 | 0.988 4 | 0.988 4 | 0.988 4 | 0.692 0 | 0.990 0 |
Kappa | 0.671 7 | 0.987 3 | 0.986 0 | 0.986 0 | 0.986 0 | 0.583 7 | 0.987 9 | |
10 | Acc | 0.418 0 | 0.796 8 | 0.810 3 | 0.817 4 | 0.853 4 | 0.586 2 | 0.853 7 |
Kappa | 0.301 2 | 0.755 9 | 0.772 0 | 0.780 8 | 0.824 0 | 0.482 2 | 0.824 3 | |
11 | Acc | 0.596 1 | 0.955 9 | 0.953 5 | 0.928 3 | 0.948 5 | 0.729 4 | 0.956 1 |
Kappa | 0.515 0 | 0.947 0 | 0.944 1 | 0.913 9 | 0.938 2 | 0.635 1 | 0.947 3 | |
12 | Acc | 0.444 3 | 0.829 6 | 0.849 3 | 0.859 0 | 0.894 5 | 0.852 1 | 0.904 8 |
Kappa | 0.332 9 | 0.795 4 | 0.819 0 | 0.830 7 | 0.873 3 | 0.782 1 | 0.885 6 | |
Mean | Acc | 0.399 5 | 0.746 8 | 0.767 9 | 0.762 3 | 0.800 7 | 0.696 9 | 0.816 1 |
Kappa | 0.279 2 | 0.696 0 | 0.721 3 | 0.714 6 | 0.759 7 | 0.615 6 | 0.779 2 |
Subject | Metric | FBCSP | Shallow ConvNet | EEGNet | Multi-branch-3D | MSFBCNN | CNN-LSTM | AMFGCN |
---|---|---|---|---|---|---|---|---|
S1 | Acc | 0.795 3 | 0.818 5 | 0.539 6 | 0.506 9 | 0.790 4 | 0.819 3 | 0.820 0 |
Kappa | 0.692 7 | 0.727 7 | 0.249 4 | 0.200 7 | 0.685 1 | 0.729 5 | 0.729 7 | |
S2 | Acc | 0.438 4 | 0.393 0 | 0.537 2 | 0.520 4 | 0.412 1 | 0.580 7 | 0.593 2 |
Kappa | 0.158 0 | 0.092 1 | 0.247 2 | 0.221 0 | 0.118 7 | 0.373 0 | 0.389 9 | |
S3 | Acc | 0.861 7 | 0.802 6 | 0.761 0 | 0.777 1 | 0.909 9 | 0.886 8 | 0.920 8 |
Kappa | 0.792 6 | 0.704 2 | 0.581 7 | 0.605 9 | 0.864 9 | 0.829 8 | 0.880 9 | |
S4 | Acc | 0.689 5 | 0.723 2 | 0.541 4 | 0.537 5 | 0.553 4 | 0.814 3 | 0.808 1 |
Kappa | 0.534 0 | 0.584 9 | 0.252 7 | 0.246 6 | 0.336 1 | 0.721 2 | 0.711 9 | |
S5 | Acc | 0.360 2 | 0.526 3 | 0.656 2 | 0.613 6 | 0.687 2 | 0.708 7 | 0.734 5 |
Kappa | 0.038 9 | 0.288 9 | 0.424 7 | 0.360 4 | 0.530 8 | 0.565 5 | 0.602 0 | |
S6 | Acc | 0.533 9 | 0.469 6 | 0.565 6 | 0.589 0 | 0.705 3 | 0.680 1 | 0.708 3 |
Kappa | 0.300 9 | 0.204 4 | 0.290 5 | 0.322 7 | 0.557 6 | 0.522 5 | 0.562 7 | |
S7 | Acc | 0.535 9 | 0.435 4 | 0.509 8 | 0.562 9 | 0.448 3 | 0.610 3 | 0.616 2 |
Kappa | 0.302 7 | 0.156 1 | 0.210 6 | 0.283 8 | 0.173 3 | 0.429 6 | 0.429 6 | |
S8 | Acc | 0.401 8 | 0.515 0 | 0.505 8 | 0.499 7 | 0.540 5 | 0.546 9 | 0.564 5 |
Kappa | 0.101 4 | 0.271 3 | 0.199 5 | 0.189 6 | 0.310 6 | 0.320 8 | 0.347 2 | |
S9 | Acc | 0.466 2 | 0.626 8 | 0.516 9 | 0.552 7 | 0.760 2 | 0.743 1 | 0.776 4 |
Kappa | 0.195 6 | 0.438 5 | 0.213 6 | 0.269 6 | 0.639 7 | 0.617 6 | 0.664 6 | |
S10 | Acc | 0.348 1 | 0.326 4 | 0.444 0 | 0.449 5 | 0.416 3 | 0.438 0 | 0.451 9 |
Kappa | 0.025 1 | 0.029 0 | 0.171 1 | 0.131 0 | 0.126 9 | 0.167 7 | 0.177 9 | |
Mean | Acc | 0.543 1 | 0.563 7 | 0.557 7 | 0.560 9 | 0.622 4 | 0.682 8 | 0.699 4 |
Kappa | 0.314 2 | 0.349 7 | 0.286 1 | 0.283 1 | 0.434 4 | 0.527 7 | 0.549 2 |
表7 不同方法在数据集AHU-MIEEG中的分类性能比较
Table 7 Performance comparison of different methods on AHU-MIEEG dataset
Subject | Metric | FBCSP | Shallow ConvNet | EEGNet | Multi-branch-3D | MSFBCNN | CNN-LSTM | AMFGCN |
---|---|---|---|---|---|---|---|---|
S1 | Acc | 0.795 3 | 0.818 5 | 0.539 6 | 0.506 9 | 0.790 4 | 0.819 3 | 0.820 0 |
Kappa | 0.692 7 | 0.727 7 | 0.249 4 | 0.200 7 | 0.685 1 | 0.729 5 | 0.729 7 | |
S2 | Acc | 0.438 4 | 0.393 0 | 0.537 2 | 0.520 4 | 0.412 1 | 0.580 7 | 0.593 2 |
Kappa | 0.158 0 | 0.092 1 | 0.247 2 | 0.221 0 | 0.118 7 | 0.373 0 | 0.389 9 | |
S3 | Acc | 0.861 7 | 0.802 6 | 0.761 0 | 0.777 1 | 0.909 9 | 0.886 8 | 0.920 8 |
Kappa | 0.792 6 | 0.704 2 | 0.581 7 | 0.605 9 | 0.864 9 | 0.829 8 | 0.880 9 | |
S4 | Acc | 0.689 5 | 0.723 2 | 0.541 4 | 0.537 5 | 0.553 4 | 0.814 3 | 0.808 1 |
Kappa | 0.534 0 | 0.584 9 | 0.252 7 | 0.246 6 | 0.336 1 | 0.721 2 | 0.711 9 | |
S5 | Acc | 0.360 2 | 0.526 3 | 0.656 2 | 0.613 6 | 0.687 2 | 0.708 7 | 0.734 5 |
Kappa | 0.038 9 | 0.288 9 | 0.424 7 | 0.360 4 | 0.530 8 | 0.565 5 | 0.602 0 | |
S6 | Acc | 0.533 9 | 0.469 6 | 0.565 6 | 0.589 0 | 0.705 3 | 0.680 1 | 0.708 3 |
Kappa | 0.300 9 | 0.204 4 | 0.290 5 | 0.322 7 | 0.557 6 | 0.522 5 | 0.562 7 | |
S7 | Acc | 0.535 9 | 0.435 4 | 0.509 8 | 0.562 9 | 0.448 3 | 0.610 3 | 0.616 2 |
Kappa | 0.302 7 | 0.156 1 | 0.210 6 | 0.283 8 | 0.173 3 | 0.429 6 | 0.429 6 | |
S8 | Acc | 0.401 8 | 0.515 0 | 0.505 8 | 0.499 7 | 0.540 5 | 0.546 9 | 0.564 5 |
Kappa | 0.101 4 | 0.271 3 | 0.199 5 | 0.189 6 | 0.310 6 | 0.320 8 | 0.347 2 | |
S9 | Acc | 0.466 2 | 0.626 8 | 0.516 9 | 0.552 7 | 0.760 2 | 0.743 1 | 0.776 4 |
Kappa | 0.195 6 | 0.438 5 | 0.213 6 | 0.269 6 | 0.639 7 | 0.617 6 | 0.664 6 | |
S10 | Acc | 0.348 1 | 0.326 4 | 0.444 0 | 0.449 5 | 0.416 3 | 0.438 0 | 0.451 9 |
Kappa | 0.025 1 | 0.029 0 | 0.171 1 | 0.131 0 | 0.126 9 | 0.167 7 | 0.177 9 | |
Mean | Acc | 0.543 1 | 0.563 7 | 0.557 7 | 0.560 9 | 0.622 4 | 0.682 8 | 0.699 4 |
Kappa | 0.314 2 | 0.349 7 | 0.286 1 | 0.283 1 | 0.434 4 | 0.527 7 | 0.549 2 |
[1] |
BIRBAUMER N, GHANAYIM N, HINTERBERGER T, et al. A spelling device for the paralysed[J]. Nature, 1999, 398(6725): 297-298.
DOI URL |
[2] |
SANTHANAM G, RYU S I, YU B M, et al. A high-performance brain-computer interface[J]. Nature, 2006, 442(7099): 195-198.
DOI URL |
[3] |
PARSONS L M, PARSONS L M, FOX P T, et al. Use of implicit motor imagery for visual shape discrimination as revealed by PET[J]. Nature, 1995, 375(6526): 54-58.
DOI URL |
[4] | JIA Z Y, LIN Y F, WANG J, et al. MMCNN: a multi-branch multi-scale convolutional neural network for motor imagery classification[C]// LNCS 12459: Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, Ghent, Sep 14-18, 2020. Cham: Springer, 2021: 736-751. |
[5] |
陈萌, 李幼军, 刘岩. 脑电信号与个人情绪状态关联性分析研究[J]. 计算机科学与探索, 2017, 11(5): 794-801.
DOI |
CHEN M, LI Y J, LIU Y. Analysis of correlation between EEG signal and personal emotional state[J]. Journal of Frontiers of Computer Science and Technology, 2017, 11(5): 794-801.
DOI |
|
[6] | JIA Z, LIN Y, WANG J, et al. GraphSleepNet: adaptive spatial-temporal graph convolutional networks for sleep stage classification[C]// Proceedings of the 29th International Joint Conference on Artificial Intelligence. San Francisco: Margan Kaufmann, 2020: 1324-1330. |
[7] |
LI Y, ZHANG X R, ZHANG B, et al. A channel-projection mixed-scale convolutional neural network for motor imagery EEG decoding[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(6): 1170-1180.
DOI URL |
[8] |
PFURTSCHELLER G, NEUPER C. Motor imagery and direct brain-computer communication[J]. Proceedings of the IEEE, 2001, 89(7): 1123-1134.
DOI URL |
[9] | ANG K K, CHIN Z Y, WANG C, et al. Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b[J]. Frontiers in Neuroscience, 2012, 6: 39. |
[10] |
GANDHI T, PANIGRAHI B K, ANAND S. A comparative study of wavelet families for EEG signal classification[J]. Neurocomputing, 2011, 74(17): 3051-3057.
DOI URL |
[11] | ZABIDI A, MANSOR W, LEE Y K, et al. Short-time Fourier transform analysis of EEG signal generated during imagined writing[C]// Proceedings of the 2012 International Conference on System Engineering and Technology, Bandung, Sep 11-12, 2012. Piscataway: IEEE, 2012: 1-4. |
[12] |
SCHIRRMEISTER R T, SPRINGENBERG J T, FIEDERER L D J, et al. Deep learning with convolutional neural networks for EEG decoding and visualization[J]. Human Brain Mapping, 2017, 38(11): 5391-5420.
DOI URL |
[13] |
ZHAO X, ZHANG H, ZHU G, et al. A multi-branch 3D convolutional neural network for EEG-based motor imagery classification[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(10): 2164-2177.
DOI URL |
[14] |
WU H, LI F, LI Y, et al. A parallel multiscale filter bank convolutional neural networks for motor imagery EEG classi-fication[J]. Frontiers in Neuroscience, 2019, 13: 1275.
DOI URL |
[15] | LI X, QIAN B, WEI J, et al. Classify EEG and reveal latent graph structure with spatio-temporal graph convolutional neural network[C]// Proceedings of the 2019 IEEE International Conference on Data Mining, Beijing, Nov 8-11, 2019. Piscataway: IEEE, 2019: 389-398. |
[16] | LUN X, JIA S, HOU Y, et al. GCNs-Net: a graph convolutional neural network approach for decoding time-resolved EEG motor imagery signals[J]. arXiv:2006.08924, 2020. |
[17] |
SUN B, ZHANG H, WU Z, et al. Adaptive spatiotemporal graph convolutional networks for motor imagery classification[J]. IEEE Signal Processing Letters, 2021, 28: 219-223.
DOI URL |
[18] | GILMER J, SCHOENHOLZ S S, RILEY P F, et al. Neural message passing for quantum chemistry[J]. arXiv:1704.01212, 2017. |
[19] | DEFFERRARD M, BRESSON X, VANDERGHEYNST P. Convolutional neural networks on graphs with fast localized spectral filtering[C]// Proceedings of the Annual Conference on Neural Information Processing Systems 2016, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates, 2016: 3844-3852. |
[20] | POLAT K, GÜNEŞ S. Classification of epileptiform EEG using a hybrid system based on decision tree classifier and fast Fourier transform[J]. Applied Mathematics and Compu-tation, 2007, 187(2): 1017-1026. |
[21] | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the Annual Conference on Neural Information Processing Systems 2017, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 5998-6008. |
[22] | WANG F, ZHONG S H, PENG J F, et al. Data augmentation for EEG-based emotion recognition with deep convolutional neural networks[C]// LNCS 1070: Proceedings of the 24th International Conference on Multimedia Modeling, Bangkok, Feb 2-5, 2018. Cham: Springer, 2018: 82-93. |
[23] |
LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: a compact convolutional network for EEG-based brain-computer interfaces[J]. Journal of Neural Engineering, 2018, 15(5): 056013.
DOI URL |
[24] |
ZHANG R, ZONG Q, DOU L, et al. A novel hybrid deep learning scheme for four-class motor imagery classification[J]. Journal of Neural Engineering, 2019, 16(6): 066004.
DOI URL |
[1] | 吕晓琦, 纪科, 陈贞翔, 孙润元, 马坤, 邬俊, 李浥东. 结合注意力与循环神经网络的专家推荐算法[J]. 计算机科学与探索, 2022, 16(9): 2068-2077. |
[2] | 杨知桥, 张莹, 王新杰, 张东波, 王玉. 改进U型网络在视网膜病变检测中的应用研究[J]. 计算机科学与探索, 2022, 16(8): 1877-1884. |
[3] | 夏鸿斌, 肖奕飞, 刘渊. 融合自注意力机制的长文本生成对抗网络模型[J]. 计算机科学与探索, 2022, 16(7): 1603-1610. |
[4] | 李运寰, 闻继伟, 彭力. 高帧率的轻量级孪生网络目标跟踪[J]. 计算机科学与探索, 2022, 16(6): 1405-1416. |
[5] | 张雁操, 赵宇海, 史岚. 融合图注意力的多特征链接预测算法[J]. 计算机科学与探索, 2022, 16(5): 1096-1106. |
[6] | 赵鹏飞, 谢林柏, 彭力. 融合注意力机制的深层次小目标检测算法[J]. 计算机科学与探索, 2022, 16(4): 927-937. |
[7] | 陆仲达, 张春达, 张佳奇, 王子菲, 许军华. 双分支网络的苹果叶部病害识别[J]. 计算机科学与探索, 2022, 16(4): 917-926. |
[8] | 黄思远, 赵宇海, 梁燚铭. 融合图嵌入和注意力机制的代码搜索[J]. 计算机科学与探索, 2022, 16(4): 844-854. |
[9] | 王燕妮, 余丽仙. 注意力与多尺度有效融合的SSD目标检测算法[J]. 计算机科学与探索, 2022, 16(2): 438-447. |
[10] | 李想, 杨兴耀, 于炯, 钱育蓉, 郑捷. 基于知识图谱卷积网络的双端推荐算法[J]. 计算机科学与探索, 2022, 16(1): 176-184. |
[11] | 蒋光峰, 胡鹏程, 叶桦, 仰燕兰. 基于重构误差的同构图分类模型[J]. 计算机科学与探索, 2022, 16(1): 185-193. |
[12] | 蔡明昕, 孙晶, 王斌. 多角度语义轨迹相似度计算模型[J]. 计算机科学与探索, 2021, 15(9): 1632-1640. |
[13] | 陈俊芬, 张明, 赵佳成, 谢博鋆, 李艳. 结合降噪和自注意力的深度聚类算法[J]. 计算机科学与探索, 2021, 15(9): 1717-1727. |
[14] | 张晗, 贾甜远, 骆方, 张生, 邬霞. 面向网络文本的BERT心理特质预测研究[J]. 计算机科学与探索, 2021, 15(8): 1459-1468. |
[15] | 陈剑南, 杜军平, 薛哲, 寇菲菲. 基于多重注意力的金融事件大数据精准画像[J]. 计算机科学与探索, 2021, 15(7): 1237-1244. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||