طبقه‌بندی سیگنال‌های ترکیبی EEG-fNIRS با استفاده از ویژگی‌های عمیق کاهش‌یافته برای کاربردهای BCI BCI

نوع مقاله : مقاله پژوهشی

نویسندگان

1 استادیار، دانشگاه فرماندهی و ستاد آجا، تهران، ایران

2 دانشجوی دکتری ، دانشگاه ارومیه، ارومیه، ایران

چکیده

رابط مغز و کامپیوتر (BCI) مبتنی بر تخیل حرکتی (MI) به‌عنوان یک روش مؤثر برای برقراری ارتباط مستقیم بین مغز و دستگاه‌های الکترونیکی خارجی ارائه شده است. مسئله اصلی در سیستم­های BCI تبدیل سیگنال­های تولید شده در مغز به دستورات قابل‌اعتماد برای کنترل دستگاه­های الکترونیکی است. سیگنال الکتروانسفالوگرافی (EEG) پرکاربردترین سیگنال در پژوهش­های مرتبط با BCI است. اخیراً ترکیب با برخی سیگنال­های حیاتی دیگر نظیر طیف­سنجی نزدیک مادون‌قرمز (NIRS) برای افزایش کار آیی سیستم­های BCI مورد توجه قرار گرفته است. مهم­ترین چالش در سیستم­های BCI با چندین سیگنال حیاتی، استخراج ویژگی و ترکیب ویژگی­های سیگنال­های مختلف است. برای این منظور، در این مقاله ابتدا سیگنال­های EEG و اجزای NIRS، شامل HbO و HbR، به باندهای فرکانسی مختلف تجزیه شدند. در ادامه با استفاده از شبکه­های عصبی کانولوشنی یک­بعدی، ویژگی­های عمیق از هر زیرباند استخراج شده و با هم ادغام می­شوند. با توجه به ابعاد بالای بردار ویژگی نهایی، با استفاده از تجزیه‌وتحلیل اجزای اصلی با هسته (KPCA)، ویژگی­های غیرمفید را حذف کرده و ویژگی­های باقیمانده با استفاده از بردار پشتیبان ماشین طبقه­بندی می­شوند. نتایج نشان می­­دهند روش پیشنهادی دقت بالایی دارد و روش­های ارائه­شده اخیر را بهبود می­دهد.

کلیدواژه‌ها


Smiley face

https://creativecommons.org/licenses/by/4.0/

[1]       Khare, S. K.; Bajaj, V. “A Facile and Flexible Motor Imagery Classification Using Electroencephalogram Signals”; Comput. Meth. Prog. Bio. 2020, 197, 105722. http://doi.org/10.1016/j.cmpb.2020.105722.
[2]       Ang, K.K.; Chin, Z.Y.; Zhang, H.; Guan, C. “Filter Bank Common Spatial Pattern (FBCSP) in Brain-Computer Interface”; IEEE Int. Conf. Neu. Net. 2008, 2390-2397. http://doi.org/10.1109/IJCNN.2008.4634130.
[3]        Zhang, R.; Li, Y.; Yan, Y.; Zhang, H.; Wu, S.; Yu,T; Gu, Z. “Control of a Wheelchair in an Indoor Environment Based on a Brain–Computer Interface and Automated Navigation”; IEEE Trans. Neu. Syst. Rehabil. Eng. 2015, 24, 128-139.
http://doi.org/10.1109/TNSRE.2015.2439298.
[4]       Wang, H.; Dong, X.; Chen, Z.; Shi, B. E. “Hybrid Gaze/EEG Brain Computer Interface for Robot Arm Control on a Pick and Place Task”; Int. Conf. IEEE Eng. Med. Bio. Soc. 2015, 1476-1479. http://doi.org/10.1109/EMBC.2015.7318649.
[5]       Paszkiel, S. “Using BCI Technology for Controlling a Mobile Vehicle”; Analysis and Classification of EEG Signals for Brain–Computer Interfaces: Springer 2020,          71-77. http://doi.org/10.1007/978-3-030-13273-6_34.
[6]       
 
LaFleur, K.; Cassady, K.; Doud, A.; Shades, K.; Rogin, E.; He, B. “Quadcopter Control in Three-Dimensional Space Using a Noninvasive Motor Imagery-Based Brain–Computer Interface”; J. Neural Eng., 2013, 046003. http://doi.org/ 10.1088/1741-2560/10/4/046003.
[7]       Schirrmeister, R.T.; Springenberg, J.T.; Fiederer, L.D.J., Glasstetter, M.; Eggensperger, K.; Tangermann, M.; Hutter, F.; Burgard, W.; Ball, T. “Deep Learning with Convolutional Neural Networks for EEG Decoding and Visualization”; Hum. Brain Mapp. 2017, 38, 5391-5420. http://doi.org/ 10.48550/arXiv.1703.05051.
[8]       Sawangjai, P.; Hompoonsup, S.; Leelaarporn, P.; Kongwudhikunakorn, S.; Wilaiprasitporn, T. “Consumer Grade EEG Measuring Sensors as Research Tools: A Review”; IEEE Sensor. J. 2019, 20, 3996-4024. http://doi.org/10.1109/JSEN.2019.2962874.
[9]       Ergün, E.; Aydemir, Ö. “A Hybrid BCI Using Singular    Value Decomposition Values of the Fast Walsh Hadamard Transform Coefficients”; IEEE Trans. Cogn. Devel. Syst. 2020.  http://doi.org/10.1109/TCDS.2020.3028785.
[10]    Ghonchi, H.; Fateh, M.; Abolghasemi, V.; Ferdowsi, S.; Rezvani, M. “Deep Recurrent–Convolutional Neural Network for Classification of Simultaneous EEG–fNIRS Signals”; IET Signal Proc. 2020, 14, 142-153. http://doi.org/10.1049/iet-spr.2019.0297.
[11]    Chae, Y.; Jeong, J.; Jo, S. “Toward Brain-Actuated Humanoid Robots: Asynchronous Direct Control Using an EEG-Based BCI”; IEEE Trans. on Robot. 2012, 28,       1131-1144. http://doi.org/10.1109/TRO.2012.2201310.
[12]    Li, Y.; Li, X.; Ratcliffe, M.; Liu, L.; Qi, Y.; Liu, Q. “A Real-Time EEG-Based BCI System for Attention Recognition in Ubiquitous Environment”; Int. Workshop Ubiquitous Affective Awareness and Intelligent Interaction 2011, 33-40.
http://doi.org/10.1145/2030092.2030099
[13]    Alazrai, A.; Alwanni, H.; Daoud, M. I. “EEG-Based BCI System for Decoding Finger Movements Within the Same Hand”; Neurosci. Lett. 2019, 698, 113-120. http://doi.org/ 10.1016/j.neulet.2018.12.045.
[14]    Mondini, V.; Mangia, A. L.; Cappello, A. “EEG-Based BCI System Using Adaptive Features Extraction and Classification Procedures”; Computat. Intell. Neurosci. 2016, 4562601. http://doi.org/10.1155/2016/4562601.
[15]    Arvaneh, M.; Guan, C.; Ang, K.; Quek, C. “Optimizing the Channel Selection and Classification Accuracy in EEG-Based BCI”; IEEE Trans. Bio-Med. Eng. 2011, 58,        1865-1873. http://doi.org/10.1109/TBME.2011.2131142.
[16]    Gaur, P.; Pachori, R. B.; Wang, H.; Prasad, G. “A          Multi-Class EEG-Based BCI Classification Using Multivariate Empirical Mode Decomposition Based Filtering and Riemannian Geometry”; Expert Syst. Appl. 2018, 95, 201-211. http://doi.org/10.1016/j.eswa.2017.11.007.
[17]    Paszkiel, S. “Using Neural Networks for Classification of the Changes in the EEG Signal Based on Facial Expressions”; Analysis and Classification of EEG Signals for Brain–Computer Interfaces: Springer 2020, 41-69. http://doi.org/ 10.1016/j.bspc.2016.11.018.
[18]    Pan, J.; Li, Y.; Wang, J. “An EEG-Based Brain-Computer Interface for Emotion Recognition”; Int. Conf. Neural Networks 2016, 2063-2067. http://doi.org/10.1109/TAFFC. 2019.2901456.
[19]    Tan, C.; Sun, F.; Zhang, W. “Deep Transfer Learning for EEG-Based Brain-Computer Interface”; IEEE Int. Conf. Acoustics, Speech and Signal Processing 2018, 916-920. http://doi.org/10.48550/arXiv.1808.01752. 
[20]    Fahimi, F.; Zhang, Z.; Goh, W. B.; Lee, T. -S.; Ang, K.; Guan, C. “Inter-Subject Transfer Learning with an End-to-End Deep Convolutional Neural Network for EEG-Based BCI”; J. Neural Eng. 2019, 16, 026007. http://doi.org/ 10.1088/1741-2552/aaf3f6.
[21]    Mousavi, H.; Shahrokh-Abadi, M. H.; Tavakkoli, H. “Classification of Brain Signals in BCI System Using Wavelet Transform” J. New Appr. Basic Sci., Tech. Eng. Res. 2020, 2, 6, 20-29. (In Persian)
[22]    Borgheai, S. B.; McLinden, J.; Zisk, A. H.; Hosni, S. I.; Deligani, R.J.; Abtahi, M.; Mankodiya, K.; Shahriari, Y. “Enhancing Communication for People in Late-Stage ALS Using an fNIRS-Based BCI System”; IEEE Trans. Neural Syst. Rehabil. Eng. 2020, 28, 1198-1207. http://doi.org/ 10.1109/TNSRE.2020.2980772.
[23]    Bauernfeind, G.; Steyrl, D.; Brunner, C.; Müller-Putz, G.R. “Single Trial Classification of fNIRS-Based Brain-Computer Interface Mental Arithmetic Data: a Comparison Between Different Classifiers”; Int. Conf. IEEE Engineering in Medicine and Biology Society 2014, 2004-2007. http://doi.org/10.1109/EMBC.2014.6944008.
[24]    Zhang, S.; Zheng, Y.; Wang, D.; Wang, L.; Ma, J.; Zhang, J.; Xu, W.; Li, D.; Zhang, D. “Application of a Common Spatial Pattern-Based Algorithm for an fNIRS-Based Motor Imagery Brain‐Computer Interface”; Neurosci. Lett. 2017, 655, 35-40. http://doi.org/10.1016/j.neulet.2017.06.044
[25]    Erdoĝan, S. B.; Özsarfati, E.; Dilek, B.; Kadak, S.L.; Hanoĝlu, K.; Akın, A. “Classification of Motor Imagery and Execution Signals with Population-Level Feature Sets: Implications for Probe Design in fNIRS Based BCI”; J. Neural Eng. 2019, 16, 026029. http://doi.org/10.1088/1741-2552/aafdca
[26]    Noori, F. M.; Naseer, N.; Qureshi, N. K.; Nazeer, H.; Khan, R.A. “Optimal Feature Selection from fNIRS Signals Using Genetic Algorithms for BCI”; Neurosci. Lett. 2017, 647, 61-66.  http://doi.org/10.1016/j.neulet.2017.03.013.
[27]    Trakoolwilaiwan, T.; Behboodi, B.; Lee, J.; Kim, K.; Choi, J.-W. “Convolutional Neural Network for High-Accuracy Functional Near-Infrared Spectroscopy in a Brain–Computer Interface: Three-Class Classification of Rest, Right-, and Left-hand Motor Execution”; Neurophotonics 2017, 5, 011008. http://doi.org/10.1117/1.NPh.5.1.011008.
[28]    Chhabra, H.; Shajil, N.; Venkatasubramanian, G.; “Investigation of Deep Convolutional Neural Network for Classification of Motor Imagery fNIRS Signals for BCI Applications”; Biomed. Signal Proc. Cont. 2020, 62, 102133. http://doi.org/10.1016/j.bspc.2020.102133.
[29]    Fazli, S.; Mehnert, J.; Steinbrink, J.; Curio, G.; Villringer, A.; Müller, K.R.; Blankertz, B. “Enhanced Performance by a Hybrid NIRS–EEG Brain Computer Interface”; Neuroimage 2012, 59, 519-529. http://doi.org/10.1016/j.neuroimage. 2011.07. 084.
[30]    Liu, Y.; Ayaz, H.; Shewokis, P. A. “Multi-Subject Learning for Mental Workload Classification Using Concurrent EEG, fNIRS, and Physiological Measures”; Fronti. Hum. Neurosci. 2017, 11, 389. http://doi.org/10.3389/fnhum. 2017.00389.
[31]    
 
Firooz. S.; Setarehdan, S. K. “IQ Estimation by Means of EEG-fNIRS Recordings During a Logical-Mathematical Intelligence Test”; Computers in biology and medicine, 2019, 110, 218-226. http://doi.org/ 10.1016/j.compbiomed. 2019.05.017.
[32] Kim, H. J.; Wang, I. N.; Kim, Y. T.; Kim, H.; Kim, D. J. “Comparative analysis of NIRS-EEG Motor Imagery Data Using Features from Spatial, Spectral and Temporal Domain”; IEEE Int. Conf. Brain-Computer Interface (BCI) 2020, 1-4. http://doi.org/10.1109/BCI48061.2020.9061636.
[33] Chiarelli, A. M.; Croce, P.; Merla, A.; Zappasodi, F. “Deep Learning for Hybrid EEG-fNIRS Brain–Computer Interface: Application to Motor Imagery Classification”; J. Neural Eng. 2018, 15, 3, 036028. http://doi.org/10.1088/1741-2552/aaaf82.
[34]    Rahman, M.; Uddin, M. S.; Ahmad, M. “Modeling and Classification of Voluntary and Imagery Movements for Brain–Computer Interface from fNIR and EEG Signals Through Convolutional Neural Network”; Health Info. Sci. Syst. 2019, 7, 1-22. http://doi.org/10.1007/s13755-019-00815.
[35]    Kwak, Y., Song; W.-J.; Kim, S. E. “FGANet: fNIRS-Guided Attention Network for Hybrid EEG-fNIRS Brain-Computer Interfaces,” IEEE Trans. Neu. Sys. Reh. Eng. 2022, 30,   329-33. http://doi.org/10.1109/TNSRE.2022.3149899.
[36]    Chen, J.; Wang, D.; Hu, B.; Yi, W.; Xu, M.; Chen, D.; Zhao, Q. “MCFHNet: Multi-Channel Fusion Hybrid Network for Efficient EEG-fNIRS Multi-modal Motor Imagery Decoding” in 44th Annual Intl. Conf. IEEE Eng. Med. & Bio.Soc.(EMBC) 2022, 4821-4825. http://doi:.org/10.1109/ EMBC48229.2022.9871385.
[37]    Hosni, S. M. I.; Borgheai, S. B.; McLinden, J.; Zhu, S.; Huang, X.; Ostadabbas, S.; Shahriari, Y. “A Graph-Based Nonlinear Dynamic Characterization of Motor Imagery Toward an Enhanced Hybrid BCI” Neuroinformatics 2022, 20, 1169-1189. http://doi.org/10.1007/s12021-022-09595-2.
[38]    Shin, J.; von Lühmann, A.; Blankertz, B.; Kim, D.W.; Jeong, J.; Hwang, H. J; Müller, K.R. “Open Access Dataset for EEG+ NIRS Single-Trial Classification”; IEEE Trans. Neural Syst. Rehabil. Eng. 2016, 25, 1735-1745. http://doi.org/10.1109/TNSRE.2016.2628057.
[39] Khan, M. J.; Hong, K. S. “Hybrid EEG–fNIRS-Based Eight-Command Decoding for BCI: Application to Quadcopter Control”; Fronti. Neurorobotics 2017, 11, 6.  http://doi.org/ 10.3389/fnbot.2017.00006.
[40] Yang, J.; Yao, S.; Wang, J. “Deep Fusion Feature Learning Network for MI-EEG Classification”; IEEE Access 2018, 6, 79050-79059. http://doi.org/10.1109/ACCESS.2018. 2877452
[41] Zeng, H.; Yang, C.; Dai, G.; Qin, F.; Zhang, J.; Kong, W.; “Classification of Driver Mental States by Deep Learning Cognitive”; Neurodyn 2018, 12, 597-606. http://doi.org/ 10.1007/ s11571-018-9496-y
[42] Zhang, D.; Yao, L.; Zhang, X.; Wang, S.; Chen, W.; Boots, R; Benatallah, B. “Cascade and Parallel Convolutional Recurrent Neural Networks on EEG-Based Intention Recognition for Brain Computer Interface”; AAAI Conf. Art. Intell. 2018.  http://doi.org/10.48550/arXiv.1708.06578
[43]    Ho, T. K. K.; Gwak, J.; Park, C. M.; Khare, A.; Song, J. I. “Deep Leaning-Based Approach for Mental Workload Discrimination from Multi-Channel fNIRS”; Recent Trends in Communication, Computing, and Electronics: Springer 2019, 431-440. http://doi.org/10.1007/978-981-13-2685-1-41.