Functional near-infrared spectroscopy (fNIRS) as a hybrid system: a review

Sensor devices and biomedical imaging technologies used in clinical application scenarios are essential for providing a comprehensive portrait of patients’ state, but these technologies, despite their outstanding advantages, have their inherent disadvantages. Beginning with the principle of complementary images of medical imaging techniques, this review examines the functional near- infrared spectroscopy (fNIRS) technique and its use as a hybrid system. The fNIRS technology delivers impressive results in terms of the biological signal classification accuracy, but its use as a hybrid system with electroencephalography (EEG) and electromyography (EMG) achieved better results because it has become a complementary tool to fill the deficit of the common technology with it, and this has been highlighted in this review. The results show that the superiority in the biological signal classification accuracy provided by hybrid systems from fNIRS with EEG and EMG would provide a comprehensive and objective assessment of the patients’ state from the stage of illness to healing. In conclusion, we have no indication from the scientific studies of the previous four years (2020–2023) that demonstrate which of the hybrid systems is better than others when used in clinical practice, and this encourages further in-depth studies to validate the combination of methods to prove their success and preference.

1. Hramov A.E., Maksimenko V.A., Pisarchik A.N. Physical principles of brain–computer interfaces and their applications for rehabilitation, robotics and control of human brain states. Physics Reports. 2021;918:1–133. DOI: 10.1016/j.physrep.2021.03.002.

2. Kwon J., Shin J., Im C.H. Toward a compact hybrid brain-computer interface (BCI): Performance evaluation of multi-class hybrid EEG-fNIRS BCIs with limited number of channels. PLoS One. 2020;15:3. DOI: 10.1371/journal.pone.0230491.

3. Tyatyushkina O.Y., Ulyanov S.V. «Brain – Computer» interface (BCI). Pt I: Classical technology. Systemanalysis in science and education. 2023;(2):62–96. EDN: HNAGLM. URL: https://sanse.ru/index.php/sanse/article/view/581.

4. Lin J.F.L. Dual-MEG interbrain synchronization during turn-taking verbal interactions between mothers and children. Cerebral Cortex. 2023;33(7):4116–4134. DOI: 10.1093/cercor/bhac330.

5. Berestov R.M., Bobkov E.A., Belov V.S., Nevedin A.V. Brain-computer interface technologies for monitoring and control of bionic systems. Journal of Physics: Conference Series. 2021;46(4):924–927. DOI: 10.1364/OL.418284.

6. Daniel N., Sybilski K., Kaczmarek W., Siemiaszko D., Małachowski J. Relationship between EMG and fNIRS during Dynamic Movements. Sensors. 2023;23(11). DOI: 10.3390/s23115004.

7. Pereira J., Direito B., Lührs M., Castelo-Branco M., Sousa T. Multimodal assessment of the spatial correspondence between fNIRS and fMRI hemodynamic responses in motor tasks. Sci Rep. 2023;13:1. DOI: 10.1038/s41598-023-29123-9.

8. Tsvetanov K.A., Henson R.N.A., Rowe J.B. Separating vascular and neuronal effects of age on fMRI BOLD signals: Neurovascular ageing. Philosophical Transactions of the Royal Society B: Biological Sciences. 2021;376:1815. DOI: 10.1098/rstb.2019.0631.

9. Androu A., Daniel M., David J.C., Udunna A., Tracy S., Mamadou D., Adrian M.O., Keith S.L. Using fMRI to investigate the potential cause of inverse oxygenation reported in fNIRS studies of motor imagery. Neurosci Lett. 2020;714:134607. DOI: 10.1016/j.neulet.2019.134607.

10. Sergio L.N., Alex C.C., Forti R.M., Fernado C., Clarissa L.Y., Rickson C.M. Revealing the spatiotemporal requirements for accurate subject identification with resting-state functional connectivity: a simultaneous fNIRS-fMRI study. Neurophotonics. 2023;10(1). DOI: 10.1117/1.NPh.10.1.013510.

11. Klein F.S., Debener K.W., Kranczioch C. fMRI-based validation of continuous-wave fNIRS of supplementary motor area activation during motor execution and motor imagery. Sci Rep. 2022;12(1). DOI: 10.1038/s41598-022-06519-7.

12. Deligani R.J., Borgheai S.B., McLinden J., Shahriari Y. Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework. Biomed Opt Express. 2021;12(3):1635. DOI: 10.1364/boe.413666.

13. Asanza V., Pelaez E., Loayza F., Lorente-Leyva L.L. Peluffo-Ordonez D.H. Identification of lower-limb motor tasks via brain-computer interfaces: a topical overview. Sensors. 2022;22(5). DOI: 10.3390/s22052028.

14. Janis P., Dmytro M. State-of-the-art on brain-computer interface technology. Sensors. 2023;23(13). Multidisciplinary Digital Publishing Institute (MDPI). DOI: 10.3390/s23136001.

15. Lacerenza M., Frabasile L., Buttafava M., Spinelli L., Bassani E., Micheloni F., Amendola C., Torricelli A., Contini D. Motor cortex hemodynamic response to goal-oriented and non-goal-oriented tasks in healthy subjects. Front Neurosci. 2023;17: 1202705. DOI: 10.3389/fnins.2023.1202705.

16. Wang H., Yan F., Xu T., Yin H., Chen P., Yue H., Chen C., Zhang H., X, L., He Y., et al. Brain-controlled wheelchair review: from wet electrode to dry electrode, from single modal to hybrid modal, from synchronous to asynchronous. IEEE Access. 2021;9:55920–55938. DOI: 10.1109/ACCESS.2021.3071599.

17. Xu B., Li W., Liu, D., Zhang K., Miao M., Xu G., Song A. Continuous hybrid BCI control for robotic arm using noninvasive electroencephalogram, computer vision, and eye tracking. Mathematics. 2022;10(4):618. DOI: 10.3390/math10040618.

18. Sun Z., Huan Z., Duan F., Liu Y. A novel multimodal approach for hybrid brain-computer interface. IEEE Access. 2020;8:89909–89918. DOI: 10.1109/ACCESS.2020.2994226.

19. Wilson H., Golbabaee M., Proulx M.J., Charles S., Neill E.O. EEG-based BCI dataset of semantic concepts for imagination and perception tasks. Sci Data. 2023;10(1). DOI: 10.1038/s41597-023-02287-9.

20. Juanning S., Yi Y., Long X., Tianshuai X., Hao L., Yujin Z., Rixing J., Jinglian L., Dongdong W., Sijin W., Jianghong H. Evaluation of residual cognition in patients with disorders of consciousness based on functional near-infrared spectroscopy. Neurophotonics. 2023;10(2):025003. DOI: 10.1117/1.nph.10.2.025003.

21. Hasan M.A.H., Khan M.U., Mishra D. A computationally efficient method for hybrid EEG-fNIRS BCI based on the Pearson correlation. Biomed Res Int. 2020. DOI: 10.1155/2020/1838140.

22. Sial M.B., Wang S., Wang X., Wyrwa J., Ali S. A survey on EEG-fNIRS based non-invasive hBCIs. In: 2021 International Conference on Artificial Intelligence (ICAI), Islamabad, Pakistan, 2021. p. 240–245. DOI: 10.1109/ICAI52203.2021.9445246.

23. Ailsworth J. Development of neurofeedback therapies for hand rehabilitation in stroke. PhD Thesis. North Carolina State University; 2023.

24. Lubo F., Haoyang L., Hongfei J. EEG-EMG analysis method in hybrid brain computer interface for hand rehabilitation training. Computing and Informatics. 2023;42:741–761. DOI: 10.31577/cai_2023_3_741.

25. Asmaa M., Saeed M.Q., Salankar N., Feng J., Ryszard T., Paweł P., Ahmed A., Mohamed H. Hybrid EEG-fNIRS brain-computer interface based on the non-linear features extraction and stacking ensemble learning. Biocybern Biomed Eng. 2023;43(2):463–475. DOI: 10.1016/j.bbe.2023.05.001.

26. Liu Z., Shore J., Wang M., Yuan F., Buss A., Zhao X. A systematic review on hybrid EEG/fNIRS in brain-computer interface. Biomed Signal Process Control. 2021;68 102595.

27. Tao X., Zhengkang Z. Motor imagery decoding enhancement based on Hybrid EEG- fNIRS signals. IEEE Access. 2023;11:65277–65288. DOI: 10.1109/ACCESS.2023.3289709.

28. Su J., Yang Z., Yang W.Y., Sun W.S. Electroencephalogram classification in motor-imagery brain-computer interface applications based on double constraint nonnegative matrix factorization. Physiol. Meas. 2020;41(7):075007.

29. Hermosilla D.M., Codorni R.T., Codorni R.L., Baracaldo R.S., Zamora D.D. Rodriguez Y.L.A., Alvarez J.R.N. Shallow convolutional network excel for classifying motor imagery EEG in BCI Applications. IEEE Access. 2021;9:98275–98286. DOI: 10.1109/ACCESS.2021.3091399.

30. Cai Y., She Q., Ji J., Ma Y., ZhangBJ., Zhang Y. Motor imagery EEG decoding using manifold embedded transfer learning. J. Neurosci. Methods. 2022;370:109489.

31. Eda A.A. Subject-specific feature selection for near infrared spectroscopy based brain–computer interfaces. Comput. Methods Programs Biomed. 2020;195:105535.

32. Ghaffar M.S.B.A., Khan U.S., Naseer N., Rashid N., Tiwana M.I. Improved classification accuracy of four class FNIRS-BCI. In: Proc.12th Int. Conf. Electron., Comput. Artif. Intell. (ECAI). Bucharest, Romania: IEEE, Jun. 2020. p. 1–5.

33. Wang H.K., Kim I., Kim Y., Kim H., Kim D. Comparative analysis of NIRS-EEG motor imagery data using features from spatial, spectral and temporal domain. In: Proc. 8th Int. Winter Conf. Brain-Comput. Interface (BCI), Feb. 2020. p. 1–4.

34. Asadullaev R.R., Afonin A.N., Shchetinina E.S. Recognition of patterns of motor activity by a neural network based on continuous optical tomography fNIRS data. Ekonomika. Informatica. 2021:48(4):735–746. DOI: 10.52575/2687-0932-2021-48-4-735-746. (In Russ.).

35. Cicalese P.A., Li R., Ahmadi M.B., Wang C., Francis J.T., Selvaraj S., Schulz P.E., Zhang Y. An EEG-fNIRS hybridization technique in the four-class classification of Alzheimer’s disease. J. Neurosci. Methods. 2020;336:108618.

36. Arshia A., Khan M.J., Kashif J., Hasan S., Saddaf R., Noman N., Khan T.I. Hemodynamic response detection using integrated EEG-fNIRS-VPA for BCI. Computers, Materials and Continua. 2021;70(1):535–555. DOI: 10.32604/cmc.2022.018318.

37. Kwak Y., Song W.J., Kim S.E. FGANet: fNIRS-guided attention network for hybrid EEG-fNIRS brain-computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2022;30:329–339. DOI: 10.1109/TNSRE.2022.3149899.

38. Abdalmalak A. Detecting command-driven brain activity in patients with detecting command-driven brain activity in patients with disorders of consciousness using TR-fNIRS. PhD thesis. The University of Western Ontario; 2020.

39. Jianeng L., Jiewei L., Zhilin S., Ningbo Y., Jianda H. An EEG-fNIRS neurovascular coupling analysis method to investigate cognitive-motor interference. Computers in Biology and Medicine. 2023;160:106968. DOI: 10.1016/j.compbiomed.2023.106968.

40. Li R., Yang D., Fang F., Hong K.S., Reiss A.L., Zhang Y. Concurrent fNIRS and EEG for brain function investigation: a systematic, methodology-focused review. Sensors. 2022;22(15). DOI: 10.3390/s22155865.

41. Ergün E., Aydemir Ö., Korkmaz O.E. A novel scrolling text reading paradigm for improving the performance of multiclass and hybrid brain computer interface systems. Available at SSRN: https://ssrn.com/abstract=4576653. DOI: 10.2139/ssrn.4576653.

42. Radek M., Martina L., Michaela S., Rene J., Khosrow B., Radana K., Aleksandra K.S. Advanced bioelectrical signal processing methods: Past, present, and future approach—part iii: Other biosignals. Sensors. 2021;21(18). DOI: 10.3390/s21186064.

43. Chunfu L., Ruite G., Zhichuan T., Xiaoyun F., Lekai Z., Keshuai Y., Xuan X. Multi-channel FES gait rehabilitation assistance system based on adaptive sEMG modulation. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2023;31:3652–3663. DOI: 10.1109/tnsre.2023.3313617.

44. Song T., Yan Z., Guo S., Li Y., LiX., Xi F. Review of sEMG for robot control: Techniques and applications. Applied Sciences (Switzerland). 2023;13(17). DOI: 10.3390/app13179546.

45. Cheng X., Sie E.J., Boas D.A., Marsili F. Choosing an optimal wavelength to detect brain activity in functional near-infrared spectroscopy. Opt Lett. 2021;46(4):924. DOI: 10.1364/ol.418284.

46. Taborri J., Keogh J., Kos A. et al. Sport biomechanics applications using inertial, force, and EMG sensors: a literature overview. Appl Bionics Biomech. 2020;2020:2041549. DOI: 10.1155/2020/2041549.

47. Neelum Y.S., Zareena K. et al. Enhancing classification accuracy of transhumeral prosthesis: a hybrid sEMG and fNIRS approach. IEEE Access. 2021;9:113246–113257. DOI: 10.1109/ACCESS.2021.3099973.

48. Kimoto H.F., Machida M. A wireless multi-layered EMG/MMG/NIRS sensor for muscular activity evaluation. Sensors. 2023;23(3). DOI: 10.3390/s23031539.

49. Giminiani R.D., Marco C., Marco F., Valentina Q. Validation of fabric-based thigh-wearable EMG sensors and oximetry for monitoring quadricep activity during strength and endurance exercises. Sensors. 2020;17:1–13. DOI: 10.3390/s20174664.

50. Walczak E.J. Exploration of speech induced suppression using functional near-infrared spectroscopy (fNIRS). Bioarxiv [Preprint]. DOI: 10.1101/2023.03.05.531176.

51. Siepsiak M., Vrana S.R., Rynkiewicz A., Rosenthal M.Z., Dragan W.Ł. Does context matter in misophonia? A multi-method experimental investigation. Front Neurosci. 2023;16:880853. DOI: 10.3389/fnins.2022.880853.

52. Ma T., Chen W., Li X., Xia Y., Zhu X., He S. Fnirs signal classification based on deep learning in rock-paper-scissors imagery task. Applied Sciences. 2021;11(11). DOI: 10.3390/app11114922.

53. Sebastian H., Erica F., Charisse P., Judit G., Janet F.W., Laurel J.T., Brett B.F. Babies, bugs and brains: How the early microbiome associates with infant brain and behavior development. PLoS One. 2023;18(8). DOI: 10.1371/journal.pone.0288689.

54. Waight J.L., Arias N., Jiménez-García A.M., Martini M. From functional neuroimaging to neurostimulation: fNIRS devices as cognitive enhancers. Behav Res Methods. 2023. DOI: 10.3758/s13428-023-02144-y.

55. Zhang X., Yao L., Wang X., MonaghanJ., Mcalpine D., Zhang Y. A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers. J Neural Eng. 2021;18(3). DOI: 10.1088/1741-2552/abc902.

56. Nadimi-Shahraki M.H., Zamani H., Mirjalili S. Enhanced whale optimization algorithm for medical feature selection: A COVID-19 case study. Comput Biol Med. 2022;148:105858. DOI: 10.1016/j.compbiomed.2022.105858.

57. Ali M.U., Kim K.S., Kallu K.D., Zafar A., Lee S.W. OptEF-BCI: an optimization-based hybrid EEG and fNIRS-brain computer interface. Bioengineering. 2023;10(5):608. DOI: 10.3390/bioengineering10050608.

58. Brekke F. Non-verbal working memory: A functional near-infrared spectroscopy (fNIRS) and functional magnetic resonance imaging (fMRI) comparison. Master’s Thesis. University of Bergen; 2023.

59. Phillips V.Z., Canoy R.J., Paik S.H., Lee S.H., Kim B.M. Functional near-infrared spectroscopy as a personalized digital healthcare tool for brain monitoring. Journal of Clinical Neurology. 2023;19(2):115–124. DOI: 10.3988/jcn.2022.0406.

60. Alessandro S. et al. A narrative review on multi-domain instrumental approaches to evaluate neuromotor function in rehabilitation. Healthcare. 2023;11(16). DOI: 10.3390/healthcare11162282.

61. Mohammadreza A., Seyed B.B., Roohollah J., Nicholas C., Rassoul D., Yalda S., Kunal M. Merging fNIRS-EEG brain monitoring and body motion capture to distinguish Parkinsons disease. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2020;28(6):1246–1253. DOI: 10.1109/TNSRE.2020.2987888.

62. Marwan H.O., Mahasweta B., Kirsten M., Søren K., Johannes G., Jesper K., Anirban D., Daniel K. Resting-state NIRS–EEG in unresponsive patients with acute brain injury: a proof-of-concept study. Neurocrit Care. 2021;34(1):31–44. DOI: 10.1007/s12028-020-00971-x.

63. Zhongpeng W., Cong C., Long C., Bin G., Shuang L., Minpeng X., Feng H., Dong M. Multimodal neural response and effect assessment during a BCI-based neurofeedback training after stroke. Front Neurosci. 2022;16:2022. DOI: 10.3389/fnins.2022.884420.

64. Anneke H., Nils C. Assessing the development of mental fatigue during simulated flights with concurrent EEG-fNIRS measurement. Sci Rep. 2023;13(1). DOI: 10.1038/s41598-023-31264-w.

65. Xiaohan W., Zichong L., Mingxia Z., Weihua Z., Songyun X., Seng F.W., Huijing H., Le L. The interaction between changes of muscle activation and cortical network dynamics during isometric elbow contraction: a sEMG and fNIRS study. Front Bioeng Biotechnol. 2023;11:1176054. DOI: 10.3389/fbioe.2023.1176054.