Abstract
Decoding fine hand movement intention from electroencephalography (EEG) remains challenging due to the poorly discriminative spatial features extracted from sensorimotor areas. A novel action observation (AO) paradigm, which can simultaneously elicit steady-state motion visual evoked potentials (SSMVEP) over the occipital region and sensorimotor rhythm (SMR) from the sensorimotor area, offers a promising alternative to improve recognition accuracy. In this study, we first conducted a systematic analysis of SMR features induced by fine hand movement AO under different spectral properties and channel selection schemes. These features were then integrated with AO-induced SSMVEP features via a proposed SMR-SSMVEP fusion method. Specifically, discriminative projection distances were used to quantify the confidence of SMR features, while differences in correlation coefficients across AO stimuli served as the confidence measure for SSMVEP features. The confidence values of both modalities were linearly weighted according to preset weights and compared to form the final decision. Experimental results show that the optimized AO-induced SMR features achieved an average accuracy of 47.67% ± 13.83% in a four-class fine hand movement intention recognition and 72.40% ± 12.72% in a binary left-right hand movement intention recognition. Furthermore, the proposed fusion method attained an average accuracy of 80.69% ± 13.77% in four-class fine hand movement intention recognition, with 14 participants achieving accuracies exceeding 90%. This work provides a systematic evaluation of AO-induced neural features and presents a practical SMR-SSMVEP fusion approach for enhanced AO-based BCI decoding.