Deep Learning Model for Decoding Subcortical Brain Activity from Simultaneous EEG-FMRI Multi-modal Data

Combining Electroencephalography (EEG) and Functional Magnetic Resonance (fMRI) EEG and fMRI can capitalize on the strengths of both techniques to provide a powerful method for examining subcortical activity of the brain. While fMRI is very good at finding out where in space brain activity happens, EEG has very high temporal resolution and can pick up the speed of neural dynamics. A model is designed in this work using deep learning framework to predict subcortical Blood-Oxygen-Level-Dependent (BOLD) signals from multichannel EEG data. It shows promise in decoding subcortical neural activity. The study’s dataset contains EEG-fMRI recordings captured while eight subjects opened and closed their eyes. The EEG signals underwent preprocessing steps that included powerline noise removal, bandpass filtering (1–100 Hz), and artifact rejection using Independent Component Analysis (ICA). The signal then underwent normalization and the noise of each channel was reduced using means of common averaging. A multi-head autoencoder was designed to effectively learn important spatiotemporal features by implementing convolutional layers for optimum downsampling and upsampling. The usefulness of the model was improved through hyper tuning based on Optuna, and the efficiency of the model was measured using Mean Squared Error (MSE), Pearson correlation value (r), and manifold covariance distance. In the subcortical regions, the average correlation (r) of 0.56 of the finding is much better than the baseline benchmark of 0.48. Future studies could involve expanding the dataset, looking at other cognitive tasks, and using more advanced signal processing techniques. Also, the predictive strength of the model may be improved by using multimodal fusion strategies.