Neuroimaging

Naturalistic stimuli: a paradigm for multi-scale functional characterization of the human brain

(Zhang, Kim, Brang, Liu, Current Opinion in Biomedical Engineering, 2021)

Movies, audio stories, and virtual reality are increasingly used as stimuli for functional brain imaging. Such naturalistic paradigms are in sharp contrast to the tradition of experimental reductionism in neuroscience research. Being complex, dynamic, and diverse, naturalistic stimuli set up a more ecologically relevant condition and induce highly reproducible brain responses across a wide range of spatiotemporal scales. Here, we review recent technical advances and scientific findings on imaging the brain under naturalistic stimuli. Then, we elaborate on the premise of using naturalistic paradigms for multiscale, multimodal, and high-throughput functional characterization of the human brain. We further highlight the growing potential of using deep learning models to infer neural information processing from brain responses to naturalistic stimuli. Finally, we advocate large-scale collaborations to combine brain imaging and recording data across experiments, subjects, and labs that use the same set of naturalistic stimuli.

Representation learning of resting state fMRI with variational auto-encoder

(Kim, Zhang, Han, Wen, Choi, Liu, NeuroImage, 241: 118423, 2021)

Resting state functional magnetic resonance imaging (rsfMRI) data exhibits complex but structured patterns. However, the underlying origins are unclear and entangled in rsfMRI data. Here we establish a variational auto-encoder, as a generative model trainable with unsupervised learning, to disentangle the unknown sources of rsfMRI activity. After being trained with large data from the Human Connectome Project, the model has learned to represent and generate patterns of cortical activity and connectivity using latent variables. The latent representation and its trajectory represent the spatiotemporal characteristics of rsfMRI activity. The latent variables reflect the principal gradients of the latent trajectory and drive activity changes in cortical networks. Representational geometry captured as covariance or correlation between latent variables, rather than cortical connectivity, can be used as a more reliable feature to accurately identify subjects from a large group, even if only a short period of data is available in each subject. Our results demonstrate that VAE is a valuable addition to existing tools, particularly suited for unsupervised representation learning of resting state fMRI activity.

Task-evoked functional connectivity does not explain functional connectivity differences between rest and task conditions

(Lynch, Lu, Wen, Zhang, Saykin, Liu. 2018. Human Brain Mapping, 39(12): 4939-4948)

During complex tasks, patterns of functional connectivity differ from those in the resting state. However, what accounts for such differences remains unclear. Brain activity during a task reflects an unknown mixture of spontaneous and task-evoked activities. The difference in functional connectivity between a task state and the resting state may reflect not only task-evoked functional connectivity, but also changes in spontaneously emerging networks. Here, we characterized the differences in apparent functional connectivity between the resting state and when human subjects were watching a naturalistic movie. Such differences were marginally explained by the task-evoked functional connectivity involved in processing the movie content. Instead, they were mostly attributable to changes in spontaneous networks driven by ongoing activity during the task. The execution of the task reduced the correlations in ongoing activity among different cortical networks, especially between the visual and non-visual sensory or motor cortices. Our results suggest that task-evoked activity is not independent from spontaneous activity, and that engaging in a task may suppress spontaneous activity and its inter-regional correlation.  

Musical Imagery Involves Wernicke’s area in Bilateral and Anti-Correlated Network Interactions

(Zhang, Chen, Wen, Lu, Liu. 2017. Scientific Reports, 7:17066)

Musical imagery is the human experience of imagining music without actually hearing it. The neural basis of this mental ability is unclear, especially for musicians capable of engaging in accurate and vivid musical imagery. Here, we created a visualization of an 8-minute symphony as a silent movie and used it as real-time cue for musicians to continuously imagine the music for repeated and synchronized sessions during functional magnetic resonance imaging (fMRI). The activations and networks evoked by musical imagery were compared with those elicited by the subjects directly listening to the same music. Musical imagery and musical perception resulted in overlapping activations at the anterolateral belt and Wernicke’s area, where the responses were correlated with the auditory features of the music. Whereas Wernicke’s area interacted within the intrinsic auditory network during musical perception, it was involved in much more complex networks during musical imagery, showing positive correlations with the dorsal attention network and the motor-control network and negative correlations with the default-mode network. Our results highlight the important role of Wernicke’s area in forming vivid musical imagery through bilateral and anti-correlated network interactions, challenging the conventional view of segregated and lateralized processing of music versus language.   

Spontaneous activity is organized by visual streams

(Lu, Jeong, Wen, Liu. 2017. Human Brain Mapping 38: 4613-4630)

Large-scale functional networks have been extensively studied using resting state functional magnetic resonance imaging. However, the pattern, organization, and function of fine-scale network activity remain largely unknown. Here we characterized the spontaneously emerging visual cortical activity by applying independent component analysis to resting state fMRI signals exclusively within the visual cortex. In this sub-system scale, we observed about 50 spatially independent components that were reproducible within and across subjects, and analyzed their spatial patterns and temporal relationships to reveal the intrinsic parcellation and organization of the visual cortex. The resulting visual cortical parcels were aligned with the steepest gradient of cortical myelination, and were organized into functional modules segregated along the dorsal/ventral pathways and foveal/peripheral early visual areas. Cortical distance could partly explain intra-hemispherical functional connectivity, but not inter-hemispherical connectivity; after discounting the effect of anatomical affinity, the fine-scale functional connectivity still preserved similar visual-stream-specific modular organization. Moreover, cortical retinotopy, folding, and cytoarchitecture impose limited constraints to the organization of resting state activity. Given these findings, we conclude that spontaneous activity patterns in the visual cortex are primarily organized by visual streams, likely reflecting feedback network interactions.   

Imaging white-matter functional organization

(Marussich, Lu, Wen, Liu. 2017. NeuroImage, 146: 128-1141)

 Despite wide applications of functional magnetic resonance imaging (fMRI) to mapping brain activation and connectivity in cortical gray matter, it has rarely been utilized to study white-matter functions. In this study, we investigated the spatiotemporal characteristics of fMRI data within the white matter acquired from humans in the resting state or watching a naturalistic movie. By using independent component analysis and hierarchical clustering, resting-state fMRI data in the white matter were denoised and decomposed into spatially independent components, further assembled into hierarchically organized axonal fiber bundles. Interestingly, such components were partly reorganized during natural vision. Relative to the resting state, the visual task specifically induced a stronger degree of temporal coherence within the optic radiations, as well as significant correlations between the optic radiations and multiple cortical visual networks. Therefore, fMRI contains rich functional information about activity and connectivity within white matter at rest and during tasks, challenging the conventional practice of taking white-matter signals as noise or artifacts.   

Influences of high-level features, gaze, and scene transitions on cortical responses to natural movies

(Lu, Hung, Wen, Marussich, Liu. 2016. Plos ONE, 11(8): e0161797)

Complex, sustained, dynamic, and naturalistic visual stimulation can evoke distributed brain activities that are highly reproducible within and across individuals. However, the precise origins of such reproducible responses remain incompletely understood. Here, we employed concurrent functional magnetic resonance imaging (fMRI) and eye tracking to investigate the experimental and behavioral factors that influence fMRI activity and its intra- and inter-subject reproducibility during repeated movie stimuli. We found that widely distributed and highly reproducible fMRI responses were attributed primarily to the high-level natural content in the movie. In the absence of such natural content, low-level visual features alone in a spatiotemporally scrambled control stimulus evoked significantly reduced degree and extent of reproducible responses, which were mostly confined to the primary visual cortex (V1). We also found that the varying gaze behavior affected the cortical response at the peripheral part of V1 and at areas in the oculomotor network, with minor effects on the response reproducibility over the extrastriate visual areas. Lastly, scene transitions in the movie stimulus due to film editing partly caused the reproducible fMRI responses at widespread cortical areas, especially along the ventral visual pathway. Therefore, the naturalistic nature of a movie stimulus is necessary for driving highly reliable visual activations. In a movie-stimulation paradigm, scene transitions and individuals’ gaze behavior should be taken as potential confounding factors in order to properly interpret cortical activity that supports natural vision.  

Scale-free electrophysiology contributes to global fMRI

(Wen and Liu. 2016. J Neurosci. 32:6030-6040).

This study provides new insights into the neural origin of resting-state fMRI. Results demonstrate that the broadband power fluctuation of scale-free electrophysiology is globally synchronized and directly coupled with the global component of spontaneous fMRI signals, in contrast to modularly synchronized fluctuations in oscillatory neural activity. These findings lead to a new hypothesis that scale-free and oscillatory neural processes account for global and modular patterns of functional connectivity in resting-state fMRI, respectively.  

Separation of oscillatory and fractal dynamics in electrophysiological signals

(Wen and Liu, 2016. Brain Topography. 29:13-26).

Neurophysiological field-potential signals consist of both arrhythmic and rhythmic patterns indicative of the fractal and oscillatory dynamics arising from likely distinct mechanisms. Here, we present a new method, namely the Irregular-Resampling Auto-Spectral Analysis (IRASA), to separate fractal and oscillatory components in the power spectrum of neurophysiological signal according to their distinct temporal and spectral characteristics. In this method, we irregularly resampled the neural signal by a set of non-integer factors, and statistically summarized the auto-power spectra of the resampled signals to separate the fractal component from the oscillatory component in the frequency domain. We tested this method on simulated data and demonstrated that IRASA could robustly separate the fractal component from the oscillatory component. In addition, applications of IRASA to macaque electrocorticography (ECoG) and human magnetoencephalography (MEG) data revealed a greater power-law exponent of fractal dynamics during sleep compared to wakefulness. The temporal fluctuation in the broadband power of the fractal component revealed characteristic dynamics within and across the eyes-closed, eyes-open and sleep states. These results demonstrate the efficacy and potential applications of this method in analyzing electrophysiological signatures of large-scale neural circuit activity. We expect that the proposed method or its future variations would potentially allow for more specific characterization of the differential contributions of oscillatory and fractal dynamics to distributed neural processes underlying various brain functions.  

Neuroelectrical decomposition of resting state fMRI

(Liu et al., 2014. Cerebral Cortex 24: 3080-3089)

Spontaneous activity in the human brain occurs in complex spatiotemporal patterns that may reflect functionally specialized neural networks. Here we propose a subspace analysis method to elucidate large scale networks by joint analysis of electroencephalography (EEG) and fMRI data. The new approach is based on the notion that the neuro-electrical activity underlying the fMRI signal may have EEG spectral features that report on regional neuronal dynamics and inter-regional interactions. Applying this approach to resting healthy adults, we indeed found characteristic spectral signatures in the EEG correlates of spontaneous fMRI signals at individual brain regions as well as the temporal synchronization among widely distributed regions. These spectral signatures not only allowed us to parcel the brain into clusters that resembled the brain’s established functional subdivision, but also offered important clues for disentangling the involvement of individual regions in fMRI network activity.