Keywords

feedforward and feedback processing in the ventral stream, computational models of vision, similarity-based fusion of MEG and fMRI

Abstract

Successful models of vision, such as DNNs and HMAX, are inspired by the human visual system, relying on a hierarchical cascade of feedforward transformations akin to the ventral stream. Despite these advances, the human visual cortex remains unique in complexity, with feedforward and feedback pathways characterized by rapid spatiotemporal dynamics as visual information is transformed into semantic content. Thus, a systematic characterization of the spatiotemporal and representational space of the ventral visual pathway can offer novel insights in the duration and sequencing of cognitive processes, suggesting computational constraints and new architectures for computer vision models.

To discern the feedforward and feedback neural processes underlying human vision, we used MEG/fMRI fusion. We collected MEG data while observers viewed a rapid-serial-visual-presentation of 11 images with an extremely fast speed (17ms/picture or 34ms/picture). Participants performed a two-alternative forced choice task reporting whether the middle image is a face or non-face. fMRI data while observers viewed the same stimuli were also collected.

We used MVPA to pairwise compare all stimuli, creating RDMs separately for MEG and fMRI data. Comparison of time-resolved MEG-RDMs with space-resolved fMRI-RDMs yielded a spatiotemporal description of the ventral stream dynamics. Starting from EVC, brain activation progressed rapidly to IT within approximately 110ms from stimulus onset. The activation cascade reversed back to EVC at around 170ms. This was accompanied by a strengthening of IT activation, leading to categorical representations enhancement.

The presented well-defined spatiotemporal dynamics can be used as constraints for developing new computational neuroscience models with recursive processes, to increase performances in challenging visual conditions.

Start Date

18-5-2017 2:44 PM

End Date

18-5-2017 3:06 PM

Share

COinS
 
May 18th, 2:44 PM May 18th, 3:06 PM

Similarity-based fusion of MEG and fMRI discerns early feedforward and feedback processing in the ventral stream

Successful models of vision, such as DNNs and HMAX, are inspired by the human visual system, relying on a hierarchical cascade of feedforward transformations akin to the ventral stream. Despite these advances, the human visual cortex remains unique in complexity, with feedforward and feedback pathways characterized by rapid spatiotemporal dynamics as visual information is transformed into semantic content. Thus, a systematic characterization of the spatiotemporal and representational space of the ventral visual pathway can offer novel insights in the duration and sequencing of cognitive processes, suggesting computational constraints and new architectures for computer vision models.

To discern the feedforward and feedback neural processes underlying human vision, we used MEG/fMRI fusion. We collected MEG data while observers viewed a rapid-serial-visual-presentation of 11 images with an extremely fast speed (17ms/picture or 34ms/picture). Participants performed a two-alternative forced choice task reporting whether the middle image is a face or non-face. fMRI data while observers viewed the same stimuli were also collected.

We used MVPA to pairwise compare all stimuli, creating RDMs separately for MEG and fMRI data. Comparison of time-resolved MEG-RDMs with space-resolved fMRI-RDMs yielded a spatiotemporal description of the ventral stream dynamics. Starting from EVC, brain activation progressed rapidly to IT within approximately 110ms from stimulus onset. The activation cascade reversed back to EVC at around 170ms. This was accompanied by a strengthening of IT activation, leading to categorical representations enhancement.

The presented well-defined spatiotemporal dynamics can be used as constraints for developing new computational neuroscience models with recursive processes, to increase performances in challenging visual conditions.