Keywords
feedforward, deep learning, convolutional neural networks, psychophysics, computational model
Abstract
Rapid categorization paradigms have a long history in experimental psychology: Characterized by short presentation times and fast behavioral responses, these tasks highlight both the speed and ease with which our visual system processes natural object categories. Previous studies have shown that feed-forward hierarchical models of the visual cortex provide a good fit to human visual decisions. At the same time, recent work has demonstrated significant gains in object recognition accuracy with increasingly deep hierarchical architectures: From AlexNet to VGG to Microsoft CNTK – the field of computer vision has championed both depth and accuracy. But it is unclear how well these models account for human visual decisions and what they may reveal about the underlying brain processes.
Here, we report initial results on a large-scale psychophysical study using Amazon Mechanical Turk to assess the correlation between computational models and human participants on a rapid animal vs. non-animal categorization task. We considered visual representations of varying complexity by analyzing the output of different stages of processing in a state-of-the-art deep network. We found that recognition accuracy increases with higher stages of visual processing (higher level stages indeed outperforming human participants for the same task) but that human decisions agreed best with predictions from intermediate stages. Overall, these results suggest that human participants may rely on visual features of intermediate complexity and that the complexity of visual representations afforded by modern deep network models may exceed that used by human participants during rapid categorization.
Funding: This work is supported by NSF early career award (IIS-1252951) and DARPA young faculty award (N66001-14-1-4037) to T.S.
Start Date
11-5-2016 4:50 PM
End Date
11-5-2016 5:15 PM
Location
Cognitive Linguistic & Psychological Sciences Department; Brown University, Providence RI 02912
How Deep is the Feature Analysis Underlying Rapid Visual Categorization?
Cognitive Linguistic & Psychological Sciences Department; Brown University, Providence RI 02912
Rapid categorization paradigms have a long history in experimental psychology: Characterized by short presentation times and fast behavioral responses, these tasks highlight both the speed and ease with which our visual system processes natural object categories. Previous studies have shown that feed-forward hierarchical models of the visual cortex provide a good fit to human visual decisions. At the same time, recent work has demonstrated significant gains in object recognition accuracy with increasingly deep hierarchical architectures: From AlexNet to VGG to Microsoft CNTK – the field of computer vision has championed both depth and accuracy. But it is unclear how well these models account for human visual decisions and what they may reveal about the underlying brain processes.
Here, we report initial results on a large-scale psychophysical study using Amazon Mechanical Turk to assess the correlation between computational models and human participants on a rapid animal vs. non-animal categorization task. We considered visual representations of varying complexity by analyzing the output of different stages of processing in a state-of-the-art deep network. We found that recognition accuracy increases with higher stages of visual processing (higher level stages indeed outperforming human participants for the same task) but that human decisions agreed best with predictions from intermediate stages. Overall, these results suggest that human participants may rely on visual features of intermediate complexity and that the complexity of visual representations afforded by modern deep network models may exceed that used by human participants during rapid categorization.
Funding: This work is supported by NSF early career award (IIS-1252951) and DARPA young faculty award (N66001-14-1-4037) to T.S.