Decoding hierarchical visual feature representations in mental imagery of aphantasics

Abstract

Mental imagery, regarded as a quasi-perceptual phenomenon, typically generates visual experiences without external stimuli and recruits neural representations associated with visual features that overlap with those used in visual perception. While these shared neural representations are often assumed to underlie the phenomenal aspects of conscious visual experiences, their contribution to subjective visual awareness remains unclear. Aphantasia, characterized by the subjective inability to voluntarily generate visual mental imagery, offers a unique opportunity to examine how mental images are represented without subjective visual experience and to test the relationship between neural representations and conscious experience. We thus measured brain activity in aphantasics as they viewed and imagined object images using functional magnetic resonance imaging, applying deep neural network (DNN) feature decoding to comprehensively analyze hierarchical visual feature representations across multiple levels. Decoding models were trained on stimulus-induced brain activity in visual cortical areas to predict feature values of viewed images. We then applied these models to imagery-induced brain activity to decode DNN features, comparing them with category-averaged features from a large-scale image database. Feature values decoded from imagery-induced brain activity positively correlated with those of imagined objects across multiple DNN layers, enabling above-chance identification of imagined object categories by matching them to averaged candidate features. Remarkably, while the decoding performance of aphantasics was somewhat lower, it approached that of typical imagers. These results demonstrate that neural representations of hierarchical visual features, shared with visual perception, are engaged during mental imagery in aphantasia, even without subjective visual awareness. This dissociation between neural representations and visual awareness highlights significant limitations in assuming that decodable neural representations directly reflect the neural correlates of consciousness. While supporting the presence of mental imagery, decodable representations alone fall short of fully explaining conscious experience, underscoring the need for cautious interpretation as evidence of consciousness.

Decoding hierarchical visual feature representations in mental imagery of aphantasics

Abstract

Mental imagery, regarded as a quasi-perceptual phenomenon, typically generates visual experiences without external stimuli and recruits neural representations associated with visual features that overlap with those used in visual perception. While these shared neural representations are often assumed to underlie the phenomenal aspects of conscious visual experiences, their contribution to subjective visual awareness remains unclear. Aphantasia, characterized by the subjective inability to voluntarily generate visual mental imagery, offers a unique opportunity to examine how mental images are represented without subjective visual experience and to test the relationship between neural representations and conscious experience. We thus measured brain activity in aphantasics as they viewed and imagined object images using functional magnetic resonance imaging, applying deep neural network (DNN) feature decoding to comprehensively analyze hierarchical visual feature representations across multiple levels. Decoding models were trained on stimulus-induced brain activity in visual cortical areas to predict feature values of viewed images. We then applied these models to imagery-induced brain activity to decode DNN features, comparing them with category-averaged features from a large-scale image database. Feature values decoded from imagery-induced brain activity positively correlated with those of imagined objects across multiple DNN layers, enabling above-chance identification of imagined object categories by matching them to averaged candidate features. Remarkably, while the decoding performance of aphantasics was somewhat lower, it approached that of typical imagers. These results demonstrate that neural representations of hierarchical visual features, shared with visual perception, are engaged during mental imagery in aphantasia, even without subjective visual awareness. This dissociation between neural representations and visual awareness highlights significant limitations in assuming that decodable neural representations directly reflect the neural correlates of consciousness. While supporting the presence of mental imagery, decodable representations alone fall short of fully explaining conscious experience, underscoring the need for cautious interpretation as evidence of consciousness.