Modelling perception as a hierarchical competition differentiates imagined, veridical, and hallucinated percepts Alexander A. Sulfaro1,†,*, Amanda K. Robinson1,2, Thomas A. Carlson1 1School of Psychology, Griffith Taylor Building, The University of Sydney, Camperdown, NSW 2006, Australia; 2Queensland Brain Institute, QBI Building 79, The University of Queensland, St Lucia, QLD 4067, Australia †Alexander A. Sulfaro, http://orcid. org/0000-0001-5525-3935 *Corresponding author. School of Psychology, Griffith Taylor Building, The University of Sydney, Camperdown, NSW 2006, Australia. E-mail: alexander. sulfaro@sydney. edu. au Abstract Mental imagery is a process by which thoughts become experienced with sensory characteristics. Yet, it is not clear why mental images appear diminished compared to veridical images, nor how mental images are phenomenologically distinct from hallucinations, another type of non-veridical sensory experience. Current evidence suggests that imagination and veridical perception share neural resources. If so, we argue that considering how neural representations of externally generated stimuli (i. e. sensory input) and internally generated stimuli (i. e. thoughts) might interfere with one another can sufficiently differentiate between veridical, imaginary, and hallucinatory perception. We here use a simple computational model of a serially connected, hierarchical network with bidirectional information flow to emulate the primate visual system. We show that modelling even first approximations of neural competition can more coher-ently explain imagery phenomenology than non-competitive models. Our simulations predict that, without competing sensory input, imagined stimuli should ubiquitously dominate hierarchical representations. However, with competition, imagination should dominate high-level representations but largely fail to outcompete sensory inputs at lower processing levels. To interpret our findings, we assume that low-level stimulus information (e. g. in early visual cortices) contributes most to the sensory aspects of perceptual experience, while high-level stimulus information (e. g. towards temporal regions) contributes most to its abstract aspects. Our findings therefore suggest that ongoing bottom-up inputs during waking life may prevent imagination from overriding veridical sensory experience. In contrast, internally generated stimuli may be hallucinated when sensory input is dampened or eradicated. Our approach can explain individual differences in imagery, along with aspects of daydreaming, hallucinations, and non-visual mental imagery. Keywords: mental imagery; hallucinations; aphantasia; hyperphantasia; daydreaming; perception © The Author(s) 2023. Published by Oxford University Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons. org/licenses/by/4. 0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. Introduction Mental images are internally generated thoughts which can be seen, heard, or in some way perceived with sensory qualities. Yet, how does the brain generate sensory experiences without a real sensory stimulus in the environment to perceive? One answer might be that mental imagery involves similar processes to those used during veridical perception, yet with an opposite direction of information flow. Instead of pooling together sensory informa-tion to extract abstract knowledge from a real stimulus, mental imagery may involve retrohierarchically reconstructing the sen-sory features of an imagined stimulus from abstract knowledge we already possess. In line with this idea, mental imagery seems to use the same neural machinery as that used during veridical perception ( Kosslyn et al. 1993; Ishai 2010 ; Cichy et al. 2012; Dijk-stra et al. 2017, 2018, 2019a; Xie et al. 2020 ), yet with neural activity during the early stages of veridical visual perception resembling neural activity during the later stages of visual mental imagery and vice versa ( Dentico et al. 2014 ; Linde-Domingo et al. 2019 ; Breedlove et al. 2020; Dijkstra et al. 2020). However, mental imagery is unlikely to be an exact reverse of feedforward processes in veridical perception given that indi-viduals reliably report that their mental images are experienced with some form of reduced sensory quality (termed 'vividness') compared to real images (Galton 1880; Marks 1973). Some researchers have proposed that this difference follows from dif-ferences between feedforward and feedback processes, the latter of which are believed to be responsible for propagating men-tal images. For instance, information about mental image con-tent tends to be most detectable in superficial and deep cortical layers, but not granular mid-layers, in contrast to information about environmental stimulus content, which can be detected across all cortical layers (Lawrence et al. 2018; Bergmann et al. 2019). Other accounts suggest that imagery could be propa-gated using weak feedback connections in the visual system, in Downloaded from https://academic. oup. com/nc/article/2023/1/niad018/7248966 by guest on 25 August 2024 