Modelling perception as a hierarchical competition 7 Discussion Distinguishing imagined, veridical, and hallucinatory percepts In this study, we aimed to explain the apparent differences in vividness between veridical, imagined, and hallucinated percepts. We show that, for a hierarchical system with bidirectional infor-mation flow, the presence of competing sensory input can prevent internally generated stimuli from dominating low-level regions most suited for supporting a modal, highly sensory neural rep-resentation. We interpret this finding as suggesting that, when competing sensory input is present, the sensory content of imag-ination does not supersede the sensory content of veridical per-ception. That is, for instance, even if we imagine a hat, our mental image of the hat does not replace the image of a real camel in our visual field: the camel is still more visible. In con-trast, the attenuation or abolition of competing external sensory input could facilitate internally generated stimuli becoming the dominant contributor to sensory experience, creating hallucina-tory imagery. Our model therefore illustrates that the apparent difference in reported vividness between veridical and imagined percepts could be explained by the degree to which imagined and veridical stimulus representations spread hierarchically and that this difference can arise due to external sensory input competing with our internally generated thoughts. Neurological consequences of perceptual competition Our model results in some counterintuitive implications. Despite the suitability of the early visual cortex for representing retino-topic, visual information, our account suggests that hierarchically low-level neural regions (e. g. primary sensory cortices) may be less involved in representing an imagined stimulus than any other region in the corresponding perceptual hierarchy. However, this implication is consistent with existing findings. Individuals still report being able to create mental images despite near-complete bilateral lesions to primary visual cortex, suggesting that the pri-mary visual cortex is not necessary for visual mental imagery (Chatterjee and Southwood 1995; Zago et al. 2010 ; Bridge et al. 2012 ; de Gelder et al. 2015 ). Even mid-level impairments to the visual system, a potential cause of visual agnosia, can leave visual mental imagery preserved ( Behrmann et al. 1994). Furthermore, a meta-analysis of the neural correlates of visual mental imagery by Spagna et al. (2021) found that the left fusiform gyrus, a high-level region within the visual ventral stream, was reliably involved during mental imagery across studies while early visual cortices were not. These findings are entirely in line with the predictions of our model: while primary sensory cortices can be involved in men-tal imagery, the only essential component of the experience is the internal generation of a non-veridical stimulus representation, which, in our model, occurs at the hierarchical apex, correspond-ing, for instance, to late in the visual ventral stream. This may be why subjective vividness ratings better correlate with how dis-tinctly imagined information is represented in retinotopic, rather than associative, neural regions (Lee et al. 2012 ). Under our model, activation at levels below the clamped internally generated representation can add progressively more sensory components to the experience of imagining, although at what stage a rela-tively amodal, abstract thought formally becomes a truly sensory mental image may be an arbitrary distinction. Our simulations predict that the dominance of internally gen-erated stimuli should increase from posterior to anterior regions of the visual hierarchy. Supporting this prediction, Van Rullen and Reddy (2019) used a mental imagery decoding paradigm to show that more information about imagined face images was present in temporal regions than occipital regions. Lee et al. (2012) likewise showed that less information about imagined object images could be decoded towards more posterior, retinotopic areas of the visual system. Both studies also found that externally and internally generated stimulus representations became more similar towards higher-level regions, aligning with the notion that imagined con-tent is propagated from areas active during late-stage veridical perception. Phenomenological consequences of perceptual competition Our model has implications for the functional and subjective char-acteristics of imagery. For instance, the relegation of imagination to relatively high-level representations suggests that thinking in a non-sensory way may be more commonplace than manifest-ing thoughts in a sensory way (e. g. as an actual visual image). This makes pragmatic sense in everyday life: thinking about what you are seeing is often favourable during veridical visual per-ception, yet it may be quite unfavourable to see what you are thinking about given that such hallucinations may supersede important survival-relevant environmental information. However, this process could involve a functional trade-off as the seman-tic content of sensory input may be processed poorly if high-level regions are already occupied by unrelated thoughts. Mental imagery could push internally generated high-level representa-tions towards mid-level regions, interfering with the ascending processing of externally generated stimuli. This could explain why internal attention can affect the processing of environmental stimuli, such as during daydreaming (Schooler et al. 2011; Small-wood 2011). States of immersive mental imagery (e. g. daydream-ing) might then loosely and transiently share phenomenological similarities with agnosia given that high-level processing of exter-nally generated stimuli may be impeded despite low-level visual feature processing being relatively unaffected. For instance, when reading text while distracted by other thoughts, one may find themselves rereading the same paragraph repeatedly, seeing the words each time, yet failing to process the meaning of the text. Another effect of abstract representations being activated without any significant low-level representation during imagery is that a perceiver might plausibly even 'feel' like they have seen their imagined thought without ever experiencing it in any sub-stantially visual way. That is, units involved in the recognition of an object may be activated despite minimal low-level activation. Hence, people could theoretically report being able to visualize objects, yet fail to be able to reconstruct the details of these imagined stimuli accurately. Additionally, experiencing imagery on a spectrum from highly abstract to highly sensory may explain why the unique quasi-sensory experience of mental imagery may be difficult to describe. Common language generally only allows perceptual experience to be described as explicitly sensory (e. g. seen) or explicitly non-sensory (e. g. unseen), forcing ambiguously defined terms like 'vividness' to fill in the gaps. While the content of mental imagery could be described in terms of objectively quantifiable stimu-lus properties (e. g. contrast, size, and duration), the experience of such content is modified by the degree to which a mental image gains sensory features as well as just abstract features. This modulation, encapsulated by the aforementioned concept of a representational equilibrium point, could be the underlying variable commonly measured (albeit somewhat indistinctly) by Downloaded from https://academic. oup. com/nc/article/2023/1/niad018/7248966 by guest on 25 August 2024 