Modelling perception as a hierarchical competition 5 Figure 2. Veridical stimuli outcompete imagined stimuli at low-level sensory representations, but not high-level abstract representations Note. How simulated neural representations of an internally generated (imagined) hat and an externally generated (veridical) camel spread through a perceptual processing hierarchy with and without competition. (a-c) Simulated neural activity patterns, coded as image pixel values, evolving over 0, 2, and 20 alternating feedforward-feedback sweeps. Initial activity patterns are the same in each scenario and mimic thinking about a hat while receiving a retinal image of a camel. Lock icons indicate clamped representations that do not evolve. The crossed arrow indicates disconnected sensory input (wl=0=0). (a) All layers except the base layer are free to evolve, causing activity patterns to be dominated by the initial veridical input. (b) Mid-layers are free to evolve only. Sensory input is present but disconnected. All evolving activity patterns become dominated by the initial imagined input. (c) Mid-layers are free to evolve only, and sensory input is present and connected. Imagined content dominates upper layers, and veridical content dominates lower layers. The dashed, magenta line indicates the 'representational equilibrium point' where the internally and externally generated stimuli equally contribute to a layer. (d-f) One-dimensional simulations of each scenario in (a)-(c), illustrating how the character of each layer evolves over time. The dominance of the original externally generated stimulus representation at l =0 can be obtained by subtracting the internally generated stimulus dominance from 100%. image of a camel as the externally generated veridical stimulus input (i. e. retinal activity pattern) and a compressed image of a hat as the internally generated imagined stimulus input (i. e. high-level neural activity pattern). Without competition from imagination, all freely evolving (unclamped) layers in the network converged asymptotically to a representation dominated by the externally generated stimu-lus representation (Fig. 2a). Conversely, without competition from veridical perception, all unclamped layers converged to a repre-sentation dominated by the internally generated stimulus repre-sentation (Fig. 2b). As all unclamped representations are eventu-ally dominated by the original externally or internally generated source stimulus, both scenarios each result in a state correspond-ing to having a visual experience of the same stimulus that is being simultaneously thought about abstractly. In the case of pure veridical perception, the system evolves to emulate a common perceptual experience: seeing an object while also knowing what it is. Yet, when the internally generated stimulus is uncontested, the sensory characteristics of the imagined stimulus should com-pletely dominate visual experience, resulting in the perception of what was imagined and not what was present in the environment. Hence, pure imagination imitates hallucinating the object that one is thinking about. Although, in both cases, layer l=1 is entirely dominated by one stimulus, each state occurred through a distinct formation route. Note that the top-down, but not bottom-up, formation route resulted in substantial distortions at lower layers, reflecting the findings of Breedlove et al. (2020). Yet, these distortions only affect the content of the imagined stimulus, not the degree to which that stimulus is able to commandeer a hierarchical layer in this model. Perception with competing imagined and veridical stimuli Where two fixed sources of information are simultaneously present within the network, neither the imagined nor veridi-cal stimulus can dominate every unclamped layer in the sys-tem (Fig. 2c). Instead, the network represents both stimuli simul-taneously at the steady state. However, the further a given layer is from an externally or internally generated stimulus input layer, the smaller the contribution of the original veridical or imagined stimulus to the representation held by the given layer. This is Downloaded from https://academic. oup. com/nc/article/2023/1/niad018/7248966 by guest on 25 August 2024 