6 Sulfaro et al. Figure 3. Decreasing feedforward-to-feedback weighting ratio increases the sensory character of thoughts Note. Hierarchical competition between simulated neural representations of an internally generated (imagined) hat and an externally generated (veridical) camel modelled under different feedforward-to-feedback weighting ratios. Simulated neural activity patterns, coded as image pixel values, are depicted after 20 alternating feedforward-feedback sweeps. Initial activity patterns are the same in each scenario and mimic thinking about a hat while receiving a retinal image of a camel. Lock icons indicate clamped representations that do not evolve. The crossed arrow indicates disconnected sensory input (wl=0=0). Each dashed, magenta line indicates the 'representational equilibrium point' where the internally and externally generated stimuli contribute equally to a layer. (a-c) Globally varying the ratio of bottom-up (wl-1) and top-down (wl+1) weightings affects representational equilibrium and the extent to which thoughts should be experienced visually (e. g. weak, moderate, or high). (d) All unclamped layers converge towards the original internally generated stimulus representation once sensory input is disconnected, even if feedforward and feedback weights are equal. shown in Fig. 2f for the case where feedforward and feedback weightings are equal (wl-1=wl+1=1). Consequently, when imagination occurs in the presence of bottom-up sensory input, the externally generated veridical stim-ulus dominates low-level representations, while the internally generated imagined stimulus dominates high-level representa-tions. Such a system is therefore representing two distinct stimuli, yet at largely separate hierarchical levels. As lower levels are more suited for representing sensory information and higher lev-els for abstract, this state corresponds to a situation where an individual has a visual experience of their veridical environment despite simultaneously having relatively non-sensory thoughts about what they are imagining. Hence, the simulation suggests that external sensory input outcompetes imagination at the low-level regions that most contribute to the sensory aspects of perceptual experience. Modelling variation in imagery quality (e. g. aphantasia and hyperphantasia) as a shift in representational equilibrium This model can also be used to understand states where thoughts are experienced in a highly sensory way (hyperphantasia) or in a non-sensory or weakly sensory way (aphantasia; Fig. 3). When sen-sory input and imagination compete, our simulation showed that spanning along the perceptual hierarchy from bottom to top cor-responded to transitioning from primarily representing a veridical stimulus to primarily representing an imagined stimulus. The exact point where the two stimuli are represented in equal propor-tion is here dubbed the 'representational equilibrium point' for the system. Representations contain more veridical stimulus charac-ter below this point and more imagined stimulus character above it. The location of the representational equilibrium point can summarize how far an imagined representation can penetrate towards lower layers before it is outcompeted by the veridical stimulus, or how far a veridical stimulus representation can penetrate towards upper layers before it is outcompeted by the imagined stimulus. Where ascending and descending informa-tion is weighted equally ( wl-1=wl+1=1; Fig. 3b), the equilibrium point lies in the middle layer at the centre of the hierarchy such that lower layers (e. g. towards the primary visual cortex) are dominated by externally generated sensory input and upper layers (e. g. towards the temporal lobe) are dominated by inter-nally generated thoughts, symmetrically. However, overweight-ing feedforward information ( wl-1/wl+1> 1) elevates the equilib-rium point such that the imagined stimulus is restricted to only the highest, least-modal hierarchical layers, correspond-ing to a state of aphantasia (Fig. 3a). In contrast, overweighting feedback information (wl-1/wl+1< 1) lowers the equilibrium point such that the imagined stimulus dominates the majority of the hierarchy, corresponding to a state of hyperphantasia (Fig. 3c). Feedforward-to-feedback weighting ratio variation could there-fore account for between-person and moment-to-moment dif-ferences in the apparent sensory quality of mental imagery experiences. Downloaded from https://academic. oup. com/nc/article/2023/1/niad018/7248966 by guest on 25 August 2024 