Attractor Network Dynamics Enable Preplay and Rapid Path Planning in Maze-like Environments
Rodents navigating in a well-known environment can rapidly learn and revisit observed reward locations, often after a single trial. While the mechanism for rapid path planning is unknown, the CA3 region in the hippocampus plays an important role, and emerging evidence suggests that place cell activity during hippocampal "preplay" periods may trace out future goal-directed trajectories. Here, we show how a particular mapping of space allows for the immediate generation of trajectories between arbitrary start and goal locations in an environment, based only on the mapped representation of the goal. We show that this representation can be implemented in a neural attractor network model, resulting in bump-like activity profiles resembling those of the CA3 region of hippocampus. Neurons tend to locally excite neurons with similar place field centers, while inhibiting other neurons with distant place field centers, such that stable bumps of activity can form at arbitrary locations in the environment. The network is initialized to represent a point in the environment, then weakly stimulated with an input corresponding to an arbitrary goal location. We show that the resulting activity can be interpreted as a gradient ascent on the value function induced by a reward at the goal location. Indeed, in networks with large place fields, we show that the network properties cause the bump to move smoothly from its initial location to the goal, around obstacles or walls. Our results illustrate that an attractor network with hippocampal-like attributes may be important for rapid path planning.
The proposed method to model preplay and rapid path planning using successor representation is a very interesting and potentially important development. It is also in my opinion a more realistic implementation of model-based RL than tree search-based alternatives.However, a few important questions remain: - is computation of advanced quantities as described in sections 2 and 3 biologically realistic - not just from the perspective of the resulting output and dynamics, but the mechanistic implementation (e.g. how weights that are derived here using least-squares learned)? - how does such successor-based representation work in more realistic scenarios when familiarity with the environment is partial and how does it change with learning - can it reproduce typically observed behavioral learning curves? - can it also be applied to other domains than space, e.g. time cells? I'm aware that addressing these questions in detail may be out of the scope of this paper, but it would be helpful if these aspects could at least be discussed briefly.Clarity of the paper, particularly of the mathematical expressions and derivations, could be improved, e.g. line 2 of eq.3 or why a weak stimulus corresponds to alpha = epsilon = 0.05 explained in more detail.