Interpretable Nonlinear Dynamic Modeling of Neural Trajectories
A central challenge in neuroscience is understanding how neural system implements computation through its dynamics. We propose a nonlinear time series model aimed at characterizing interpretable dynamics from neural trajectories. Our model assumes low-dimensional continuous dynamics in a finite volume. It incorporates a prior assumption about globally contractional dynamics to avoid overly enthusiastic extrapolation outside of the support of observed trajectories. We show that our model can recover qualitative features of the phase portrait such as attractors, slow points, and bifurcations, while also producing reliable longterm future predictions in a variety of dynamical models and in real neural data.
The authors demonstrate a fitting approach using basis functions and a regularization term to robustly fit nonlinear dynamical systems. The inclusion of the global regularization term is their main contribution, and they clearly illustrate how it makes their procedure robust against test cases outside of the training range (Table 1 & Fig 6 in supplementary material). They demonstrate their approach for various types of dynamics: a fixed-point attractor, FitzHugh-Nagumo, a ring attractor and Lorentz chaotic dynamics. They also very briefly discusses how the approach can be applied to neural recordings.
Overall I found the paper to be solid and rather enjoyable, and I would qualify it as a strong candidate for a poster. The authors' method of plotting velocity fields by decomposing the velocity into direction and speed, which they've apparently introduced, is especially effective. It made their arguments and conclusions much easier to follow, and will hopefully be picked up by others.  In my opinion stating that this approach leads to “interpretable models” might be somewhat overselling the results – the interpretability of the results is still hampered by the fact that models are composed by 10-100 more or less arbitrary basis functions. That being said, their capacity to reproduce salient features of the phase diagram certainly makes them more interpretable than, say, recurrent neural networks.  *** Update following authors' response *** "Interpretable model" can mean many things. For instance, it can mean that the value of each c_i, σ_i,… is interpretable – the meaning I originally had in mind – or that the model results in an interpretable phase diagram – the meaning the authors had in mind. It should suffice to define the desired meaning the first time this expression is used.  Discussions are for the most part appropriately detailed, although sometimes they feel somewhat condensed, perhaps due to the large number of examples the authors discuss within their allowed 8 pages. Specific issues are noted below.  *** Section 2 — Model - The linearization in eq. (2) seems incorrect to me. One typically linearizes an update function (F here) around some other point (say x*) in the neighborhood of x_t, so that we have something of the form x_{t+1} = x_t + A(x*)(x_t - x*) + B(x*)(x_t - x*)u_t + ε_t We can then collect the constant terms as b(x_t) = -A(x*)x* - B(x*)x* Note that it is critical that x* and x_t be different in general, otherwise the linear terms are always zero. This distinction seems to be made in the given ref. [9], but it is not done here. Thus eq. (2) needs to be corrected, or if it is correct, a more detailed derivation needs to be given. These issues are further illustrated by the comment that A is not unique: if A(x_t) is already defined as the Jacobian of F_x at x_t, how can it not be unique ? Fixing the derivation of (2) should help clarify this comment.  *** Update following authors' response *** I think the issue boils down to clarifying that b(x_t) and B(x_t) are not linearizations around x_t, but rather around some other point h(x_t). It would then be clear that they are not unique, as they depend on the function h. One should probably write b(x)=Fx(h(x)) on line 38.  - line 39: B should be a function from ℝ^d → ℝ^d x ℝ^{di}.  - The previous issue might also be addressed by simply removing the first part of section 2 up to “redundant and over parametrizes the dynamics.” Indeed, this whole section is never used for the rest of the paper. I found eq. (3) a clearer starting point for the arguments than eq. (2).  - The authors should state that W_g, W_B and τ all depend on the step size Δ.  - Eq. (4). I feel that the regularization term should be exp(-τ² x_t). If it is correct as is, the authors should comment on the benefit of writing the regularization constant in this way (rather than just using a constant β ≡ exp(-τ²) ).  - line 61: Unless B is constant, the Jacobian should also include a term ∂B/∂x.  - line 63: As far as I am aware, “ghost point” is not a frequent term in studies of dynamical systems. Perhaps the authors could provide a reference, or a short definition ? Same comment regarding “slow point”, which first appears on line 92.   *** Section 4.2 — Nonlinear oscillator model - Since the text refers to figures 2a-2d, the subfigures should show alphabetic labels.  *** Section 4.3 — Ring attractor dynamics To me this is the most interesting of the presented examples. The fact that only a subset of qualitative features are reproduced makes this a good test case to explore what this fitting procedure can and cannot do. Unfortunately the authors limit their discussion to a description, simply stating that the fit and true models show qualitatively different behavior. I think it would be worth trying to distill a what parts of the dynamics are difficult to fit and why. For instance, why is it that the radial dynamics seem to completely dominate Fig 3a, even along the ring (where dr/dt should be almost zero) ?  - Fig 3b : As far as I can tell, there are in fact two fit and two true trajectories in this figure (distinguished by dashed and full lines). The legend says there is one of each.  *** Section 4.4 — Chaotic system - Fig 4c : The true trajectories are almost impossible to see. Also the line styles in the legend should match those in the figure.  *** Section 5 — Learning V1 neural dynamics This section feels incomplete. The LDS model used is claimed to be the “best”, but no justification or reference is given to support this claim. In fact it is not even stated which LDS model is used – this could be included in the supplementary material, similar to what was done for section 4.1. The example studied in this section is by far the paper's most complex, and yet its explanation is one of the shortest. Consequently, it is difficult to picture what it is exactly that the model is fitting. Ideally a figure would be added to illustrate the inputs, but at the very least the second paragraph should be lengthened to include a more precise description of the inputs. As it stands, it is not even specified what time of recording the data stems from (only that it is “large scale”).  *** Supplementary material - As far as I understand, the interest in giving eq. (15) in terms of the x_i is to relate these dynamics to physical quantities. Since the authors never do this in the text, I would remove the middle expression in eq. (15). Similarly, I think defining x_2 in eq. (16) is superfluous; moreover, x_1 is never properly defined. The expressions in terms of s_1 and s_2 already appear as-is in ref. [20]. Also, the parameters J_{A,ext} and c' which appear in eq. (17) are never defined. - In fig 7, green and red dots should be above the blue lines rather than below.  *** Grammar  Some minor grammatical typos were noted, which in no way affect the quality of the paper. Possible corrections are listed here for the author's reference. 4: Our model incorporates a prior assumption 7: bifurcations 17: dynamical systems [5,6] 27-28: These features encourage the model 36: of an autoregressive 40: in the multivariate RBF 60: dynamics are locally 61: via the Hartman-Grobman 82-83: where the tasks typically involve 150: The initial state of each trajectory is picked from a standard normal 151: We use 19 trajectories 152: and the last one for testing. 166: The input was 3 dimensional 173: This end of sentence could be better formulated: “robustly predicts well.”