Variational inference of latent hierarchical dynamical systems in neuroscience: an application to calcium imaging data
A key problem in neuroscience, and life sciences more generally, is that data is generated by a hierarchy of dynamical systems. One example of this is in 	extit{in-vivo} calcium imaging data, where data is generated by a lower-order dynamical system governing calcium flux in neurons, which itself is driven by a higher-order dynamical system of neural computation. Ideally, life scientists would be able to infer the dynamics of both the lower-order systems and the higher-order systems, but this is difficult in high-dimensional regimes. A recent approach using sequential variational auto-encoders demonstrated it was possible to learn the latent dynamics of a single dynamical system for computations during reaching behaviour in the brain, using spiking data modelled as a Poisson process. Here we extend this approach using a ladder method to infer a hierarchy of dynamical systems, allowing us to capture calcium dynamics as well as neural computation. In this approach, spiking events drive lower-order calcium dynamics, and are themselves controlled by a higher-order latent dynamical system. We generate synthetic data by generating firing rates, sampling spike trains, and converting spike trains to fluorescence transients, from two dynamical systems that have been used as key benchmarks in recent literature: a Lorenz attractor, and a chaotic recurrent neural network. We show that our model is better able to reconstruct Lorenz dynamics from fluorescence data than competing methods. However, though our model can reconstruct underlying spike rates and calcium transients from the chaotic neural network well, it does not perform as well at reconstructing firing rates as basic techniques for inferring spikes from calcium data. These results demonstrate that VLAEs are a promising approach for modelling hierarchical dynamical systems data in the life sciences, but that inferring the dynamics of lower-order systems can potentially be better achieved with simpler methods.
The paper contributes to the space of hierarchical latent modeling with applications to neuroscience as follows: 1) A model combining variational ladder auto-encoders with RNNs is presented; 2) An inference method based on ELBO maximization is presented; 3) Results on synthetic different data sets show good performance although firing rates are reconstructed better by another approach on a key task (firing rate reconstruction). The paper is fairly well written but it seems that the last paragraph and conclusion have been written with more haste (also a period is missing at the beginning of the last sentence of the first paragraph in the conclusion). I am a bit skeptical about whether to accept the paper. Although hierarchical latent modeling is interesting and producing models and training approaches that are modular and composable is valuable, it is not clear here that the model under consideration really provides strong improvements on the state of the art. Furthermore, it appears that the set of benchmarks being considered is narrowly focused on a subfield of neuroscience without providing clear wins on any task involving actual data. As the theoretical contribution of the paper is rather weak and the empirical results are not very strong, I recommend a weak reject. I believe that a stronger experimental section could help the authors increase the impact of their paper
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Variational inference of latent hierarchical dynamical systems in neuroscience: an application to calcium imaging data
A key problem in neuroscience, and life sciences more generally, is that data is generated by a hierarchy of dynamical systems. One example of this is in 	extit{in-vivo} calcium imaging data, where data is generated by a lower-order dynamical system governing calcium flux in neurons, which itself is driven by a higher-order dynamical system of neural computation. Ideally, life scientists would be able to infer the dynamics of both the lower-order systems and the higher-order systems, but this is difficult in high-dimensional regimes. A recent approach using sequential variational auto-encoders demonstrated it was possible to learn the latent dynamics of a single dynamical system for computations during reaching behaviour in the brain, using spiking data modelled as a Poisson process. Here we extend this approach using a ladder method to infer a hierarchy of dynamical systems, allowing us to capture calcium dynamics as well as neural computation. In this approach, spiking events drive lower-order calcium dynamics, and are themselves controlled by a higher-order latent dynamical system. We generate synthetic data by generating firing rates, sampling spike trains, and converting spike trains to fluorescence transients, from two dynamical systems that have been used as key benchmarks in recent literature: a Lorenz attractor, and a chaotic recurrent neural network. We show that our model is better able to reconstruct Lorenz dynamics from fluorescence data than competing methods. However, though our model can reconstruct underlying spike rates and calcium transients from the chaotic neural network well, it does not perform as well at reconstructing firing rates as basic techniques for inferring spikes from calcium data. These results demonstrate that VLAEs are a promising approach for modelling hierarchical dynamical systems data in the life sciences, but that inferring the dynamics of lower-order systems can potentially be better achieved with simpler methods.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The paper contributes to the space of hierarchical latent modeling with applications to neuroscience as follows: 1) A model combining variational ladder auto-encoders with RNNs is presented; 2) An inference method based on ELBO maximization is presented; 3) Results on synthetic different data sets show good performance although firing rates are reconstructed better by another approach on a key task (firing rate reconstruction). The paper is fairly well written but it seems that the last paragraph and conclusion have been written with more haste (also a period is missing at the beginning of the last sentence of the first paragraph in the conclusion). I am a bit skeptical about whether to accept the paper. Although hierarchical latent modeling is interesting and producing models and training approaches that are modular and composable is valuable, it is not clear here that the model under consideration really provides strong improvements on the state of the art. Furthermore, it appears that the set of benchmarks being considered is narrowly focused on a subfield of neuroscience without providing clear wins on any task involving actual data. As the theoretical contribution of the paper is rather weak and the empirical results are not very strong, I recommend a weak reject. I believe that a stronger experimental section could help the authors increase the impact of their paper
assistant
The paper contributes to the space of hierarchical latent modeling with applications to neuroscience as follows: 1) A model combining variational ladder auto-encoders with RNNs is presented; 2) An inference method based on ELBO maximization is presented; 3) Results on synthetic different data sets show good performance although firing rates are reconstructed better by another approach on a key task (firing rate reconstruction). The paper is fairly well written but it seems that the last paragraph and conclusion have been written with more haste (also a period is missing at the beginning of the last sentence of the first paragraph in the conclusion). I am a bit skeptical about whether to accept the paper. Although hierarchical latent modeling is interesting and producing models and training approaches that are modular and composable is valuable, it is not clear here that the model under consideration really provides strong improvements on the state of the art. Furthermore, it appears that the set of benchmarks being considered is narrowly focused on a subfield of neuroscience without providing clear wins on any task involving actual data. As the theoretical contribution of the paper is rather weak and the empirical results are not very strong, I recommend a weak reject. I believe that a stronger experimental section could help the authors increase the impact of their paper