Latent variable models have been widely applied for the analysis and visualization of large datasets. In the case of sequential data, closed-form inference is possible when the transition and observation functions are linear. However, approximate inference techniques are usually necessary when dealing with nonlinear evolution and observations. Here, we propose a novel variational inference framework for the explicit modeling of time series, Variational Inference for Nonlinear Dynamics (VIND), that is able to uncover nonlinear observation and latent dynamics from sequential data. The framework includes a structured approximate posterior, and an algorithm that relies on the fixed-point iteration method to find the best estimate for latent trajectories. We apply the method to several datasets and show that it is able to accurately infer the underlying dynamics of these systems, in some cases substantially outperforming state-of-the-art methods.
This paper discusses a algorithm for variational inference of a non-linear dynamical models. In this paper model assumption is to use single stage Markov model in latent space with every latent variable Z_t to be defined Gaussian distributed with mean depends on Z_(t-1) and time invariant variance matrix lambda. The non linearity in transition is encoded in mean of Guassian distribution. For modeling the likelihood and observation model, the Poisson or Normal distribution are used with X_t being sampled from another Gaussian or Poisson distribution with the non-linearty being encoded in the parameters of distribution with variable Z_t.  This way of modeling resembles so of many linear dynamical model with the difference of transition and observation distribution have nonlinearity term encoded in them. 
The contribution of this paper can be summarized over following points:
- The authors proposed the nonlinear transition and observation model and introduced a tractable inference model using Laplace approximation in which for every given set of model parameter solves for parameters of Laplace approximation of posteriori and then model parameters get updated until converges
-the second point is to show how this model is successful to capture the non-linearity of the data while other linear models do not have that capabilities 
Novelty and Quality: 
The main contribution of this paper is summarized above. The paper do not contain any significant theorem or mathematical claims, except derivation steps for finding Laplace approximation of the posteriori. The main challenge here is to address effectiveness of this model in comparison to other non-linear dynamical system that we can name papers as early as Ghahramani, Zoubin, and Sam T. Roweis. "Learning nonlinear dynamical systems using an EM algorithm." Advances in neural information processing systems. 1999. 
or more recent RNN paper LSTM based papers. I think authors need to distinguish what this paper can give to community beside approximate posteriori of latent variables that other competing models are not capable of. If the aim is to have that posteriori, the authors should show what type of interpretation they have drawn from that in experiments.
There are lots of literature exist on speech, language models and visual prediction which can be used as reference as well.
Clarity: 
The paper is well written and some previous relevant methods have been reviewed . There are a few issues that are listed below: 
1- as mentioned in Quality sections authors should be more clear about what is distinguished in this paper that other non-linear dynamical systems 
2- they used short form RM for Recognition model or FPI for fixed point iteration that need need to be defined before being used
significance and experiments: 
The experiments are extensive and authors have compared their algorithm with some other linear dynamical systems (LDS) competing algorithms and showed improvement in many of the cases for trajectory reconstruction. 
A few points can be addressed better, it can be seen for many of experiments exhaustive search is used for finding dimension of latent variable. This issue is addressed in Kalantari, Rahi, Joydeep Ghosh, and Mingyuan Zhou. "Nonparametric Bayesian sparse graph linear dynamical systems." arXiv preprint arXiv:1802.07434 (2018). That paper can use non-parametric approaches to find best latent dimension, although the paper applied the technique on linear system, same technique could be adopted to non-linear models. Also that model is capable of finding multiple linear system that model the non linearity by switching between diffrent linear system, for switching linear system, this paper can be named as well: Linderman, Scott, et al. "Bayesian learning and inference in recurrent switching linear dynamical systems." Artificial Intelligence and Statistics. 2017.
It is shown that the model can reconstruct the spikes very well while linear model do not have that power (which is expected), but it is interesting to see how other non-linear models would compare to this model under those certain conditions
It is desired and interesting to see how the model behave one step ahead and K-step ahead prediction. Please address why it cannot be done if there is difficulties in that.