Adaptive Path-Integral Autoencoder: Representation Learning and Planning for Dynamical Systems
We present a representation learning algorithm that learns a low-dimensional latent dynamical system from high-dimensional sequential raw data, e.g., video. The framework builds upon recent advances in amortized inference methods that use both an inference network and a refinement procedure to output samples from a variational distribution given an observation sequence, and takes advantage of the duality between control and inference to approximately solve the intractable inference problem using the path integral control approach. The learned dynamical model can be used to predict and plan the future states; we also present the efficient planning method that exploits the learned low-dimensional latent dynamics. Numerical experiments show that the proposed path-integral control based variational inference method leads to tighter lower bounds in statistical model learning of sequential data. The supplementary video 1 and the implementation code 2 are available online.
Paper : 5353 This paper discusses representation learning problem, and designs an algorithm to learn variational parameters of approximate inference. The paper has adopted a semi-amortized technique for learning variational parameters of dynamic system for temporal data. In this paper, dynamic system is generated by state space model. In order to express the state space model, just initial state and control input at every time steps needs to be inferred. Paper has utilized adaptive path-integral technique for variational parameter refinement of dynamic system to mitigate the amortization gap( induced by limitation of inference network parameters compared to optimal variational distribution ).  Expressing the problem(finding variational distribution parameters ) in the format that is addressed by stochastic optimal control and adaptation of adaptive  path-integral  technique to dynamic system can be considered main contributions of this paper . Experiments has been done on inverted pendulum and  3D human motion data, results show some small  relative advantage for current method than baselines. For one of datasets (inverted pendulum), the model has shown good predictive capability although not compared with other baseline for quantitative prediction power.   Quality: claims and supporting material both in main paper and supplementary have been verified and raised a few questions that.   One issue found is in equation 10 appeared on page 4, as we know ELBO = E_{q_u(Z)}[log p(X|Z)] -KL(q_u(Z|X)||p(Z)), in supplementary materials shown that KL(q_u(Z|X)||p(Z)) = E_{q_u}[\integ{V(Z)+1/2 ||u(t)||^2}] +log \zetta where log\zetta does not contain the variational parameters, then ELBO will turn out to be  ELBO = E_{q_u(Z)}[log p(X|Z) +\integ{-V(Z)-1/2 ||u(t)||^2}] , although what we have in  10 does not match with what I found in ELBO above, can you please clarify? Also I am not sure if V(z) in line 129 is same as V(z) in line 370 or just used same notation for 2 different expression.  In experimental results of this paper,in inverted pendulum example, the prediction has been good and shows that parameters of variational distribution describes the dynamic system well. It will be informative if prediction for 3D human motion data is presented as well.  Clarity: Although paper has been motivated well and general concepts and background have been explained well, there are some points which need clarification.  On lines 137 and 138 has  mentioned that feed forward/feedback structure can be implemented using RNN, but RNN has not  been used in this paper for inferring parameters, please be more specific on that. Having schematic of network and specifying  inference vs generative network and their parameters help the reader to understand your proposed solutions better.  In experiment section baseline algorithms specification and setups are not mentioned    Originality: As mentioned in summary the main contribution of this paper could be summarized as bellow  - Expressing the problem(finding variational distribution parameters for modeling sequential data ) in the format that is addressed by stochastic optimal control  (SOC)  -  adaptation of adaptive path-integral technique for variational parameter refinement of dynamic system to mitigate the amortization gap  authors have cited many of recent works, and seems have done an in depth literature review also could distinguish their contribution to what has been done so far clearly.    Significance:  As discussed, in summary section, the results show some small relative improvement with baseline algorithms. But as suggested, it will be very useful to have a quantitative prediction comparison  between the proposed algorithm and baselines.