Supervised Learning for Dynamical System Learning
Recently there has been substantial interest in spectral methods for learning dynamical systems. These methods are popular since they often offer a good tradeoff between computational and statistical efficiency. Unfortunately, they can be difficult to use and extend in practice: e.g., they can make it difficult to incorporate prior information such as sparsity or structure. To address this problem, we present a new view of dynamical system learning: we show how to learn dynamical systems by solving a sequence of ordinary supervised learning problems, thereby allowing users to incorporate prior knowledge via standard techniques such as L 1 regularization. Many existing spectral methods are special cases of this new framework, using linear regression as the supervised learner. We demonstrate the effectiveness of our framework by showing examples where nonlinear regression or lasso let us learn better state representations than plain linear regression does; the correctness of these instances follows directly from our general analysis.
Summary -------The authors present a new algorithm for learning dynamical systems, which generalizes some existing spectral learning algorithms. The approach is based on keeping track of the latent state via a observable representation, which can be updated using a linear mapping for the model classes consider in the manuscript (eg HMMs, linear dynamical systems). To arrive at an unbiased estimate of this mapping from data the authors apply instrumental variable regression.  Comments --------The article contains interesting contributions to the literature on learning dynamical systems. Basic versions of the idea of formulating dynamical system learning as supervised learning problems have be, but the paper gives useful generalizations of the approach and proves error bounds. The main drawbacks are the relative simple numerical experiments and that the manuscript feels a little hastily written. The latter might have contributed to me understanding the approach less well than I would like to. - A main advantage of the proposed approach, that the authors point out, is that the formulation of the learning procedure as regression problems allows to tap into the regression literature for improving and generalizing the algorithm. However, I don't understand why this should mostly apply to the stage 1 regression. E.g. the experiment presented in 5.2: I'm guessing in this case the $W$ is closely related to the transition matrix $T$. Wouldn't it make sense to try to learn a structured $W$ that somehow captures the block-structure of the prior, instead of using a L1 regression in the S1 pre-processing step. Also, in a non-linearity dynamical system (say with linear Gaussian observations), would we want to use a non-linear regression for S2? What am I missing here? This should be clarified in the manuscript.- Sec 5.1: This a very simple model to learn. It would be much more convincing if the authors also presented a slightly more high-dimensional case. Also, how does naive EM-HMM learning perform here; this comparison would be quite instructive.- Sec 5.1: The authors should explicitly spell out why their algorithm gives better performance in this case, and why model 3 performs better than model 2.  - Sec 5.1: Why is logistic regression a sensible choice here? This is an important demonstration for the method, so the manuscript would benefit from a more detailed explanation/justification. If the authors need more space, I suggestion possible deleting figure 1.- The authors should cite some of the traditional system identification literature, where instrumental variables have been used for a long time, eg [Ljung. System identification, 1999] and references therein and explain where the similarities and differences lie.- Is the regression stage S1B really necessary? If so, why? - minor points: -- l25: missing "of"-- l79: missing ")" -- l94: It would be useful if you could briefly remind the reader of the definition of k-observability and how this affects the choice of future feature $\psi_t$ -- l99: maybe replace "is determined" by "can be computed from" -- l121: Should it be conditioned on $o_{t-k:t+k-1}$ -- l190: typo, should prob be $P(o_{t:t+k-1}\vert o_{1:t-1})$ -- table one, top left box: there's some confusion with the indices here. For a stationary HMM, $P(o_2,o_1)$ and$P(o_t,o_t)$ are the same, but a consistent notation should be used.