  We introduce deep Markov spatio-temporal factorization (DMSTF), a generative
model for dynamical analysis of spatio-temporal data. Like other factor
analysis methods, DMSTF approximates high dimensional data by a product between
time dependent weights and spatially dependent factors. These weights and
factors are in turn represented in terms of lower dimensional latents inferred
using stochastic variational inference. The innovation in DMSTF is that we
parameterize weights in terms of a deep Markovian prior extendable with a
discrete latent, which is able to characterize nonlinear multimodal temporal
dynamics, and perform multidimensional time series forecasting. DMSTF learns a
low dimensional spatial latent to generatively parameterize spatial factors or
their functional forms in order to accommodate high spatial dimensionality. We
parameterize the corresponding variational distribution using a bidirectional
recurrent network in the low-level latent representations. This results in a
flexible family of hierarchical deep generative factor analysis models that can
be extended to perform time series clustering or perform factor analysis in the
presence of a control signal. Our experiments, which include simulated and
real-world data, demonstrate that DMSTF outperforms related methodologies in
terms of predictive performance for unseen data, reveals meaningful clusters in
the data, and performs forecasting in a variety of domains with potentially
nonlinear temporal transitions.

Machine Learning;Machine Learning