Deep Dynamic Poisson Factorization Model
A new model, named as deep dynamic poisson factorization model, is proposed in this paper for analyzing sequential count vectors. The model based on the Poisson Factor Analysis method captures dependence among time steps by neural networks, representing the implicit distributions. Local complicated relationship is obtained from local implicit distribution, and deep latent structure is exploited to get the long-time dependence. Variational inference on latent variables and gradient descent based on the loss functions derived from variational distribution is performed in our inference. Synthetic datasets and real-world datasets are applied to the proposed model and our results show good predicting and fitting performance with interpretable latent structure.
This papers introduces the deep dynamic Poisson factorization model, a model that builds on PF to allow for temporal dependencies. In contrast to previous works on dynamic PF, this paper uses a simplified version of a recurrent neural network to allow for long-term dependencies. Inference is carried out via variational inference, with an extra step to maximize over the neural network parameters. The paper reports experiments on 5 real-world datasets.
Overall, I found the idea potentially interesting, but I still think the paper needs some improvements in its execution before it can be accepted in a conference like NIPS. Please find my comments below.
+The formulation in Eq. 4 doesn't correspond to a RNN structure, because h_t^{(n)} depends only on the top-layer memory vectors, and it does not depend on previous time steps for the same layer, as in the RNN setup. Is there any reason for this simplified structure?
+In Section 2, some of the model equations are missing (e.g., it is not clear how to parameterize the Gamma in Eq. 4, or what's the distribution that generates theta_t in Eq. 3).
+I disagree with the discussion in lines 110-118 regarding "implicit distributions". Eqs. 9 and 10 are actually explicit: they are Gammas with given shapes and rates. Since the distribution of theta_tk (Eq. 9) can be computed in closed form conditioned on h^{(0)}, it cannot be called implicit. The same comment applies to h in Eq. 10.
+Lines 30-31: The sentence "This method may have weak points in analyzing other data with different pattern long-time dependence, such as fanatical data and disaster data" wasn't clear to me; can you elaborate?
+The subsection in lines 133-155 isn't clear to me. What is exactly the "loss function" in Eq. 15?
+MSE/PMSE is typically not a good metric for PF-like models. The results should additionally include at least predictive log-likelihood.
+The value of K in the experiments (line 207) seems too small to me. I suggest the authors use larger values, such as K=100 (at least).
+I couldn't connect the discussion in lines 224-232 with Fig. 3: it is not clear.
+The update equations for the variational inference procedure (Eqs. 11-14) can safely be moved to the supplement.
+The caption in Fig. 4 cannot be read clearly.
+I *strongly* recommend to follow general writing advice about the use of the passive voice (specially for the abstract).
+The writing quality is poor, and the paper contains many typos. Here are some examples: 
  -Line 15, "high-dimensional"
  -Line 22, "factorize"
  -Line 24, "property of the"
  -Lines 29, 34, "Dirichlet"
  -Line 34, "nested"
  -Fig. 1, "visual representation"
  -Fig.1, "transmission" (not sure if that's an appropriate word anyways)
  -Incomplete sentences (e.g., line 49)
  -Many missing articles (e.g., "Although Dirichlet distribution is often used as prior distribution")