DYREP: LEARNING REPRESENTATIONS OVER DYNAMIC GRAPHS
Representation Learning over graph structured data has received significant attention recently due to its ubiquitous applicability. However, most advancements have been made in static graph settings while efforts for jointly learning dynamic of the graph and dynamic on the graph are still in an infant stage. Two fundamental questions arise in learning over dynamic graphs: (i) How to elegantly model dynamical processes over graphs? (ii) How to leverage such a model to effectively encode evolving graph information into low-dimensional representations? We present DyRep -a novel modeling framework for dynamic graphs that posits representation learning as a latent mediation process bridging two observed processes namely -dynamics of the network (realized as topological evolution) and dynamics on the network (realized as activities between nodes). Concretely, we propose a two-time scale deep temporal point process model that captures the interleaved dynamics of the observed processes. This model is further parameterized by a temporal-attentive representation network that encodes temporally evolving structural information into node representations which in turn drives the nonlinear evolution of the observed graph dynamics. Our unified framework is trained using an efficient unsupervised procedure and has capability to generalize over unseen nodes. We demonstrate that DyRep outperforms state-of-the-art baselines for dynamic link prediction and time prediction tasks and present extensive qualitative insights into our framework.
Overall, the contribution of the paper is somewhat limited [but a little more than my initial assessment, thanks to the rebuttal]. It is essentially an extension of (Trivedi et al. 2017), adding attention to provide self-exciting rates, applied to two types of edges (communication edges and “friendship” edges). Conditioned on past edges, future edges are assumed independent, which makes the math trivial. The work would be better described as modeling a Marked Point Process with marks k \in {0,1}.
Other comments:
1.	[addressed] DyRep-No-SP is as good as the proposed approach, maybe because the graph is assumed undirected and the embedding of u can be described by its neighbors (author rebuttal describes as Localized Propagation), as the neighbors themselves use the embedding of u for their own embedding (which means that self-propagation is never "really off"). Highly active nodes have a disproportional effect in the embedding, resulting in the better separated embeddings of Figure 4. [after rebuttal: what is the effect of node activity on the embeddings?]
2.	[unresolved, comment still misundertood] The Exogenous Drive W_t(t_p – t_{p−1}) should be more personalized. Some nodes are intrinsically more active than others. [after rebuttal: answer "$W_t(t_p - t_{p-1})$ is personalized as $t_p$ is node specific", I meant personalized as in Exogenous Drive of people like Alice or Bob]
3.	[unresolved] Fig 4 embeddings should be compared against (Trivedi et al. 2017) [after rebuttal: author revision does not make qualitative comparison against Trivedi et al. (2017)]
Besides the limited innovation, the writing needs work. 
4.	[resolved] Equation 1 defines $g_k(\bar{t})$ but does not define \bar{t}. Knowing (Trivedi et al. 2017), I immediately knew what it was, but this is not standard notation and should be defined. 
5.	[resolved] $g_k$ must be a function of u and v
6.	[resolved] “$k$ represent the dynamic process” = >  “$k$ represent the type of edge” . The way it is written $k$ would need to be a stochastic process (it is just a mark, k \in {0,1})
7.	[resolved] Algorithm 1 is impossibly confusing. I read it 8 times and I still cannot tell what it is supposed to do. It contains recursive definitions like $z_i = b + \lambda_k^{ji}(t)$, where $\lambda_k^{ji}(t)$ itself is a function of $z_i(t)$. Maybe the z_i(t) and z_i are different variables with the same name?
8.	[resolved] The only hint that the graph under consideration is undirected comes from Algorithm 1, A_{uv}(t) = A_{vu}(t) = 1. It is *very* important information for the reader.
Related work (to be added to literature):
Dynamic graph embedding: (Yuan et al., 2017) (Ghassen et al., 2017)
Dynamic sub-graph embedding: (Meng et al., 2018)
Minor:
state-of-arts => state-of-the-art methods
list enumeration “1.)” , “2.)” is strange. Decide either 1) , 2) or 1. , 2. . I have never seen both.
MAE => mean absolute error (MAE)
Yuan, Y., Liang, X., Wang, X., Yeung, D. Y., & Gupta, A., Temporal Dynamic Graph LSTM for Action-Driven Video Object Detection. ICCV, 2017.
Jerfel,  , Mehmet E. Basbug, and Barbara E. Engelhardt. "Dynamic Collaborative Filtering with Compound Poisson Factorization." AISTATS 2017. 
Meng, C., Mouli, S.C., Ribeiro, B. and Neville, J., Subgraph Pattern Neural Networks for High-Order Graph Evolution Prediction. AAAI 2018.
--- --- After rebuttal 
Authors addressed most of my concerns. The paper has merit and would be of interest to the community. I am increasing my score.