Inductive representation learning on temporal graphs is an important step toward salable machine learning on real-world dynamic networks. The evolving nature of temporal dynamic graphs requires handling new nodes while learning temporal patterns. The node embeddings, which become functions of time under the temporal setting, should capture both static node features and evolving topological structures. Moreover, node and topological features may exhibit temporal patterns that are informative for prediction, of which the temporal node embeddings should also be aware. We propose the temporal graph attention (TGAT) layer to effectively aggregate temporal-topological neighborhood features as well as learning time-feature interactions. For TGAT, we use the self-attention mechanism as the building block and develop the novel functional time encoding technique based on the classical Bochner's theorem from harmonic alaysis. By stacking TGAT layers, the network learns node embeddings as functions of time and can inductively infer embeddings for both new and observed nodes whenever the graph evolves. The proposed approach handles both node classification and link prediction task, and can be naturally extended to aggregate edge features. We evaluate our method with transductive and inductive tasks under temporal setting with two benchmark and one industrial dataset. Our TGAT model compares favorably to state-of-theart baselines and prior temporal graph embedding approaches.
Summary: This paper addresses the problem of representation learning for temporal graphs. That is, graphs where the topology can evolve over time. The contribution is a temporal graph attention (TGAT) layer aims to exploit learned temporal dynamics of graph evolution in tasks such as node classification and link prediction. This TGAT layer can work in an inductive manner unlike much prior work which is restricted to the transduction setting. Specifically, a temporal-kernel is introduced to generate time-related features, and incorporated into the self-attention mechanism. The results on some standard and new graph-structured benchmarks show improved performance vs a variety of baselines in both transduction and inductive settings. 
Pros: 
+ Dynamic graphs are an important but challenging data structure for many problems. Improved methods in this area are welcome. 
+ Dealing with the inductive setting is an important advantage. 
+ Clear performance improvements on prior state of the art is visible in both transductive+inductive settings and node+edge related tasks.
Cons+Questions:
1. Technical significance: Some theory is presented to underpin the approach, but in practice it seems to involve concatenating or adding temporal kernels element-wise to the features already used by GAT. In terms of implementation the concatenation in Eq 6 seems to be the only major change to GAT. I’m not sure if this is a major advance.  
2. Insight. The presented method apparently improves on prior work by learning something about temporal evolution and exploiting it in graph-prediction tasks. But it's currently rather black-box. It would be better if some insight could be extracted about *what* this actually learns. What kind of temporal trends exist in the data that this method has learned? And how are they exploited in by the prediction tasks?
3. Writing. The English is rather flaky throughout. One particular recurring frustration is the use of the term “architect” which seems wrong. Probably “architecture” is the correct alternative. 
4. Clarity of explanation. The paper is rather hard to follow and ambiguous. A few specific things that are not explained so well: 
4.1. Eq 1+2 is not a sufficiently clear and self-contained recap of prior work. 
4.2. Symbol d_T used at the start of Sec 3.1 seems to be used without prior definition making it hard to connect to previous Eq1+2. 
4.3 The claim made about alternative approaches (Pg4) “Reparameterization is only applicable to local-scale distribution family, which is not rich enough”. Seems both too vague and unjustified. 
4.4 The relationship between $t_i$ and the neighbours of the target node in Eq. 6 is not very clear.