RetGK: Graph Kernels based on Return Probabilities of Random Walks
Graph-structured data arise in wide applications, such as computer vision, bioinformatics, and social networks. Quantifying similarities among graphs is a fundamental problem. In this paper, we develop a framework for computing graph kernels, based on return probabilities of random walks. The advantages of our proposed kernels are that they can effectively exploit various node attributes, while being scalable to large datasets. We conduct extensive graph classification experiments to evaluate our graph kernels. The experimental results show that our graph kernels significantly outperform existing state-of-the-art approaches in both accuracy and computational efficiency.
The paper proposes a kernel for graphs able to deal with discrete and continuous labels. In particular, the topology information of a graph is encoded at node level by a return random walk probability vector (each dimension being associated to a different walk length). This probability vector is obtained by classical equations used by random walk kernels for graphs (T. Gartner et al. classical LTKM paper) but just focussing on return random walks. Thanks to that the computational complexity can be reduced since only the entries on the diagonal of the powers of the transition probability matrix need to be computed. This can be done via eigen-decomposition of a rescaled version of the adjacency matrix. Notwithstanding that, for large  graphs  the computational complexity is still too high and Monte Carlo methods are suggested as a practical way to get estimates of the quantities of interest. Having obtained the (approximated) return probability vector for each node of the graph, an embedding of a graph combining also the original information attached to each node (via tensor product of kernels) is obtained by kernel mean embedding (proposed in a JMLR 2012 paper).  Again, the embedding is computationally too expensive for large graphs, so the paper proposes to exploit previous work on approximate explicit feature maps (NIPS 2008 paper by Rahimi & Recht) to get a moderate size explicit tensor representation of graphs.  All the proposed kernels exploit either a polynomial or a RBF transformation (exploiting random Fourier feature map for the approximated version). Experimental assessment is performed on classical benchmarking datasets involving just structural information or additional discrete and continuous information at node level.  Quality: The mathematical treatment is solid as well as all the concepts that are used as building blocks of the proposed approach. I just have few concerns on the following points:  i) one of the problems of random walks is tottering (Mahe et al., ICML 2004), i.e. the fact that short paths in the graph can be visited many times forward and backward by a walk, thus over representing that topological feature with respect to walks supported by longer paths. This problem is particularly severe for return walks. In the paper there is no discussion about this well known problem as well as there is no analysis of the impact of this problem on the quality of the obtained vectorial representations. ii) the properties of the return probability features (RPF) may not be preserved by their approximated versions. For example, isomorphism-invariance is not valid anymore since the same graph with different (or even same) representations can get different RPFs because of the Monte Carlo approximation. The paper does not mention this issues as well as does not present empirical evidence of the quality of the obtained representations. iii) since nonlinear transformations (polynomial and RBF) are used on top of RPFs and additional node information, it is not clear to me how fair is the experimental comparison. In fact, I guess that a fair comparison would imply the use of the very same kernels on top of the other kernels taken from the literature. It is not clear to me how much of the observed improvements  (in some cases marginal with respect to the state-of-the-art) are due to that. iv) concerning the experimental comparison versus graph kernels for continuous attributes, the author missed a recent proposal that is relevant and uses some of the paper datasets:  Giovanni Da San Martino, Nicolò Navarin, Alessandro Sperduti: Tree-Based Kernel for Graphs With Continuous Attributes. IEEE Trans. Neural Netw. Learning Syst. 29(7): 3270-3276 (2018).  Clarity: the paper is well written and can be followed with no problem by an expert in the field. Organisation of the material is good. The level of presentation allows the reader to both have a general idea of the flow of reasoning and some technical details that characterise the proposed approach.  Originality: the paper builds on results from the literature. The idea to just use the return probabilities of random walks is new to my knowledge, although it can be considered a bit derivative. Some effort has been put into the derivation of the theoretical results concerning RPFs and new proposed kernels. Overall I see some original contribution here.  Significance: I guess the main contribution of the proposed kernels is the possibility to deal with a large variety of graphs (with no information/discrete/continuous attached to nodes). Not sure about the impact on cases where information is attached to nodes. Follow-up could involve other ways to code structural information.  After rebuttal. I am not convinced about the argument with tottering. Even if labels are not considered, still there will be a strong bias on closer nodes. The authors should find a more convincing argument.