GRAPH WAVELET NEURAL NETWORK
We present graph wavelet neural network (GWNN), a novel graph convolutional neural network (CNN), leveraging graph wavelet transform to address the shortcomings of previous spectral graph CNN methods that depend on graph Fourier transform. Different from graph Fourier transform, graph wavelet transform can be obtained via a fast algorithm without requiring matrix eigendecomposition with high computational cost. Moreover, graph wavelets are sparse and localized in vertex domain, offering high efficiency and good interpretability for graph convolution. The proposed GWNN significantly outperforms previous spectral graph CNNs in the task of graph-based semi-supervised classification on three benchmark datasets: Cora, Citeseer and Pubmed.
This is an empirical paper that proposes to design wavelets on graphs, that can be integrated to neural networks on graphs. It permits to reducing the number of parameters of the « convolution » and exploits the sparsity of sparse weighted graphs for computations. I think it’s an interesting work.
The perspective I enjoy in using wavelets is that they typically provide a good trade-off in localization in the spectral and graph domains. For instance, large eigenvalues of the laplacian could be potentially captured in a more stable way. This type of work might be a first step.
Pros :
- good numerical results
- nice incorporation of structure via wavelets
Cons :
- Sometimes the paper is not really clear
I have severals comments:
1/ (1) « Some subsequent works devote to making spectral methods spectrum-free… avoiding high computational cost ». I think also those types of representations can be potentially unstable. That’s a second reason.
2/ I’m not sure to understand the point (1) of the fourth paragraph of the introduction. (1.)« graph wavelet does not depend on the eigen decomposition of Laplacian matrix ». Does it mean numerically ? This sentence is not clear, because even numerically, it can be done in a dependent way to this matrix. However, if it is implied that the fastest algorithm can be obtained without eigen-decomposition of the Laplacian Matrix, in a cheap way, then I agree and a small rephrasing could be nice.
3/ Please remove all the sentences that are supposed to be an introduction for a section, i.e. the sentences between 2 and 2.1 (« We use graph… ») and 3 and 3.1 (« In many research… »). They are poorly written and do not help the reader.
5/ .(2.2) “However, such a polynomial approximation also limits the flexibility to define appropriate convolution on graph” I’m not sure to understand. In the paper you refer to, the set of filters span the polynomial of degree less than n, of a diagonalizable matrix of size nxn. Thus, lagrange polynomial basis could be used to interpolate any desired values? Does it signify learning non-diagonal(in the appropriate basis) matrix?
6/ (2.3) $s$ how is this parameter chosen, crossvalidation? Is it adjusted such that the singular values of $\psi_s$ have a certain decay? Why is $s$ constant across layers? How is the spectrum of psi_s? Is it well conditionned? A huge difference with (Hammond et al,2011) is that they use a collection of wavelets. Did you consider this kind of approach ? Is there a loss of information in some cases?(like if $\psi_s$ has a fast decay)
7/ (2.3) “The matrix $\psi_s$ and $\psi_s^{-1}$ are both sparse”. This is a critical affirmation which is not in general true. It is possibly sparse if the weighted laplacian graph is sparse as well, as explained by the remark after theorem 5.5 of page 16 of (Hammond et al, 2011). However, I do agree this typically happens in the application you present. 
8/ The (c) of Figure 1 is missing.(in my version at least)
9/ In section 5.3, why not trying to compare the number of parameters with other papers? I think also more results, with maybe a bit higher performances, could be reported, such as: GraphSGAN, GAT. But that’s fine to me.
10/ Appendix A, isn’t it a simple rephrasing of (Hammond et al, 2011)?
11/ How do you compare in term of interpretability with [1]?
12/ Just as a suggestion and/or a comment: it seems similar to approaches such as lifting schemes(which basically builds wavelets on graphs/manifolds), except that there is no learning involved. (e.g. [2]) I think there could be great connexions.
[1] Relational inductive biases, deep learning, and graph networks, Battaglia et al 2011?
[2] THE LIFTING SCHEME: A CONSTRUCTION OF SECOND GENERATION WAVELETS, Wim Sweldgens