Demystifying Graph Neural Network Via Graph Filter Assessment
Graph Neural Networks (GNNs) have received tremendous attention recently due to their power in handling graph data for different downstream tasks across different application domains. The key of GNN is its graph convolutional filters, and recently various kinds of filters are designed. However, there still lacks in-depth analysis on (1) Whether there exists a best filter that can perform best on all graph data; (2) Which graph properties will influence the optimal choice of graph filter; (3) How to design appropriate filter adaptive to the graph data. In this paper, we focus on addressing the above three questions. We first propose a novel assessment tool to evaluate the effectiveness of graph convolutional filters for a given graph. Using the assessment tool, we find out that there is no single filter as a `silver bullet* that perform the best on all possible graphs. In addition, different graph structure properties will influence the optimal graph convolutional filter*s design choice. Based on these findings, we develop Adaptive Filter Graph Neural Network (AFGNN), a simple but powerful model that can adaptively learn task-specific filter. For a given graph, it leverages graph filter assessment as regularization and learns to combine from a set of base filters. Experiments on both synthetic and real-world benchmark datasets demonstrate that our proposed model can indeed learn an appropriate filter and perform well on graph tasks.
This paper introduces an assessment framework for an in-depth analysis of the effect of graph convolutional filters and proposes a novel graph neural network with adaptable filters based on the analysis. The assessment framework builts on the Fisher discriminant score of features and can also be used as an additional (regularization) term for choosing optimal filters in training. The assessment result shows that there is no single graph filter for all types of graph structures. Experiments on both synthetic and real-world benchmark datasets demonstrate that the proposed adaptive GNN can learn appropriate filters for different graph tasks. The proposed analysis using the Fisher score is reasonable and interesting, giving us an insight into the role of graph filters. Even though the analysis is limited (using simple graph models and filter family) and the result is not surprising (given no free lunch theorem, there is very likely to be no single silver bullet fo graph filters), I appreciate the analysis and the result. But, I have some concerns as follows. 1) The proposed GNN and the optimization process The proposed method is to extend CNN to a simple linear combination of different filter bases with learnable weights, which I don*t think is very novel. Adding the GFD score as an additional constraint term is interesting, but the way of optimizing the whole objective function is unclear. (In addition, I think calling it the *regularization term* is inadequate since the term actually involves data observation, rather than a prior on parameters only.) In the case of AFGNN_inf, I don*t think it is equivalent to applying infinite lamda. If lamda is infinite, L_CE needs to be completely ignored. This needs to be clarified. In the case of AFGNN1, I don*t clearly understand how the whole objective function is properly optimized with fixed data representation. Is it also iteratively optimized? I hope this is also clarified in more detail. 2) Unconvincing experiments The results on three real datasets do not show significant gains, and two of them are even worse than those of GAT. Furthermore, inductive learning (e.g., protein-protein interaction (PPI) dataset used in GAT) is not tested, which I think needs to be also evaluated. While two synthetic datasets (SmallGap and SmallRatio) created by the authors show significant improvement, these datasets appear to be extreme and unrealistic and look carefully selected in favor of the proposed method. I recommend the authors use for evaluation more realistic datasets that can be found in related research.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Demystifying Graph Neural Network Via Graph Filter Assessment
Graph Neural Networks (GNNs) have received tremendous attention recently due to their power in handling graph data for different downstream tasks across different application domains. The key of GNN is its graph convolutional filters, and recently various kinds of filters are designed. However, there still lacks in-depth analysis on (1) Whether there exists a best filter that can perform best on all graph data; (2) Which graph properties will influence the optimal choice of graph filter; (3) How to design appropriate filter adaptive to the graph data. In this paper, we focus on addressing the above three questions. We first propose a novel assessment tool to evaluate the effectiveness of graph convolutional filters for a given graph. Using the assessment tool, we find out that there is no single filter as a `silver bullet* that perform the best on all possible graphs. In addition, different graph structure properties will influence the optimal graph convolutional filter*s design choice. Based on these findings, we develop Adaptive Filter Graph Neural Network (AFGNN), a simple but powerful model that can adaptively learn task-specific filter. For a given graph, it leverages graph filter assessment as regularization and learns to combine from a set of base filters. Experiments on both synthetic and real-world benchmark datasets demonstrate that our proposed model can indeed learn an appropriate filter and perform well on graph tasks.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper introduces an assessment framework for an in-depth analysis of the effect of graph convolutional filters and proposes a novel graph neural network with adaptable filters based on the analysis. The assessment framework builts on the Fisher discriminant score of features and can also be used as an additional (regularization) term for choosing optimal filters in training. The assessment result shows that there is no single graph filter for all types of graph structures. Experiments on both synthetic and real-world benchmark datasets demonstrate that the proposed adaptive GNN can learn appropriate filters for different graph tasks. The proposed analysis using the Fisher score is reasonable and interesting, giving us an insight into the role of graph filters. Even though the analysis is limited (using simple graph models and filter family) and the result is not surprising (given no free lunch theorem, there is very likely to be no single silver bullet fo graph filters), I appreciate the analysis and the result. But, I have some concerns as follows. 1) The proposed GNN and the optimization process The proposed method is to extend CNN to a simple linear combination of different filter bases with learnable weights, which I don*t think is very novel. Adding the GFD score as an additional constraint term is interesting, but the way of optimizing the whole objective function is unclear. (In addition, I think calling it the *regularization term* is inadequate since the term actually involves data observation, rather than a prior on parameters only.) In the case of AFGNN_inf, I don*t think it is equivalent to applying infinite lamda. If lamda is infinite, L_CE needs to be completely ignored. This needs to be clarified. In the case of AFGNN1, I don*t clearly understand how the whole objective function is properly optimized with fixed data representation. Is it also iteratively optimized? I hope this is also clarified in more detail. 2) Unconvincing experiments The results on three real datasets do not show significant gains, and two of them are even worse than those of GAT. Furthermore, inductive learning (e.g., protein-protein interaction (PPI) dataset used in GAT) is not tested, which I think needs to be also evaluated. While two synthetic datasets (SmallGap and SmallRatio) created by the authors show significant improvement, these datasets appear to be extreme and unrealistic and look carefully selected in favor of the proposed method. I recommend the authors use for evaluation more realistic datasets that can be found in related research.
assistant
This paper introduces an assessment framework for an in-depth analysis of the effect of graph convolutional filters and proposes a novel graph neural network with adaptable filters based on the analysis. The assessment framework builts on the Fisher discriminant score of features and can also be used as an additional (regularization) term for choosing optimal filters in training. The assessment result shows that there is no single graph filter for all types of graph structures. Experiments on both synthetic and real-world benchmark datasets demonstrate that the proposed adaptive GNN can learn appropriate filters for different graph tasks. The proposed analysis using the Fisher score is reasonable and interesting, giving us an insight into the role of graph filters. Even though the analysis is limited (using simple graph models and filter family) and the result is not surprising (given no free lunch theorem, there is very likely to be no single silver bullet fo graph filters), I appreciate the analysis and the result. But, I have some concerns as follows. 1) The proposed GNN and the optimization process The proposed method is to extend CNN to a simple linear combination of different filter bases with learnable weights, which I don*t think is very novel. Adding the GFD score as an additional constraint term is interesting, but the way of optimizing the whole objective function is unclear. (In addition, I think calling it the *regularization term* is inadequate since the term actually involves data observation, rather than a prior on parameters only.) In the case of AFGNN_inf, I don*t think it is equivalent to applying infinite lamda. If lamda is infinite, L_CE needs to be completely ignored. This needs to be clarified. In the case of AFGNN1, I don*t clearly understand how the whole objective function is properly optimized with fixed data representation. Is it also iteratively optimized? I hope this is also clarified in more detail. 2) Unconvincing experiments The results on three real datasets do not show significant gains, and two of them are even worse than those of GAT. Furthermore, inductive learning (e.g., protein-protein interaction (PPI) dataset used in GAT) is not tested, which I think needs to be also evaluated. While two synthetic datasets (SmallGap and SmallRatio) created by the authors show significant improvement, these datasets appear to be extreme and unrealistic and look carefully selected in favor of the proposed method. I recommend the authors use for evaluation more realistic datasets that can be found in related research.