Measuring and Improving the Use of Graph Information in Graph Neural Networks
Graph neural networks (GNNs) have been widely used for representation learning on graph data. However, there is limited understanding on how much performance GNNs actually gain from graph data. This paper introduces a context-surrounding GNN framework and proposes two smoothness metrics to measure the quantity and quality of information obtained from graph data. A new, improved GNN model, called CS-GNN, is then devised to improve the use of graph information based on the smoothness values of a graph. CS-GNN is shown to achieve better performance than existing methods in different types of real graphs.
This paper proposes two smoothness metrics to measure the quantity and quality of the graph information that GNNs employ in the model. With the smoothness measures, this paper proposes a new GNN model to improve the use of graph information. Overall, the paper is well-organized and clearly written. The main concern is the novelty, since the proposed method is pretty close to the graph attention network (GAT), except using the two smoothness metrics and a slightly different way to compute the attention coefficients. Experimental results show marginal improvement over existing GNN methods on the node classification task. Given these aspects, it is not that convincing that the introduced smoothness metrics are necessary. I would like to recommend a weak reject for this paper. Suggestions to improve the paper: 1) It would be better to provide more convincing evidence and motivation for the smoothness metrics, either in theory or empirical analysis. 2) When describing the proposed method, organize the key differences in a more clear way. For example, add the proposed method directly in Table 1 and summarize the key differences in bullet points. Currently, this important part is deferred to Section 3.3, which may cause confusion for the readers to understand the paper. 3) In Section 3.2, this paper claims that the proposed method can easily include side information on graphs to improve performance, by using the topology feature in the last fully connected layer for class prediction. However, this technique can also be used in existing GNN models, and it is not clear why this is described as something unique for the proposed method. Also, there is no corresponding ablation experiments to compare the performance of the proposed method with and without using local topology features.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Measuring and Improving the Use of Graph Information in Graph Neural Networks
Graph neural networks (GNNs) have been widely used for representation learning on graph data. However, there is limited understanding on how much performance GNNs actually gain from graph data. This paper introduces a context-surrounding GNN framework and proposes two smoothness metrics to measure the quantity and quality of information obtained from graph data. A new, improved GNN model, called CS-GNN, is then devised to improve the use of graph information based on the smoothness values of a graph. CS-GNN is shown to achieve better performance than existing methods in different types of real graphs.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes two smoothness metrics to measure the quantity and quality of the graph information that GNNs employ in the model. With the smoothness measures, this paper proposes a new GNN model to improve the use of graph information. Overall, the paper is well-organized and clearly written. The main concern is the novelty, since the proposed method is pretty close to the graph attention network (GAT), except using the two smoothness metrics and a slightly different way to compute the attention coefficients. Experimental results show marginal improvement over existing GNN methods on the node classification task. Given these aspects, it is not that convincing that the introduced smoothness metrics are necessary. I would like to recommend a weak reject for this paper. Suggestions to improve the paper: 1) It would be better to provide more convincing evidence and motivation for the smoothness metrics, either in theory or empirical analysis. 2) When describing the proposed method, organize the key differences in a more clear way. For example, add the proposed method directly in Table 1 and summarize the key differences in bullet points. Currently, this important part is deferred to Section 3.3, which may cause confusion for the readers to understand the paper. 3) In Section 3.2, this paper claims that the proposed method can easily include side information on graphs to improve performance, by using the topology feature in the last fully connected layer for class prediction. However, this technique can also be used in existing GNN models, and it is not clear why this is described as something unique for the proposed method. Also, there is no corresponding ablation experiments to compare the performance of the proposed method with and without using local topology features.
assistant
This paper proposes two smoothness metrics to measure the quantity and quality of the graph information that GNNs employ in the model. With the smoothness measures, this paper proposes a new GNN model to improve the use of graph information. Overall, the paper is well-organized and clearly written. The main concern is the novelty, since the proposed method is pretty close to the graph attention network (GAT), except using the two smoothness metrics and a slightly different way to compute the attention coefficients. Experimental results show marginal improvement over existing GNN methods on the node classification task. Given these aspects, it is not that convincing that the introduced smoothness metrics are necessary. I would like to recommend a weak reject for this paper. Suggestions to improve the paper: 1) It would be better to provide more convincing evidence and motivation for the smoothness metrics, either in theory or empirical analysis. 2) When describing the proposed method, organize the key differences in a more clear way. For example, add the proposed method directly in Table 1 and summarize the key differences in bullet points. Currently, this important part is deferred to Section 3.3, which may cause confusion for the readers to understand the paper. 3) In Section 3.2, this paper claims that the proposed method can easily include side information on graphs to improve performance, by using the topology feature in the last fully connected layer for class prediction. However, this technique can also be used in existing GNN models, and it is not clear why this is described as something unique for the proposed method. Also, there is no corresponding ablation experiments to compare the performance of the proposed method with and without using local topology features.