Dimensional Reweighting Graph Convolution Networks
In this paper, we propose a method named Dimensional reweighting Graph Convolutional Networks (DrGCNs), to tackle the problem of variance between dimensional information in the node representations of GCNs. We prove that DrGCNs can reduce the variance of the node representations by connecting our problem to the theory of the mean field. However, practically, we find that the degrees DrGCNs help vary severely on different datasets. We revisit the problem and develop a new measure K to quantify the effect. This measure guides when we should use dimensional reweighting in GCNs and how much it can help. Moreover, it offers insights to explain the improvement obtained by the proposed DrGCNs. The dimensional reweighting block is light-weighted and highly flexible to be built on most of the GCN variants. Carefully designed experiments, including several fixes on duplicates, information leaks, and wrong labels of the well-known node classification benchmark datasets, demonstrate the superior performances of DrGCNs over the existing state-of-the-art approaches. Significant improvements can also be observed on a large scale industrial dataset.
This paper proposes a method called *Dimensional Reweighting* for graph convolutional networks. The method involves a reparametrization of GCNs (by adding an extra reweighting block in each layer), which the authors show theoretically can reduce variance. The authors supplement this with extensive empirical experiments showing slightly improved performance by adding their method to existing methods. I would vote to weakly reject this paper for two key reasons - first, I think the writing can be improved and explanations can be clarified, especially for people less familiar with the field like myself. Second, I am not certain how significant the final experimental improvements are (other than on the Reddit dataset), as most of the numbers are not that far apart, and it seems that different methods in the literature already produce fairly different results. Overall, I think the structure of the paper is fairly good. I feel that a few things should be modified for clarity. - You claim a 40% improvement in error rate in the intro, which sounds enormous. I would say *relative error rate* to avoid overclaiming, because 40% improvement sounds like you are reducing (absolute) error from 60% to 20%, while in reality you are reducing error from 3.6% to 2.1%. - In section 2.1, did not know if the projection matrix W was learned or predefined. - I was not sure why you used sigma_g and sigma_s as opposed to just sigma in equation (5). Do you use different activation functions? Also, I did not find what activation function the authors end up using in their experiments. - I may have misunderstood something, but the theory does not seem to match the proposed method exactly. The mean-field theory analysis has the activation function after the reweighting by S but before multiplying by W, while the framework in Section 2.2 has the activation after the reweighting by S and after the multiplying by W. I am not sure how much this difference makes, or if it is significant, but I think it should be explained by the authors. - I also did not understand exactly what the *variance* the authors are reducing is. The authors talks about *reducing the learning variance brought by perturbations on the input,* but when is the input ever perturbed for GCNs? Explaining this more clearly would improve the motivation for this work. - I would appreciate a better intuitive explanation of the measure *K.* I gather that it is related to the *variance* being reduced, but it is different from that. The experimental results are good overall, as the proposed method tends to give the best results (by a small margin) across the board. I especially appreciate that the authors performed many experiments over many different datasets and repeated runs 20 times to try to get confident estimates of how well each method performs. I also appreciate that the authors cleaned up the Citeseer and Cora datasets, and I hope the cleaned datasets will be useful for the research community. With that said, I do not know how significant the improvement is. I think something that would be helpful would be to measure the *variance* that the method is supposed to be reducing (since it sounds like it is not exactly the same thing as K), and showing that in a table as well. This would show experimentally that the method achieves its intended goal. Minor comments - I would recommend that the authors proofread for English grammar and style in updated versions of the paper. For example, in the first paragraph of the introduction, the authors use *is proposed* instead of *were proposed* and typo *broad* as *board.* - Just curious, why did you choose a 2 layer network with 2 activation functions for the Dr block? Why not just have 1 hidden layer?
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Dimensional Reweighting Graph Convolution Networks
In this paper, we propose a method named Dimensional reweighting Graph Convolutional Networks (DrGCNs), to tackle the problem of variance between dimensional information in the node representations of GCNs. We prove that DrGCNs can reduce the variance of the node representations by connecting our problem to the theory of the mean field. However, practically, we find that the degrees DrGCNs help vary severely on different datasets. We revisit the problem and develop a new measure K to quantify the effect. This measure guides when we should use dimensional reweighting in GCNs and how much it can help. Moreover, it offers insights to explain the improvement obtained by the proposed DrGCNs. The dimensional reweighting block is light-weighted and highly flexible to be built on most of the GCN variants. Carefully designed experiments, including several fixes on duplicates, information leaks, and wrong labels of the well-known node classification benchmark datasets, demonstrate the superior performances of DrGCNs over the existing state-of-the-art approaches. Significant improvements can also be observed on a large scale industrial dataset.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes a method called *Dimensional Reweighting* for graph convolutional networks. The method involves a reparametrization of GCNs (by adding an extra reweighting block in each layer), which the authors show theoretically can reduce variance. The authors supplement this with extensive empirical experiments showing slightly improved performance by adding their method to existing methods. I would vote to weakly reject this paper for two key reasons - first, I think the writing can be improved and explanations can be clarified, especially for people less familiar with the field like myself. Second, I am not certain how significant the final experimental improvements are (other than on the Reddit dataset), as most of the numbers are not that far apart, and it seems that different methods in the literature already produce fairly different results. Overall, I think the structure of the paper is fairly good. I feel that a few things should be modified for clarity. - You claim a 40% improvement in error rate in the intro, which sounds enormous. I would say *relative error rate* to avoid overclaiming, because 40% improvement sounds like you are reducing (absolute) error from 60% to 20%, while in reality you are reducing error from 3.6% to 2.1%. - In section 2.1, did not know if the projection matrix W was learned or predefined. - I was not sure why you used sigma_g and sigma_s as opposed to just sigma in equation (5). Do you use different activation functions? Also, I did not find what activation function the authors end up using in their experiments. - I may have misunderstood something, but the theory does not seem to match the proposed method exactly. The mean-field theory analysis has the activation function after the reweighting by S but before multiplying by W, while the framework in Section 2.2 has the activation after the reweighting by S and after the multiplying by W. I am not sure how much this difference makes, or if it is significant, but I think it should be explained by the authors. - I also did not understand exactly what the *variance* the authors are reducing is. The authors talks about *reducing the learning variance brought by perturbations on the input,* but when is the input ever perturbed for GCNs? Explaining this more clearly would improve the motivation for this work. - I would appreciate a better intuitive explanation of the measure *K.* I gather that it is related to the *variance* being reduced, but it is different from that. The experimental results are good overall, as the proposed method tends to give the best results (by a small margin) across the board. I especially appreciate that the authors performed many experiments over many different datasets and repeated runs 20 times to try to get confident estimates of how well each method performs. I also appreciate that the authors cleaned up the Citeseer and Cora datasets, and I hope the cleaned datasets will be useful for the research community. With that said, I do not know how significant the improvement is. I think something that would be helpful would be to measure the *variance* that the method is supposed to be reducing (since it sounds like it is not exactly the same thing as K), and showing that in a table as well. This would show experimentally that the method achieves its intended goal. Minor comments - I would recommend that the authors proofread for English grammar and style in updated versions of the paper. For example, in the first paragraph of the introduction, the authors use *is proposed* instead of *were proposed* and typo *broad* as *board.* - Just curious, why did you choose a 2 layer network with 2 activation functions for the Dr block? Why not just have 1 hidden layer?
assistant
This paper proposes a method called *Dimensional Reweighting* for graph convolutional networks. The method involves a reparametrization of GCNs (by adding an extra reweighting block in each layer), which the authors show theoretically can reduce variance. The authors supplement this with extensive empirical experiments showing slightly improved performance by adding their method to existing methods. I would vote to weakly reject this paper for two key reasons - first, I think the writing can be improved and explanations can be clarified, especially for people less familiar with the field like myself. Second, I am not certain how significant the final experimental improvements are (other than on the Reddit dataset), as most of the numbers are not that far apart, and it seems that different methods in the literature already produce fairly different results. Overall, I think the structure of the paper is fairly good. I feel that a few things should be modified for clarity. - You claim a 40% improvement in error rate in the intro, which sounds enormous. I would say *relative error rate* to avoid overclaiming, because 40% improvement sounds like you are reducing (absolute) error from 60% to 20%, while in reality you are reducing error from 3.6% to 2.1%. - In section 2.1, did not know if the projection matrix W was learned or predefined. - I was not sure why you used sigma_g and sigma_s as opposed to just sigma in equation (5). Do you use different activation functions? Also, I did not find what activation function the authors end up using in their experiments. - I may have misunderstood something, but the theory does not seem to match the proposed method exactly. The mean-field theory analysis has the activation function after the reweighting by S but before multiplying by W, while the framework in Section 2.2 has the activation after the reweighting by S and after the multiplying by W. I am not sure how much this difference makes, or if it is significant, but I think it should be explained by the authors. - I also did not understand exactly what the *variance* the authors are reducing is. The authors talks about *reducing the learning variance brought by perturbations on the input,* but when is the input ever perturbed for GCNs? Explaining this more clearly would improve the motivation for this work. - I would appreciate a better intuitive explanation of the measure *K.* I gather that it is related to the *variance* being reduced, but it is different from that. The experimental results are good overall, as the proposed method tends to give the best results (by a small margin) across the board. I especially appreciate that the authors performed many experiments over many different datasets and repeated runs 20 times to try to get confident estimates of how well each method performs. I also appreciate that the authors cleaned up the Citeseer and Cora datasets, and I hope the cleaned datasets will be useful for the research community. With that said, I do not know how significant the improvement is. I think something that would be helpful would be to measure the *variance* that the method is supposed to be reducing (since it sounds like it is not exactly the same thing as K), and showing that in a table as well. This would show experimentally that the method achieves its intended goal. Minor comments - I would recommend that the authors proofread for English grammar and style in updated versions of the paper. For example, in the first paragraph of the introduction, the authors use *is proposed* instead of *were proposed* and typo *broad* as *board. * - Just curious, why did you choose a 2 layer network with 2 activation functions for the Dr block? Why not just have 1 hidden layer?