Answers.Body: <p>The Cora dataset is unbalanced (s. <a href="https://stellargraph.readthedocs.io/en/v1.0.0rc1/demos/node-classification/gcn/gcn-cora-node-classification-example.html#Loading-the-CORA-network" rel="nofollow noreferrer">here</a>). It's graph consists of 2708 nodes and the label distribution (for labels 1 to 7) is <code>818, 426, 418, 351, 298, 217, 180</code>, i.e. there are 818 nodes labelled <code>0</code>, 426 nodes labelled <code>1</code> and so on. If your network always predicts the label <code>0</code>, then its accuracy should be <span class="math-container">$\frac{818}{2708} \approx 0.38$</span>. Which roughly matches your number, probably this is the reason why the accuracy is not <span class="math-container">$\frac{1}{7}$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-02T13:37:57.440Z
Answers.Text: Yes, indeed, when I generated uniformly 7 classes at random, the final accuracy on the test set was around 1/7, whereas, for the train set it was above 0.9 as expected.
Answers.UserDisplayName: RobJan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-02T11:50:35.193Z
Answers.LastActivityDate: 2023-06-02T11:50:35.193Z
Answers.UserDisplayName: Chillston
Body: <p>I decided to train GCN on the Cora dataset for the node classification task, however, with the random labels, i.e., applying <code>np.random.shuffle(labels)</code>. For the default set of parameters, I am getting an accuracy of around 0.3 for the test set and 0.4 for the train set. I expect that for the random labels, the accuracy would be <code>1/number of classes</code>. So in the case of Cora: <code>1/7 = 0.14</code>.</p>
  <p>Do you have any intuition why graph neural networks perform better than the random case? I am aware that in [1] authors trained the models on the random labels and achieved perfect results on the train set. However, for test size they still were around the <code>1/number of classes</code>.</p>
  <p>I checked simpler models such as RandomForests or SVC and the final accuracy for the test size is indeed 1/7.</p>
  <p>[1] Zhang, C., Bengio, S., Hardt, M., Recht, B., &amp; Vinyals, O. (2021). Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3), 107-115.</p>
  
Tags: <classification><graph-neural-networks>
Title: Node classification with random labels for GNNs
UserDisplayName: RobJan
ThreadText: # Node classification with random labels for GNNs
  
  (Asked by: username_0 on 2023-06-02)
  
  I decided to train GCN on the Cora dataset for the node classification task, however, with the random labels, i.e., applying `np.random.shuffle(labels)`. For the default set of parameters, I am getting an accuracy of around 0.3 for the test set and 0.4 for the train set. I expect that for the random labels, the accuracy would be `1/number of classes`. So in the case of Cora: `1/7 = 0.14`.
  
  Do you have any intuition why graph neural networks perform better than the random case? I am aware that in [1] authors trained the models on the random labels and achieved perfect results on the train set. However, for test size they still were around the `1/number of classes`.
  
  I checked simpler models such as RandomForests or SVC and the final accuracy for the test size is indeed 1/7.
  
  [1] Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021). Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3), 107-115.
  
  **Tags:** classification, graph-neural-networks
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-06-02. Score: 3 | Accepted answer
  
  The Cora dataset is unbalanced (s. [here](https://stellargraph.readthedocs.io/en/v1.0.0rc1/demos/node-classification/gcn/gcn-cora-node-classification-example.html#Loading-the-CORA-network)). It's graph consists of 2708 nodes and the label distribution (for labels 1 to 7) is `818, 426, 418, 351, 298, 217, 180`, i.e. there are 818 nodes labelled `0`, 426 nodes labelled `1` and so on. If your network always predicts the label `0`, then its accuracy should be $\frac{818}{2708} \approx 0.38$. Which roughly matches your number, probably this is the reason why the accuracy is not $\frac{1}{7}$.
  
  > **Comments:**
  > **username_0**: Yes, indeed, when I generated uniformly 7 classes at random, the final accuracy on the test set was around 1/7, whereas, for the train set it was above 0.9 as expected.  
  
  ---
  
  