Answers.Body: <p>The simplest way I could come with is to pad with 0 each feature which is not present. You said that you're going to add too much noise to the network, but I don't see the problem (please correct me if I'm wrong). For example we have two nodes, the first one has only 2 features with the 3rd one missing and the second node has all features X=[[1,2,0], [3,4,5]]. Now we can project the nodes to a hidden representation (pretty common). I'm going to use a weight matrix of W=[[1], [1], [1]]. The output of XW will be [[3], [12]]. Now let's add a new feature to the second node X=[[1,2,0, 0], [3,4,5,6]] and apply the same transformation W=[[1], [1], [1], [1]] the output will be [[3], [18]] you can see that the first node is not affected by the number of missing features. </p>
  
  <p>Another way you could achieve this if you don't want to use the projection could be using a mask. For example give the same example above we could create a mask M=[[1,1,0], [1,1,1]] where each entry represents if a specific feature is present in a specific node. Now usually a GCN layer is defined as H=f(AHW) where A is the adjacency matrix. We could change the propagation rule to H=f(AH*MW) where * is the pointwise multiplication. Like this if a node is missing a feature it can not "access" information from others that are having that feature.  </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-06T15:19:18.020Z
Answers.Text: Hello @razvanc92, thank you. I like the masking idea, thank you for the suggestion.
Answers.UserDisplayName: PasDeSence
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-06T15:12:30.660Z
Answers.LastActivityDate: 2020-05-22T07:59:47.497Z
Answers.LastEditDate: 2020-05-22T07:59:47.497Z
Answers.UserDisplayName: razvanc92
Answers.Body: <p>My immediate suggestion would be to zero-fill the missing values, but I recalled the below comment suggesting a more sophisticated method:</p>
  <blockquote>
  <p><strong>Karim:</strong> <em>How to deal with different size of feature vectors?</em></p>
  <p><strong>Nabila:</strong> <em>That's a problem I'm actually working on. I've seen that you can create separate networks for each type of node feature, and sort of project them - so train them separately, and project them to the same size.</em></p>
  <p><em>Or you can do concatenation so you don't have to worry about that, but at some point they all need to be the same size to do classification at the end.</em></p>
  </blockquote>
  <ul>
  <li><a href="https://youtu.be/54ohmr2LYuI?t=2346" rel="nofollow noreferrer">A Literature Review on Graph Neural Networks</a></li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-26T22:25:38.333Z
Answers.LastActivityDate: 2020-05-26T22:25:38.333Z
Answers.LastEditDate: 2020-06-17T09:57:20.710Z
Answers.UserDisplayName: brazofuerte
Body: <p>I have a question regarding features representation for graph convolutional neural network.</p>
  
  <p>For my case, all nodes have a different number of features, and for now, I don't really understand how should I work with these constraints. I can not just reduce the number of features or add meaningless features in order to make the number of features on each node the same - because it will add to much extra noise to the network.</p>
  
  <p>Are there any ways to solve this problem? How should I construct the feature matrix?</p>
  
  <p>I'll appreciate any help and if you have any links to papers that solve this problem.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-06T14:53:42.263Z
Comments.Text: Hi and welcome to AI SE! The addition of dummy values isn't that uncommon in deep learning. For example, when doing sequence processing, one may require all sequences to have the same length. In that case, the sequences may be padded with dummy values. (Anyway, although I am familiar with GDL and graph NNs, I am not qualified enough to give an answer to your question).
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-06T14:58:41.533Z
Comments.Text: Hi @nbro and thanks for your answer. May I ask you, how to choose the dummy values in this case? Do you have any idea of methods that can be used, so that  I don't broke completely the distribution?
Comments.UserDisplayName: PasDeSence
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-06T15:09:57.193Z
Comments.Text: Honestly, I should not suggest something that may not be appropriate. But maybe try some dummy values that are consistent with your representation of the features, train the networks, see the results, and iterate (i.e try other ideas). For example, whenever there are missing features, you could have zeros, which, when multiplied around, will have the effect of zeroing contributions. I hope someone else will provide more help, but GDL is a quite new area, so many techniques have yet to be developed. It's also been a while I've not read a GDL paper, so I am a bit rusty :)
Comments.UserDisplayName: nbro
Tags: <machine-learning><deep-learning><convolutional-neural-networks><geometric-deep-learning>
Title: How to represent and work with the feature matrix for graph convolutional network (GCN) if the number of features for each node is different?
UserDisplayName: PasDeSence
ThreadText: # How to represent and work with the feature matrix for graph convolutional network (GCN) if the number of features for each node is different?
  
  (Asked by: username_0 on 2020-02-06)
  
  I have a question regarding features representation for graph convolutional neural network.
  
  For my case, all nodes have a different number of features, and for now, I don't really understand how should I work with these constraints. I can not just reduce the number of features or add meaningless features in order to make the number of features on each node the same - because it will add to much extra noise to the network.
  
  Are there any ways to solve this problem? How should I construct the feature matrix?
  
  I'll appreciate any help and if you have any links to papers that solve this problem.
  
  **Tags:** machine-learning, deep-learning, convolutional-neural-networks, geometric-deep-learning
  > **Comments:**
  > **username_1**: Hi and welcome to AI SE! The addition of dummy values isn't that uncommon in deep learning. For example, when doing sequence processing, one may require all sequences to have the same length. In that case, the sequences may be padded with dummy values. (Anyway, although I am familiar with GDL and graph NNs, I am not qualified enough to give an answer to your question).  
  > **username_0**: Hi @nbro and thanks for your answer. May I ask you, how to choose the dummy values in this case? Do you have any idea of methods that can be used, so that  I don't broke completely the distribution?  
  > **username_1**: Honestly, I should not suggest something that may not be appropriate. But maybe try some dummy values that are consistent with your representation of the features, train the networks, see the results, and iterate (i.e try other ideas). For example, whenever there are missing features, you could have zeros, which, when multiplied around, will have the effect of zeroing contributions. I hope someone else will provide more help, but GDL is a quite new area, so many techniques have yet to be developed. It's also been a while I've not read a GDL paper, so I am a bit rusty :)  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-02-06. Score: 2
  
  The simplest way I could come with is to pad with 0 each feature which is not present. You said that you're going to add too much noise to the network, but I don't see the problem (please correct me if I'm wrong). For example we have two nodes, the first one has only 2 features with the 3rd one missing and the second node has all features X=[[1,2,0], [3,4,5]]. Now we can project the nodes to a hidden representation (pretty common). I'm going to use a weight matrix of W=[[1], [1], [1]]. The output of XW will be [[3], [12]]. Now let's add a new feature to the second node X=[[1,2,0, 0], [3,4,5,6]] and apply the same transformation W=[[1], [1], [1], [1]] the output will be [[3], [18]] you can see that the first node is not affected by the number of missing features.
  
  Another way you could achieve this if you don't want to use the projection could be using a mask. For example give the same example above we could create a mask M=[[1,1,0], [1,1,1]] where each entry represents if a specific feature is present in a specific node. Now usually a GCN layer is defined as H=f(AHW) where A is the adjacency matrix. We could change the propagation rule to H=f(AH\*MW) where \* is the pointwise multiplication. Like this if a node is missing a feature it can not "access" information from others that are having that feature.
  
  > **Comments:**
  > **username_0**: Hello @razvanc92, thank you. I like the masking idea, thank you for the suggestion.  
  
  ---
  
  ### Answer by username_3 on 2020-05-26. Score: 0
  
  My immediate suggestion would be to zero-fill the missing values, but I recalled the below comment suggesting a more sophisticated method:
  
  > **Karim:** *How to deal with different size of feature vectors?*
  >
  > **Nabila:** *That's a problem I'm actually working on. I've seen that you can create separate networks for each type of node feature, and sort of project them - so train them separately, and project them to the same size.*
  >
  > *Or you can do concatenation so you don't have to worry about that, but at some point they all need to be the same size to do classification at the end.*
  
  * [A Literature Review on Graph Neural Networks](https://youtu.be/54ohmr2LYuI?t=2346)
  
  
  ---
  
  