Body: <p>I have a graph with many disjoint subgraphs that are not connected to each other. Essentially these subgraphs could represent different clusters. What is a general process to figure out node embeddings so that when I process a new incoming node, the resulting embedding will be near other nodes that belong to the same cluster that the inference time node would belong to. At inference time I am just receiving the node with its features but none of the structural information.</p>
  <p>I have been thinking about whether one could create an embedding space that contains individual nodes as well as entire clusters(these disjoint clusters are essentially graphs themselves so cluster embeddings would be the same as global graph embeddings of the subgraphs). Essentially if I have a node A, then the embedding for that node would be similar to the embedding of the entire cluster that Node A resides in(global cluster embedding). However I do not know if this is the right way to go about it.</p>
  <p>Overall my question how would someone create this node/cluster embedding space or if this is not possible, how would someone train a model that could take a new node with only its features at inference time and classify it to an already existing cluster?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-09T09:44:27.867Z
Comments.Text: For clarification, what is this task about? For example, what does it mean in the context of your task when two nodes are connected by an edge? The reason I am asking is that at inference time, you cannot really use the structure anymore, because you only have an individual node. To solve the task successfully, this purely local information must be enough, so I am wondering if you really need the representation as a graph.
Comments.UserDisplayName: Chillston
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-09T15:21:55.027Z
Comments.Text: @Chillston I have multiple subgraphs of accounts that are connected amongst themselves dues to some sort of similarity or interaction these accounts had. For example I could have 2 groups, dog lovers and cat lovers. Given a new node, I want to classify that isolated node to one of these graphs. Is it possible to use something like node2vec to generate embeddings for the nodes and then just perform clustering?
Comments.UserDisplayName: Kiran Manicka
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-12T11:58:28.500Z
Comments.Text: For node2vec you'd need the nodes embedded in a graph, so I don't think this would work very well if you only have a single node without knowledge about how it is part of the graph. My intuition would be that you have two main ways to approach this: 1) You treat this completely as a classification problem and disregard the structure (I would at least try this as a baseline) 2) You could try link prediction, given a graph and a node, predict which nodes it connects to and get the embedding this way.
Comments.UserDisplayName: Chillston
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-12T17:52:22.453Z
Comments.Text: @Chillston, I see why you don't think node2vec would work, because there would no one hot encoded representation of a new node at inference time. Do you think it would be possible to generate embeddings using a skip gram that takes in node features instead the one hot encoded vectors. So instead of passing a one hot encoded vector and trying to predict the surrounding nodes, the network would take a feature vector of a node and try to predict the surrounding nodes(output will still be one hot encoded). Therefore, this network can still produce embeddings on new nodes at inference time.
Comments.UserDisplayName: Kiran Manicka
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-13T09:43:03.853Z
Comments.Text: I don't think that there is THE way to solve this, but my intuition is that to solve the task in your question, you actually don't need the graph. You only want to assign an area of your embedding space to a single node, which is standard (non graph) classification. Node2Vec works by generating node representation based on the nodes position in the graph. By default node2vec also disregards node features. For you, the opposite is interesting, as you have node features but no idea how this node is situated in the graph, right? Maybe I am also missing a point about your task here.
Comments.UserDisplayName: Chillston
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-13T09:43:20.730Z
Comments.Text: Maybe you can help me understand the following better: If your node contains purely local information (no info on how it is connected to other nodes) then how would you use your graph to inform any prediction on this node?
Comments.UserDisplayName: Chillston
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-14T04:03:47.060Z
Comments.Text: @Chillston My task is I am trying to use graphSage to generate node embedding a graph that contains a high number of disjoint subgraphs. These disjoints subgraphs are pretty much their own graphs. After I create embeddings, I want to to take a new node, such as user application, and find out which of these disjoint subgraphs the node belongs to with embeddings. The existing nodes has structural and feature information, the new node only contains feature information. This where I am trying to figure out if I can incorporate structural information into the embeddings or only use feature info
Comments.UserDisplayName: Kiran Manicka
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-14T04:23:59.393Z
Comments.Text: @Chillston And even though Node2Vec is structural in nature, in that a node's embedding is indicative of the nodes that are likely to surround it(be in the same walk), for my usecase can't I pass in a node's feature information and attempt to predict the nodes that surround it(one hot encoded vector). This would mean at inference time for a new node, I could pass in its features and obtain an embedding that represents the nodes that are likely to surround it which would be close to the existing node embeddings in the appropriate cluster?
Comments.UserDisplayName: Kiran Manicka
Tags: <graph-neural-networks><embeddings><clustering><probabilistic-graphical-models><spectral-clustering>
Title: How to classify a new node into an existing cluster of nodes?
UserDisplayName: Kiran Manicka
ThreadText: # How to classify a new node into an existing cluster of nodes?
  
  (Asked by: username_0 on 2024-02-09)
  
  I have a graph with many disjoint subgraphs that are not connected to each other. Essentially these subgraphs could represent different clusters. What is a general process to figure out node embeddings so that when I process a new incoming node, the resulting embedding will be near other nodes that belong to the same cluster that the inference time node would belong to. At inference time I am just receiving the node with its features but none of the structural information.
  
  I have been thinking about whether one could create an embedding space that contains individual nodes as well as entire clusters(these disjoint clusters are essentially graphs themselves so cluster embeddings would be the same as global graph embeddings of the subgraphs). Essentially if I have a node A, then the embedding for that node would be similar to the embedding of the entire cluster that Node A resides in(global cluster embedding). However I do not know if this is the right way to go about it.
  
  Overall my question how would someone create this node/cluster embedding space or if this is not possible, how would someone train a model that could take a new node with only its features at inference time and classify it to an already existing cluster?
  
  **Tags:** graph-neural-networks, embeddings, clustering, probabilistic-graphical-models, spectral-clustering
  
  > **Comments:**
  > **username_1**: For clarification, what is this task about? For example, what does it mean in the context of your task when two nodes are connected by an edge? The reason I am asking is that at inference time, you cannot really use the structure anymore, because you only have an individual node. To solve the task successfully, this purely local information must be enough, so I am wondering if you really need the representation as a graph.  
  > **username_0**: @username_1 I have multiple subgraphs of accounts that are connected amongst themselves dues to some sort of similarity or interaction these accounts had. For example I could have 2 groups, dog lovers and cat lovers. Given a new node, I want to classify that isolated node to one of these graphs. Is it possible to use something like node2vec to generate embeddings for the nodes and then just perform clustering?  
  > **username_1**: For node2vec you'd need the nodes embedded in a graph, so I don't think this would work very well if you only have a single node without knowledge about how it is part of the graph. My intuition would be that you have two main ways to approach this: 1) You treat this completely as a classification problem and disregard the structure (I would at least try this as a baseline) 2) You could try link prediction, given a graph and a node, predict which nodes it connects to and get the embedding this way.  
  > **username_0**: @username_1, I see why you don't think node2vec would work, because there would no one hot encoded representation of a new node at inference time. Do you think it would be possible to generate embeddings using a skip gram that takes in node features instead the one hot encoded vectors. So instead of passing a one hot encoded vector and trying to predict the surrounding nodes, the network would take a feature vector of a node and try to predict the surrounding nodes(output will still be one hot encoded). Therefore, this network can still produce embeddings on new nodes at inference time.  
  > **username_1**: I don't think that there is THE way to solve this, but my intuition is that to solve the task in your question, you actually don't need the graph. You only want to assign an area of your embedding space to a single node, which is standard (non graph) classification. Node2Vec works by generating node representation based on the nodes position in the graph. By default node2vec also disregards node features. For you, the opposite is interesting, as you have node features but no idea how this node is situated in the graph, right? Maybe I am also missing a point about your task here.  
  > **username_1**: Maybe you can help me understand the following better: If your node contains purely local information (no info on how it is connected to other nodes) then how would you use your graph to inform any prediction on this node?  
  > **username_0**: @username_1 My task is I am trying to use graphSage to generate node embedding a graph that contains a high number of disjoint subgraphs. These disjoints subgraphs are pretty much their own graphs. After I create embeddings, I want to to take a new node, such as user application, and find out which of these disjoint subgraphs the node belongs to with embeddings. The existing nodes has structural and feature information, the new node only contains feature information. This where I am trying to figure out if I can incorporate structural information into the embeddings or only use feature info  
  > **username_0**: @username_1 And even though Node2Vec is structural in nature, in that a node's embedding is indicative of the nodes that are likely to surround it(be in the same walk), for my usecase can't I pass in a node's feature information and attempt to predict the nodes that surround it(one hot encoded vector). This would mean at inference time for a new node, I could pass in its features and obtain an embedding that represents the nodes that are likely to surround it which would be close to the existing node embeddings in the appropriate cluster?  
  
  ---
  
  ## 0 answers
  
  