Answers.Body: <p>Embeddings are vectors. Layers are functions.</p>
  <p>So, node embeddings (e.g. produced by <a href="https://proceedings.neurips.cc/paper/2013/file/1cecc7a77928ca8133fa24680a88d2f9-Paper.pdf" rel="nofollow noreferrer">TransE</a>) are analogous to <a href="https://proceedings.neurips.cc/paper/2013/file/9aa42b31882ec039965f3c4923ce901b-Paper.pdf" rel="nofollow noreferrer">word embeddings</a> or <a href="https://arxiv.org/pdf/1803.09473.pdf" rel="nofollow noreferrer">code embeddings</a>, i.e. they are vector (and lower-dimensional) representations of the original objects, such that similar objects have similar embeddings, for some notion of similarity.</p>
  <p>Your question is analogous to &quot;Are convolutional layers analogous to feature maps?&quot;. So, you could use neural networks with layers to learn embeddings, but this does not have to be the case. For example, TransE does not &quot;really&quot; use a neural network (i.e. a sequence of layers that compute a non-linear function of the inputs), but it just optimizes the <span class="math-container">$L_2$</span> norm of a simple function of the vector representation of the nodes and edges, although it uses gradient descent.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-30T06:12:32.950Z
Answers.Text: makes so much sense, so that means embeddings are linked to model they are learned with? I thought embeddings are more universal as a space that separate two dissimilar things but here it appears that to use the embedding we need, the network on which it has been learned on, as well
Answers.UserDisplayName: user0193
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-03-12T22:14:41.327Z
Answers.LastActivityDate: 2022-03-12T22:14:41.327Z
Answers.UserDisplayName: nbro
Body: <p>So in Graph Neural Network (GNN) we have node embeddings which is a feature vector that describes the node, is it analogous to hidden layer of Artificial neural network such as feed-forward neural network? After all hidden layer stores set of weights and biases to extract some &quot;feature&quot;, which begs the question if node embeddings are analogous to a hidden layer?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-03-05T21:21:49.720Z
Comments.Text: In my understanding embedding is a static vector, but hidden layer is a function. But this is a good question as hidden layer is indeed extracting a feature, while node embedding represents features of the node.
Comments.UserDisplayName: gfdsal
Tags: <neural-networks><feedforward-neural-networks><graph-neural-networks><embeddings>
Title: Is "node embedding" in GNN analogous to "hidden layer" of FFN?
UserDisplayName: user0193
ThreadText: # Is "node embedding" in GNN analogous to "hidden layer" of FFN?
  
  (Asked by: username_0 on 2022-03-05)
  
  So in Graph Neural Network (GNN) we have node embeddings which is a feature vector that describes the node, is it analogous to hidden layer of Artificial neural network such as feed-forward neural network? After all hidden layer stores set of weights and biases to extract some "feature", which begs the question if node embeddings are analogous to a hidden layer?
  
  **Tags:** neural-networks, feedforward-neural-networks, graph-neural-networks, embeddings
  > **Comments:**
  > **username_1**: In my understanding embedding is a static vector, but hidden layer is a function. But this is a good question as hidden layer is indeed extracting a feature, while node embedding represents features of the node.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-03-12. Score: 1 | Accepted answer
  
  Embeddings are vectors. Layers are functions.
  
  So, node embeddings (e.g. produced by [TransE](https://proceedings.neurips.cc/paper/2013/file/1cecc7a77928ca8133fa24680a88d2f9-Paper.pdf)) are analogous to [word embeddings](https://proceedings.neurips.cc/paper/2013/file/9aa42b31882ec039965f3c4923ce901b-Paper.pdf) or [code embeddings](https://arxiv.org/pdf/1803.09473.pdf), i.e. they are vector (and lower-dimensional) representations of the original objects, such that similar objects have similar embeddings, for some notion of similarity.
  
  Your question is analogous to "Are convolutional layers analogous to feature maps?". So, you could use neural networks with layers to learn embeddings, but this does not have to be the case. For example, TransE does not "really" use a neural network (i.e. a sequence of layers that compute a non-linear function of the inputs), but it just optimizes the $L\_2$ norm of a simple function of the vector representation of the nodes and edges, although it uses gradient descent.
  
  > **Comments:**
  > **username_0**: makes so much sense, so that means embeddings are linked to model they are learned with? I thought embeddings are more universal as a space that separate two dissimilar things but here it appears that to use the embedding we need, the network on which it has been learned on, as well  
  
  ---
  
  