<p>I was looking at two papers </p>

<ul>
<li><a href="https://arxiv.org/pdf/1710.10903.pdf" rel="nofollow noreferrer">Graph Attention Networks</a> (GAT) by <em>Petar Veličković</em> and</li>
<li><a href="https://arxiv.org/pdf/1803.07294.pdf" rel="nofollow noreferrer">GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs</a> by <em>Jiani Zhang</em>.</li>
</ul>

<p>I'm trying to implement the second paper and I'm having some troubles understanding the differences between GAT and GaAN. By looking at equation 1 in GaAN paper I can see only two differences from GAT. </p>

<ul>
<li>The first difference is that they are doing a dot product with the initial feature map and </li>
<li>Have another fully connected layer to project the result. </li>
</ul>

<p>Is there something else that I'm missing?</p>

CC BY-SA 4.0
2019-06-18T13:10:22.527Z
wow the stuff is realy hardcore. About (1), looks like they are implementing it differently but still maintaing the structure, so its basicly the same. And (2) fc-layer is not important.  
But did you understand what do thay do further with input? to put a graph of any length they must do some aggregation over variable length and i
user8426627
CC BY-SA 4.0
2019-06-21T13:48:36.273Z
Regarding point 1 in the GaAN paper they are stating: "The difference
between our aggregator and that in GAT is that we have adopted the key-value attention mechanism and the dot product attention while GAT does not compute additional value vectors and uses a fully-connected layer". From my understanding those two implementations are quite different. Lets try to break it down. Lets say we have 2 nodes i,j where each have an embedding size of 3. When we are computing the GAT aggregator we combine the two nodes embeddings and feed them to a single ff neural net so we are learning 6 parameters.
razvanc92
CC BY-SA 4.0
2019-06-21T13:55:56.433Z
When computing the GaAN aggregator we define two weight matrices of size [3,3]. We than multiply node i with the first one and node j with the second one and have two matrices of size [1,3]. Afterwards we can use the dot product between them so that we only have 1 (the attention score). Now looking at the number of parameters GAT uses 6 vs GaAN 18 I think we can say that the two implementations are not similar. Please correct me if I am wrong.
razvanc92
CC BY-SA 4.0
2020-10-06T08:07:27.207Z
Did you completed the implementation of GaAN? I have found the implementation of GaAN in here https://github.com/jennyzhang0215/GaAN, but environment setup for that implementation is quite hardly, so I haven't been able to run it yet!
phamduy
<neural-networks><deep-learning><comparison><attention><geometric-deep-learning>
What is the difference between GAT and GaAN?
razvanc92
# What is the difference between GAT and GaAN?

(Asked by: username_0 on 2019-06-18)

I was looking at two papers

* [Graph Attention Networks](https://arxiv.org/pdf/1710.10903.pdf) (GAT) by *Petar Veličković* and
* [GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs](https://arxiv.org/pdf/1803.07294.pdf) by *Jiani Zhang*.

I'm trying to implement the second paper and I'm having some troubles understanding the differences between GAT and GaAN. By looking at equation 1 in GaAN paper I can see only two differences from GAT.

* The first difference is that they are doing a dot product with the initial feature map and
* Have another fully connected layer to project the result.

Is there something else that I'm missing?

**Tags:** neural-networks, deep-learning, comparison, attention, geometric-deep-learning

> **Comments:**
> **username_1**: wow the stuff is realy hardcore. About (1), looks like they are implementing it differently but still maintaing the structure, so its basicly the same. And (2) fc-layer is not important.   But did you understand what do thay do further with input? to put a graph of any length they must do some aggregation over variable length and i  
> **username_0**: Regarding point 1 in the GaAN paper they are stating: "The difference between our aggregator and that in GAT is that we have adopted the key-value attention mechanism and the dot product attention while GAT does not compute additional value vectors and uses a fully-connected layer". From my understanding those two implementations are quite different. Lets try to break it down. Lets say we have 2 nodes i,j where each have an embedding size of 3. When we are computing the GAT aggregator we combine the two nodes embeddings and feed them to a single ff neural net so we are learning 6 parameters.  
> **username_0**: When computing the GaAN aggregator we define two weight matrices of size [3,3]. We than multiply node i with the first one and node j with the second one and have two matrices of size [1,3]. Afterwards we can use the dot product between them so that we only have 1 (the attention score). Now looking at the number of parameters GAT uses 6 vs GaAN 18 I think we can say that the two implementations are not similar. Please correct me if I am wrong.  
> **username_2**: Did you completed the implementation of GaAN? I have found the implementation of GaAN in here https://github.com/jennyzhang0215/GaAN, but environment setup for that implementation is quite hardly, so I haven't been able to run it yet!  

---

## 0 answers

