Answers.Body: <p>GraphSage does not have attention at all. Yes, it randomly samples (not  most important as you claim) a subset of neighbors, but it does not compute attention score for each neighbor.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-13T15:34:25.990Z
Answers.LastActivityDate: 2020-03-13T15:34:25.990Z
Answers.UserDisplayName: kkonevets
Body: <p>I was reading the recent paper <a href="https://arxiv.org/abs/1907.04652" rel="nofollow noreferrer">Graph Representation Learning via Hard and Channel-Wise Attention Networks</a>, where the authors claim that there is <em>no hard attention operator for graph data</em>. </p>
  
  <p>From my understanding, the difference between hard and soft attention is that for soft attention you're computing the attention scores between the nodes and all their neighbors while for hard attention you have a sampling function that selects only the most important neighbors. If that is the case, then <a href="https://arxiv.org/abs/1907.04652" rel="nofollow noreferrer">GraphSage</a> is an example of hard attention, because they apply the attention only on a subset of each node's neighbors. </p>
  
  <p>Is my understanding of hard and soft attention wrong, or the claim that the authors made does not hold?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-06T23:40:13.227Z
Comments.Text: If you now know the answer to this question, I suggest that you review the answer below and accept it if it answers the question. If not, feel free to leave an answer to your own question and accept it. Then please flag this comment as "no longer needed".
Comments.UserDisplayName: nbro
Tags: <comparison><geometric-deep-learning><graphs><attention>
Title: Does GraphSage use hard attention?
UserDisplayName: razvanc92
ThreadText: # Does GraphSage use hard attention?
  
  (Asked by: username_0 on 2019-07-18)
  
  I was reading the recent paper [Graph Representation Learning via Hard and Channel-Wise Attention Networks](https://arxiv.org/abs/1907.04652), where the authors claim that there is *no hard attention operator for graph data*.
  
  From my understanding, the difference between hard and soft attention is that for soft attention you're computing the attention scores between the nodes and all their neighbors while for hard attention you have a sampling function that selects only the most important neighbors. If that is the case, then [GraphSage](https://arxiv.org/abs/1907.04652) is an example of hard attention, because they apply the attention only on a subset of each node's neighbors.
  
  Is my understanding of hard and soft attention wrong, or the claim that the authors made does not hold?
  
  **Tags:** comparison, geometric-deep-learning, graphs, attention
  
  > **Comments:**
  > **username_1**: If you now know the answer to this question, I suggest that you review the answer below and accept it if it answers the question. If not, feel free to leave an answer to your own question and accept it. Then please flag this comment as "no longer needed".  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-03-13. Score: 1
  
  GraphSage does not have attention at all. Yes, it randomly samples (not most important as you claim) a subset of neighbors, but it does not compute attention score for each neighbor.
  
  
  ---
  
  