Body: <p>I have a transform-encoder only architecture, which has the following structure:</p>
  <pre><code>      Input
          |
          v
       ResNet(-50)
          |
          v
  fully-connected (on embedding dimension)
          |
          v
  positional-encoding
          |
          v
  transformer encoder
          |
          v
  Linear layer to alphabet.
  </code></pre>
  <p>I am trying to visualize the self-attention of the encoder layer to check how each input of the attention attends other inputs. (E.g. <a href="https://github.com/jessevig/bertviz" rel="nofollow noreferrer">https://github.com/jessevig/bertviz</a>)</p>
  <p>Where I encounter difficulty is in how I can visualize these activations in terms of the original input of the ResNet and not its output, in order to make my model visually interpretable.</p>
  <p>Do you have any ideas or suggestions?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-01T09:05:23.523Z
Comments.Text: Hello. Is this just a programming issue or are you asking to conceptually solve your problem? This is not clear from your post.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-01T14:35:44.317Z
Comments.Text: My question is conceptual but I plan to implement it.
Comments.UserDisplayName: John Sig
Tags: <pytorch><transformer><attention><data-visualization>
Title: Visualizing encoder-attention after ResNet in terms of ResNet input
UserDisplayName: John Sig
ThreadText: # Visualizing encoder-attention after ResNet in terms of ResNet input
  
  (Asked by: username_0 on 2021-05-31)
  
  I have a transform-encoder only architecture, which has the following structure:
  
  ```
        Input
          |
          v
       ResNet(-50)
          |
          v
  fully-connected (on embedding dimension)
          |
          v
  positional-encoding
          |
          v
  transformer encoder
          |
          v
  Linear layer to alphabet.
  
  ```
  
  I am trying to visualize the self-attention of the encoder layer to check how each input of the attention attends other inputs. (E.g. <https://github.com/jessevig/bertviz>)
  
  Where I encounter difficulty is in how I can visualize these activations in terms of the original input of the ResNet and not its output, in order to make my model visually interpretable.
  
  Do you have any ideas or suggestions?
  
  **Tags:** pytorch, transformer, attention, data-visualization
  
  > **Comments:**
  > **username_1**: Hello. Is this just a programming issue or are you asking to conceptually solve your problem? This is not clear from your post.  
  > **username_0**: My question is conceptual but I plan to implement it.  
  
  ---
  
  ## 0 answers
  
  