Answers.Body: <p>So I mailed the authors, and they were kind enough to reply.</p>
  <p>The reasoning behind using gradients instead of matrix multiplications is the convenience we get in calculating gradients. The convenience is not much if you are just dealing with linear layers. For example: consider the following code</p>
  <pre><code>
  import torch
  torch.manual_seed(123)
  
  
  fc1 = torch.nn.Linear(4,2,bias=False) # disabling bias is important
  i = torch.rand(4, requires_grad=True) # activations from previous layer aLminus1
  o = fc1.forward(i) # notice this is w*aLminus1 without activation
  out = torch.relu(o) # this is activation of aL layer, which is also our output layer, hence they become Relevance values.
  print('activation j',i)
  print('forward k',o)
  print('activation k',out)
  print('w_jk',fc1.weight)
  print('bias_k',fc1.bias)
  print()
  
  w = fc1.weight.data
  Rk = aL = out.data
  aLminus1 = i.data
  
  z = aLminus1.unsqueeze(0).numpy().dot(w.T.numpy())
  s = Rk/z
  c = s.numpy().dot(w.numpy())
  print(&quot;With matrix multiplication&quot;)
  print(c)
  
  z = 1e-9 + o
  s = Rk/(z+1e-9)
  (z*s.data).sum().backward()
  c = i.grad
  print('With gradients method')
  print(c)
  </code></pre>
  <p>You will see that both the method gives the same answer. But the real convenience is seen when you are dealing with convolutional layers. Without the gradient method, you will need to write an iterative code to pass the relevance values from kth layer to jth layer because <code>c = s.numpy().dot(w.numpy())</code> will no longer hold for convolutional layers since shape of w will be <code>[in_maps, filter_size, filter_size]</code>.</p>
  <p>Okay but then why this gradient method works for linear layers? Well the derivation is quite simple.</p>
  <p>target: <code>c_i = w @ s</code> ; where <code>shape(w) = [ j , k ]</code> , <code>shape(s) = [ k , 1 ]</code>, and @ is matrix multiplication
  let us do this for simplicity, <code>w = rho(w)</code></p>
  <ol>
  <li>we know that <code>z = w.forward(a)</code> which is equivalent to <code>w.T @ a</code> where <code>shape(a) = [ j , 1 ]</code>, hence <code>shape(z) = [ k , 1 ]</code></li>
  <li><code>s = Rk/z</code> where <code>shape(Rk) = [ k , 1 ]</code> and hence <code>shape(s) = [ k , 1 ]</code></li>
  <li>Now if we use gradients, then <code>temp = z*s.data</code> is equivalent to <code>( w.T @ a ) * s.data</code> where * is element-wise multiplication and <code>shape(temp) = [ k , 1 ]</code></li>
  <li>when this expression is summed and gradients are back-propagated using <code>.backward()</code> The gradients will be attached to w and a both</li>
  <li><code>a.grad</code> will be the differentiation of temp with respect to a and treating others as constant</li>
  <li>hence <code>a.grad</code> will be <code>w @ s</code> why? For example: if <code>f = x*y*z</code> then <code>df/dx = y*z</code> i.e. <code>d(temp)/da = w @ s</code>  it is easy to observe this on pen-paper, just put 1 in all the values of a in temp, and we will get <code>w @ s</code></li>
  <li>Hence <code>c_i = a.grad = w @ s</code></li>
  <li>The reason why <code>s.data</code> is used instead of simply <code>s</code> is because, if we use simply <code>s</code> then gradients will find a new path to flow into a because <code>s</code> is dependent on <code>a</code>.</li>
  </ol>
  <p>However, be mindful of one small thing. If you are trying to see for yourself whether gradient based method and matrix multiplication method gives the same values of <span class="math-container">$c_i$</span>, then make sure your linear layers have bias set to <code>False</code>. Otherwise both the methods will give different values to <span class="math-container">$c_i$</span>. Why? Bias changes the way how you should calculate <span class="math-container">$z_{jk}$</span>, hence the derivation changes.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-15T11:13:13.133Z
Answers.Text: Can I ask you a question? Why S is constant? Why the .data? Why s should be independent of a? I see it like a trick, but i would like to know the key explanation. Thanks!
Answers.UserDisplayName: Pedro Delgado De Robles
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-02-17T07:01:32.483Z
Answers.Text: Read point number 8 above. It will surely answer your questions.
Answers.UserDisplayName: Ritwik
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-24T14:58:30.983Z
Answers.LastActivityDate: 2022-05-24T14:58:30.983Z
Answers.UserDisplayName: Ritwik
Body: <p>To give you an overview, Layer-wise Relevance Propagation is a technique by which we can get relevance values at each node of the neural network. These calculated relevance values (per node) are representative of the importance that that node plays, in deciding the predicted output. The final values at the output layer are considered the relevance values of those nodes, then the relevance values are calculated in a downward manner i.e. from output layer to input layer.</p>
  <p><a href="https://i.sstatic.net/SnaLI.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/SnaLI.png" alt="LRP" /></a>
  <a href="https://youtu.be/PDRewtcqmaI" rel="nofollow noreferrer">Image Source</a></p>
  <p>Some fundamental equations for calculating LRP are as follows:
  <a href="https://i.sstatic.net/UlPAz.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/UlPAz.png" alt="LRP_equations" /></a></p>
  <p>For the sake of simplicity, you can imagine the function <span class="math-container">$\rho(w_{jk})$</span> as an identity function i.e. <span class="math-container">$\rho(w_{jk}) = w_{jk}$</span></p>
  <p><a href="https://link.springer.com/chapter/10.1007/978-3-030-28954-6_10" rel="nofollow noreferrer">This paper</a> just mentions that <span class="math-container">$c_j$</span> can be computed using gradients on the activations of lower-layer (the one which closer to the input layer), but why? What is the reasoning behind that?</p>
  <p>I have mailed the authors asking these same questions. I will write an answer to this when they respond.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-19T15:50:41.160Z
Comments.Text: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.
Comments.UserDisplayName: Community
Tags: <neural-networks><deep-neural-networks><explainable-ai>
Title: Why gradients are used in Layer-wise Relevance Propagation (LRP)?
UserDisplayName: Ritwik
ThreadText: # Why gradients are used in Layer-wise Relevance Propagation (LRP)?
  
  (Asked by: username_0 on 2022-05-19)
  
  To give you an overview, Layer-wise Relevance Propagation is a technique by which we can get relevance values at each node of the neural network. These calculated relevance values (per node) are representative of the importance that that node plays, in deciding the predicted output. The final values at the output layer are considered the relevance values of those nodes, then the relevance values are calculated in a downward manner i.e. from output layer to input layer.
  
  [![LRP](https://i.sstatic.net/SnaLI.png)](https://i.sstatic.net/SnaLI.png)
  [Image Source](https://youtu.be/PDRewtcqmaI)
  
  Some fundamental equations for calculating LRP are as follows:
  [![LRP_equations](https://i.sstatic.net/UlPAz.png)](https://i.sstatic.net/UlPAz.png)
  
  For the sake of simplicity, you can imagine the function $\rho(w\_{jk})$ as an identity function i.e. $\rho(w\_{jk}) = w\_{jk}$
  
  [This paper](https://link.springer.com/chapter/10.1007/978-3-030-28954-6_10) just mentions that $c\_j$ can be computed using gradients on the activations of lower-layer (the one which closer to the input layer), but why? What is the reasoning behind that?
  
  I have mailed the authors asking these same questions. I will write an answer to this when they respond.
  
  **Tags:** neural-networks, deep-neural-networks, explainable-ai
  
  > **Comments:**
  > **username_1**: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2022-05-24. Score: 2 | Accepted answer
  
  So I mailed the authors, and they were kind enough to reply.
  
  The reasoning behind using gradients instead of matrix multiplications is the convenience we get in calculating gradients. The convenience is not much if you are just dealing with linear layers. For example: consider the following code
  
  ```
  
  import torch
  torch.manual_seed(123)
  
  fc1 = torch.nn.Linear(4,2,bias=False) # disabling bias is important
  i = torch.rand(4, requires_grad=True) # activations from previous layer aLminus1
  o = fc1.forward(i) # notice this is w*aLminus1 without activation
  out = torch.relu(o) # this is activation of aL layer, which is also our output layer, hence they become Relevance values.
  print('activation j',i)
  print('forward k',o)
  print('activation k',out)
  print('w_jk',fc1.weight)
  print('bias_k',fc1.bias)
  print()
  
  w = fc1.weight.data
  Rk = aL = out.data
  aLminus1 = i.data
  
  z = aLminus1.unsqueeze(0).numpy().dot(w.T.numpy())
  s = Rk/z
  c = s.numpy().dot(w.numpy())
  print("With matrix multiplication")
  print(c)
  
  z = 1e-9 + o
  s = Rk/(z+1e-9)
  (z*s.data).sum().backward()
  c = i.grad
  print('With gradients method')
  print(c)
  
  ```
  
  You will see that both the method gives the same answer. But the real convenience is seen when you are dealing with convolutional layers. Without the gradient method, you will need to write an iterative code to pass the relevance values from kth layer to jth layer because `c = s.numpy().dot(w.numpy())` will no longer hold for convolutional layers since shape of w will be `[in_maps, filter_size, filter_size]`.
  
  Okay but then why this gradient method works for linear layers? Well the derivation is quite simple.
  
  target: `c_i = w @ s` ; where `shape(w) = [ j , k ]` , `shape(s) = [ k , 1 ]`, and @ is matrix multiplication
  let us do this for simplicity, `w = rho(w)`
  
  1. we know that `z = w.forward(a)` which is equivalent to `w.T @ a` where `shape(a) = [ j , 1 ]`, hence `shape(z) = [ k , 1 ]`
  2. `s = Rk/z` where `shape(Rk) = [ k , 1 ]` and hence `shape(s) = [ k , 1 ]`
  3. Now if we use gradients, then `temp = z*s.data` is equivalent to `( w.T @ a ) * s.data` where \* is element-wise multiplication and `shape(temp) = [ k , 1 ]`
  4. when this expression is summed and gradients are back-propagated using `.backward()` The gradients will be attached to w and a both
  5. `a.grad` will be the differentiation of temp with respect to a and treating others as constant
  6. hence `a.grad` will be `w @ s` why? For example: if `f = x*y*z` then `df/dx = y*z` i.e. `d(temp)/da = w @ s` it is easy to observe this on pen-paper, just put 1 in all the values of a in temp, and we will get `w @ s`
  7. Hence `c_i = a.grad = w @ s`
  8. The reason why `s.data` is used instead of simply `s` is because, if we use simply `s` then gradients will find a new path to flow into a because `s` is dependent on `a`.
  
  However, be mindful of one small thing. If you are trying to see for yourself whether gradient based method and matrix multiplication method gives the same values of $c\_i$, then make sure your linear layers have bias set to `False`. Otherwise both the methods will give different values to $c\_i$. Why? Bias changes the way how you should calculate $z\_{jk}$, hence the derivation changes.
  
  > **Comments:**
  > **username_2**: Can I ask you a question? Why S is constant? Why the .data? Why s should be independent of a? I see it like a trick, but i would like to know the key explanation. Thanks!  
  > **username_0**: Read point number 8 above. It will surely answer your questions.  
  
  ---
  
  