Answers.Body: <p>Equation (7) in the Grad-CAM++ paper is linear.  In fact for a given class <span class="math-container">$c$</span> we have just one equation and many unknowns (the <span class="math-container">$\alpha_{ab}^{kc}$</span>), hence the equation is underdetermined and will have infinitely many solutions. The general solution will consist of picking an <span class="math-container">$\alpha_{ab}^{kc}$</span> with non zero coefficient, and assigning arbitrary values to the other alphas. Then, the <span class="math-container">$\alpha_{ab}^{kc}$</span> picked can be written as a linear combination of the other alphas.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-11T16:59:57.260Z
Answers.LastActivityDate: 2022-05-11T16:59:57.260Z
Answers.UserDisplayName: mlerma54
Body: <p>In the <a href="https://arxiv.org/pdf/1710.11063.pdf" rel="nofollow noreferrer">Grad-CAM++ paper</a> the following equation (7) is posed (written here without the relu function):
  <span class="math-container">$$
      Y^c =
      \sum_k \Bigl( \Bigl\{ \sum_{a,b} \alpha_{ab}^{kc}
      \cdot \frac{\partial Y^c}{\partial A_{ab}^k} \Bigr\}
      \Bigl[ \sum_{i,j} A_{ij}^k \Bigr] \Bigr)
      \qquad (7)
  $$</span>
  <span class="math-container">$Y^c$</span> is supposed to be a function of the <span class="math-container">$A_{ij}^k$</span>, and <span class="math-container">$\alpha_{ab}^{kc}$</span> are unknowns to be determined.</p>
  <p>In order to isolate the alphas the paper computes partial derivatives w.r.t. <span class="math-container">$A_{ij}^k$</span> on both sides of the equation.  In the computation no partial derivatives of the alphas with respect to <span class="math-container">$A_{ij}^k$</span> occur, so it seems that the alphas are considered independent from the <span class="math-container">$A_{ij}^k$</span>. However this does not look right.  To illustrate the problem assume that we want to find <span class="math-container">$\alpha$</span> in the equation <span class="math-container">$\alpha x = x^2$</span>. If we follow the method used in the paper, i.e. differentiate both sides of the equation w.r.t. <span class="math-container">$x$</span> while assuming that <span class="math-container">$\alpha$</span> does not depend on <span class="math-container">$x$</span> (hence <span class="math-container">$d\alpha/dx = 0$</span>), we will get <span class="math-container">$\alpha = 2x$</span>, which is clearly incorrect. In fact differentiating both sides with respect to <span class="math-container">$x$</span> yields <span class="math-container">$\frac{d\alpha}{dx} x + \alpha = 2x$</span>, which is not going to help isolating <span class="math-container">$\alpha$</span>. The actual solution in this case is better obtained by dividing both sides by <span class="math-container">$x$</span>, so <span class="math-container">$\alpha = x^2/x = x$</span>, <span class="math-container">$x\neq 0$</span>.</p>
  <p>So, what it would be a right way to obtain alphas satisfying equation (7)?</p>
  
Tags: <deep-learning><papers><math><grad-cam++>
Title: What is the right way to find the alphas in this equation?
UserDisplayName: mlerma54
ThreadText: # What is the right way to find the alphas in this equation?
  
  (Asked by: username_0 on 2022-05-11)
  
  In the [Grad-CAM++ paper](https://arxiv.org/pdf/1710.11063.pdf) the following equation (7) is posed (written here without the relu function):
  $$
  Y^c =
  \sum\_k \Bigl( \Bigl\{ \sum\_{a,b} \alpha\_{ab}^{kc}
  \cdot \frac{\partial Y^c}{\partial A\_{ab}^k} \Bigr\}
  \Bigl[ \sum\_{i,j} A\_{ij}^k \Bigr] \Bigr)
  \qquad (7)
  $$
  $Y^c$ is supposed to be a function of the $A\_{ij}^k$, and $\alpha\_{ab}^{kc}$ are unknowns to be determined.
  
  In order to isolate the alphas the paper computes partial derivatives w.r.t. $A\_{ij}^k$ on both sides of the equation. In the computation no partial derivatives of the alphas with respect to $A\_{ij}^k$ occur, so it seems that the alphas are considered independent from the $A\_{ij}^k$. However this does not look right. To illustrate the problem assume that we want to find $\alpha$ in the equation $\alpha x = x^2$. If we follow the method used in the paper, i.e. differentiate both sides of the equation w.r.t. $x$ while assuming that $\alpha$ does not depend on $x$ (hence $d\alpha/dx = 0$), we will get $\alpha = 2x$, which is clearly incorrect. In fact differentiating both sides with respect to $x$ yields $\frac{d\alpha}{dx} x + \alpha = 2x$, which is not going to help isolating $\alpha$. The actual solution in this case is better obtained by dividing both sides by $x$, so $\alpha = x^2/x = x$, $x\neq 0$.
  
  So, what it would be a right way to obtain alphas satisfying equation (7)?
  
  **Tags:** deep-learning, papers, math, grad-cam++
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2022-05-11. Score: 1
  
  Equation (7) in the Grad-CAM++ paper is linear. In fact for a given class $c$ we have just one equation and many unknowns (the $\alpha\_{ab}^{kc}$), hence the equation is underdetermined and will have infinitely many solutions. The general solution will consist of picking an $\alpha\_{ab}^{kc}$ with non zero coefficient, and assigning arbitrary values to the other alphas. Then, the $\alpha\_{ab}^{kc}$ picked can be written as a linear combination of the other alphas.
  
  
  ---
  
  