Answers.Body: <p>After some reflection I noticed that the actual final expression should contain derivatives of the alphas w.r.t. <span class="math-container">$A_{ij}^k$</span> too, because the alphas cannot be constants that do not depend on <span class="math-container">$A_{ij}^k$</span>.  So, the equation becomes:
  <span class="math-container">$$
  \frac{\partial Y^c}{\partial A_{ij}^k} = 
  \sum_{a,b} \alpha_{ab}^{kc} \cdot \frac{\partial Y^c}{\partial A_{ab}^k} +
  \sum_l \Bigl(\Bigl[\sum_{u,v} A_{uv}^l \Bigr] 
  \Bigl\{ \sum_{a,b} \alpha_{ab}^{lc} \cdot
  \frac{\partial^2 Y^c}{\partial A_{ij}^k \partial A_{ab}^l} \Bigr\} \Bigr) + 
  \sum_l \Bigl( \Bigl[\sum_{u,v} A_{uv}^l\Bigr] 
  \Bigl\{ \sum_{a,b} \frac{\partial \alpha_{ab}^{lc}}{\partial A_{ij}^k} \cdot \frac{\partial Y^c}{\partial A_{ab}^l} \Bigr\} \Bigr) 
  $$</span>
  More details in the appendix of <a href="https://sites.math.northwestern.edu/%7Emlerma/papers-and-preprints/Grad_CAM___is_equivalent_to_Grad_CAM_with_positive_gradients.pdf" rel="nofollow noreferrer">this document</a>.</p>
  <p>I believe this is the final answer to my question.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-24T19:58:34.567Z
Answers.LastActivityDate: 2022-05-24T19:58:34.567Z
Answers.UserDisplayName: mlerma54
Body: <p>I have a question about the <a href="https://arxiv.org/pdf/1710.11063.pdf" rel="nofollow noreferrer">Grad-CAM++ paper</a>. I do not understand how the following equation (10) for the alphas is obtained:
  <span class="math-container">$$
  \alpha_{ij}^{kc} =
  \frac{\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}}
  {2\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}
    + \sum_{ab} A_{ab}^k \{ \frac{\partial^3 Y^c}{(\partial A_{ij}^k)^3} \}}
  \qquad (10)
  $$</span>
  I found various issues with the derivation, here I will focus on the step from equation (7) to (8).</p>
  <p>Equation (7) without the relu function is:
  <span class="math-container">$$
      Y^c =
      \sum_k \Bigl( \Bigl\{ \sum_{a,b} \alpha_{ab}^{kc}
      \cdot \frac{\partial Y^c}{\partial A_{ab}^k} \Bigr\}
      \Bigl[ \sum_{i,j} A_{ij}^k \Bigr] \Bigr)
      \qquad (7)
  $$</span>
  Here <span class="math-container">$Y^c$</span> is supposed to be a function of the <span class="math-container">$A_{ij}^k$</span>, and <span class="math-container">$\alpha_{ab}^{kc}$</span> are unknowns to be determined.</p>
  <p>The next step consists of computing the partial derivative of (7) w.r.t. <span class="math-container">$A_{ij}^{kc}$</span>, which according to the paper yields equation (8):
  <span class="math-container">$$
    \frac{\partial Y^c}{\partial A_{jk}^k} = \sum_{a,b} \alpha_{ab}^{kc}
    \cdot \frac{\partial Y^c}{\partial A_{ab}^k} +
      \sum_{a,b} A_{ab}^k \Bigl\{ \alpha_{ij}^{kc}
      \cdot \frac{\partial^2 Y^c}{(\partial A_{jk}^k)^2} \Bigr\}
      \qquad (8)
  $$</span>
  However, when I did the computation myself I got the following:
  <span class="math-container">$$
    \frac{\partial Y^c}{\partial A_{ij}^k} =
      \sum_{a,b} \alpha_{ab}^{kc} \cdot \frac{\partial Y^c}{\partial A_{ab}^k} +
      \sum_l \Bigl(\Bigl[\sum_{u,v} A_{uv}^l \Bigr]
      \Bigl\{ \sum_{a,b} \alpha_{ab}^{lc} \cdot
      \frac{\partial^2 Y^c}{\partial A_{ij}^k \partial A_{ab}^l} \Bigr\} \Bigr)
      \qquad (8')
  $$</span>
  Note the extra sum, and the cross-derivatives.</p>
  <p>What is right, (8) or (8')?</p>
  
Tags: <deep-learning><papers><math><derivative><grad-cam++>
Title: What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?
UserDisplayName: mlerma54
ThreadText: # What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?
  
  (Asked by: username_0 on 2022-05-09)
  
  I have a question about the [Grad-CAM++ paper](https://arxiv.org/pdf/1710.11063.pdf). I do not understand how the following equation (10) for the alphas is obtained:
  $$
  \alpha\_{ij}^{kc} =
  \frac{\frac{\partial^2 Y^c}{(\partial A\_{ij}^k)^2}}
  {2\frac{\partial^2 Y^c}{(\partial A\_{ij}^k)^2}
  + \sum\_{ab} A\_{ab}^k \{ \frac{\partial^3 Y^c}{(\partial A\_{ij}^k)^3} \}}
  \qquad (10)
  $$
  I found various issues with the derivation, here I will focus on the step from equation (7) to (8).
  
  Equation (7) without the relu function is:
  $$
  Y^c =
  \sum\_k \Bigl( \Bigl\{ \sum\_{a,b} \alpha\_{ab}^{kc}
  \cdot \frac{\partial Y^c}{\partial A\_{ab}^k} \Bigr\}
  \Bigl[ \sum\_{i,j} A\_{ij}^k \Bigr] \Bigr)
  \qquad (7)
  $$
  Here $Y^c$ is supposed to be a function of the $A\_{ij}^k$, and $\alpha\_{ab}^{kc}$ are unknowns to be determined.
  
  The next step consists of computing the partial derivative of (7) w.r.t. $A\_{ij}^{kc}$, which according to the paper yields equation (8):
  $$
  \frac{\partial Y^c}{\partial A\_{jk}^k} = \sum\_{a,b} \alpha\_{ab}^{kc}
  \cdot \frac{\partial Y^c}{\partial A\_{ab}^k} +
  \sum\_{a,b} A\_{ab}^k \Bigl\{ \alpha\_{ij}^{kc}
  \cdot \frac{\partial^2 Y^c}{(\partial A\_{jk}^k)^2} \Bigr\}
  \qquad (8)
  $$
  However, when I did the computation myself I got the following:
  $$
  \frac{\partial Y^c}{\partial A\_{ij}^k} =
  \sum\_{a,b} \alpha\_{ab}^{kc} \cdot \frac{\partial Y^c}{\partial A\_{ab}^k} +
  \sum\_l \Bigl(\Bigl[\sum\_{u,v} A\_{uv}^l \Bigr]
  \Bigl\{ \sum\_{a,b} \alpha\_{ab}^{lc} \cdot
  \frac{\partial^2 Y^c}{\partial A\_{ij}^k \partial A\_{ab}^l} \Bigr\} \Bigr)
  \qquad (8')
  $$
  Note the extra sum, and the cross-derivatives.
  
  What is right, (8) or (8')?
  
  **Tags:** deep-learning, papers, math, derivative, grad-cam++
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2022-05-24. Score: 1 | Accepted answer
  
  After some reflection I noticed that the actual final expression should contain derivatives of the alphas w.r.t. $A\_{ij}^k$ too, because the alphas cannot be constants that do not depend on $A\_{ij}^k$. So, the equation becomes:
  $$
  \frac{\partial Y^c}{\partial A\_{ij}^k} =
  \sum\_{a,b} \alpha\_{ab}^{kc} \cdot \frac{\partial Y^c}{\partial A\_{ab}^k} +
  \sum\_l \Bigl(\Bigl[\sum\_{u,v} A\_{uv}^l \Bigr]
  \Bigl\{ \sum\_{a,b} \alpha\_{ab}^{lc} \cdot
  \frac{\partial^2 Y^c}{\partial A\_{ij}^k \partial A\_{ab}^l} \Bigr\} \Bigr) +
  \sum\_l \Bigl( \Bigl[\sum\_{u,v} A\_{uv}^l\Bigr]
  \Bigl\{ \sum\_{a,b} \frac{\partial \alpha\_{ab}^{lc}}{\partial A\_{ij}^k} \cdot \frac{\partial Y^c}{\partial A\_{ab}^l} \Bigr\} \Bigr)
  $$
  More details in the appendix of [this document](https://sites.math.northwestern.edu/%7Emlerma/papers-and-preprints/Grad_CAM___is_equivalent_to_Grad_CAM_with_positive_gradients.pdf).
  
  I believe this is the final answer to my question.
  
  
  ---
  
  