<p>After some reflection I noticed that the actual final expression should contain derivatives of the alphas w.r.t. <span class="math-container">$A_{ij}^k$</span> too, because the alphas cannot be constants that do not depend on <span class="math-container">$A_{ij}^k$</span>.  So, the equation becomes:
<span class="math-container">$$
\frac{\partial Y^c}{\partial A_{ij}^k} = 
\sum_{a,b} \alpha_{ab}^{kc} \cdot \frac{\partial Y^c}{\partial A_{ab}^k} +
\sum_l \Bigl(\Bigl[\sum_{u,v} A_{uv}^l \Bigr] 
\Bigl\{ \sum_{a,b} \alpha_{ab}^{lc} \cdot
\frac{\partial^2 Y^c}{\partial A_{ij}^k \partial A_{ab}^l} \Bigr\} \Bigr) + 
\sum_l \Bigl( \Bigl[\sum_{u,v} A_{uv}^l\Bigr] 
\Bigl\{ \sum_{a,b} \frac{\partial \alpha_{ab}^{lc}}{\partial A_{ij}^k} \cdot \frac{\partial Y^c}{\partial A_{ab}^l} \Bigr\} \Bigr) 
$$</span>
More details in the appendix of <a href="https://sites.math.northwestern.edu/%7Emlerma/papers-and-preprints/Grad_CAM___is_equivalent_to_Grad_CAM_with_positive_gradients.pdf" rel="nofollow noreferrer">this document</a>.</p>
<p>I believe this is the final answer to my question.</p>

CC BY-SA 4.0
2022-05-24T19:58:34.567Z
2022-05-24T19:58:34.567Z
mlerma54
<p>I have a question about the <a href="https://arxiv.org/pdf/1710.11063.pdf" rel="nofollow noreferrer">Grad-CAM++ paper</a>. I do not understand how the following equation (10) for the alphas is obtained:
<span class="math-container">$$
\alpha_{ij}^{kc} =
\frac{\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}}
{2\frac{\partial^2 Y^c}{(\partial A_{ij}^k)^2}
  + \sum_{ab} A_{ab}^k \{ \frac{\partial^3 Y^c}{(\partial A_{ij}^k)^3} \}}
\qquad (10)
$$</span>
I found various issues with the derivation, here I will focus on the step from equation (7) to (8).</p>
<p>Equation (7) without the relu function is:
<span class="math-container">$$
    Y^c =
    \sum_k \Bigl( \Bigl\{ \sum_{a,b} \alpha_{ab}^{kc}
    \cdot \frac{\partial Y^c}{\partial A_{ab}^k} \Bigr\}
    \Bigl[ \sum_{i,j} A_{ij}^k \Bigr] \Bigr)
    \qquad (7)
$$</span>
Here <span class="math-container">$Y^c$</span> is supposed to be a function of the <span class="math-container">$A_{ij}^k$</span>, and <span class="math-container">$\alpha_{ab}^{kc}$</span> are unknowns to be determined.</p>
<p>The next step consists of computing the partial derivative of (7) w.r.t. <span class="math-container">$A_{ij}^{kc}$</span>, which according to the paper yields equation (8):
<span class="math-container">$$
  \frac{\partial Y^c}{\partial A_{jk}^k} = \sum_{a,b} \alpha_{ab}^{kc}
  \cdot \frac{\partial Y^c}{\partial A_{ab}^k} +
    \sum_{a,b} A_{ab}^k \Bigl\{ \alpha_{ij}^{kc}
    \cdot \frac{\partial^2 Y^c}{(\partial A_{jk}^k)^2} \Bigr\}
    \qquad (8)
$$</span>
However, when I did the computation myself I got the following:
<span class="math-container">$$
  \frac{\partial Y^c}{\partial A_{ij}^k} =
    \sum_{a,b} \alpha_{ab}^{kc} \cdot \frac{\partial Y^c}{\partial A_{ab}^k} +
    \sum_l \Bigl(\Bigl[\sum_{u,v} A_{uv}^l \Bigr]
    \Bigl\{ \sum_{a,b} \alpha_{ab}^{lc} \cdot
    \frac{\partial^2 Y^c}{\partial A_{ij}^k \partial A_{ab}^l} \Bigr\} \Bigr)
    \qquad (8')
$$</span>
Note the extra sum, and the cross-derivatives.</p>
<p>What is right, (8) or (8')?</p>

<deep-learning><papers><math><derivative><grad-cam++>
What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?
mlerma54
# What is the correct partial derivative of $Y^c$ with respect to $A_{ij}^{kc}$?

(Asked by: username_0 on 2022-05-09)

I have a question about the [Grad-CAM++ paper](https://arxiv.org/pdf/1710.11063.pdf). I do not understand how the following equation (10) for the alphas is obtained:
$$
\alpha\_{ij}^{kc} =
\frac{\frac{\partial^2 Y^c}{(\partial A\_{ij}^k)^2}}
{2\frac{\partial^2 Y^c}{(\partial A\_{ij}^k)^2}
+ \sum\_{ab} A\_{ab}^k \{ \frac{\partial^3 Y^c}{(\partial A\_{ij}^k)^3} \}}
\qquad (10)
$$
I found various issues with the derivation, here I will focus on the step from equation (7) to (8).

Equation (7) without the relu function is:
$$
Y^c =
\sum\_k \Bigl( \Bigl\{ \sum\_{a,b} \alpha\_{ab}^{kc}
\cdot \frac{\partial Y^c}{\partial A\_{ab}^k} \Bigr\}
\Bigl[ \sum\_{i,j} A\_{ij}^k \Bigr] \Bigr)
\qquad (7)
$$
Here $Y^c$ is supposed to be a function of the $A\_{ij}^k$, and $\alpha\_{ab}^{kc}$ are unknowns to be determined.

The next step consists of computing the partial derivative of (7) w.r.t. $A\_{ij}^{kc}$, which according to the paper yields equation (8):
$$
\frac{\partial Y^c}{\partial A\_{jk}^k} = \sum\_{a,b} \alpha\_{ab}^{kc}
\cdot \frac{\partial Y^c}{\partial A\_{ab}^k} +
\sum\_{a,b} A\_{ab}^k \Bigl\{ \alpha\_{ij}^{kc}
\cdot \frac{\partial^2 Y^c}{(\partial A\_{jk}^k)^2} \Bigr\}
\qquad (8)
$$
However, when I did the computation myself I got the following:
$$
\frac{\partial Y^c}{\partial A\_{ij}^k} =
\sum\_{a,b} \alpha\_{ab}^{kc} \cdot \frac{\partial Y^c}{\partial A\_{ab}^k} +
\sum\_l \Bigl(\Bigl[\sum\_{u,v} A\_{uv}^l \Bigr]
\Bigl\{ \sum\_{a,b} \alpha\_{ab}^{lc} \cdot
\frac{\partial^2 Y^c}{\partial A\_{ij}^k \partial A\_{ab}^l} \Bigr\} \Bigr)
\qquad (8')
$$
Note the extra sum, and the cross-derivatives.

What is right, (8) or (8')?

**Tags:** deep-learning, papers, math, derivative, grad-cam++


---

## 1 answers

### Answer by username_0 on 2022-05-24. Score: 1 | Accepted answer

After some reflection I noticed that the actual final expression should contain derivatives of the alphas w.r.t. $A\_{ij}^k$ too, because the alphas cannot be constants that do not depend on $A\_{ij}^k$. So, the equation becomes:
$$
\frac{\partial Y^c}{\partial A\_{ij}^k} =
\sum\_{a,b} \alpha\_{ab}^{kc} \cdot \frac{\partial Y^c}{\partial A\_{ab}^k} +
\sum\_l \Bigl(\Bigl[\sum\_{u,v} A\_{uv}^l \Bigr]
\Bigl\{ \sum\_{a,b} \alpha\_{ab}^{lc} \cdot
\frac{\partial^2 Y^c}{\partial A\_{ij}^k \partial A\_{ab}^l} \Bigr\} \Bigr) +
\sum\_l \Bigl( \Bigl[\sum\_{u,v} A\_{uv}^l\Bigr]
\Bigl\{ \sum\_{a,b} \frac{\partial \alpha\_{ab}^{lc}}{\partial A\_{ij}^k} \cdot \frac{\partial Y^c}{\partial A\_{ab}^l} \Bigr\} \Bigr)
$$
More details in the appendix of [this document](https://sites.math.northwestern.edu/%7Emlerma/papers-and-preprints/Grad_CAM___is_equivalent_to_Grad_CAM_with_positive_gradients.pdf).

I believe this is the final answer to my question.


---

