<p>While this may not be the answer you were looking for, I hope this explanation will help you to understand applying backpropagation to a CNN. Fundamentally, convolutional layers are no different than dense layers, however there are restrictions. The key one is weight-sharing which allows a CNN to be much more efficient than a regular dense layer (as well as it being sparse due to locality). Imagine we are transforming a 4x4 image into a 2x2 image. Since we are inputting a 16-vector, and outputting a 4-vector, we need a weights matrix of 4x16:</p>

<p><a href="https://i.sstatic.net/YHJad.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/YHJad.png" alt="enter image description here"></a></p>

<p>This has 64 parameters. In a convolutional layer, we can accomplish this by convolving a 3x3 kernel over the image:</p>

<p><span class="math-container">$$ K=
\begin{bmatrix}
k_{1,1} &amp; k_{1,2} &amp; k_{1,3} \\
k_{2,1} &amp; k_{2,2} &amp; k_{2,3} \\
k_{3,1} &amp; k_{3,2} &amp; k_{3,3}
\end{bmatrix} $$</span></p>

<p>This convolution is equivalent to multiplying by the weights matrix:</p>

<p><a href="https://i.sstatic.net/jyY0V.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jyY0V.png" alt="enter image description here"></a></p>

<p>As you can see, this only requires 9 parameters and backpropagation can be applied to update these parameters.</p>

<p>Image Source: <a href="https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1" rel="nofollow noreferrer">https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1</a></p>

CC BY-SA 4.0
2020-05-27T08:15:45.740Z
thanks but as you said it is not my question's answer.
I finally find $dL/dX$ for each channel in my example ``16`` then I add all of them and 
I get my $dL/dX$
A.Najafi
CC BY-SA 4.0
2020-05-22T12:20:48.813Z
2020-05-22T12:20:48.813Z
Archie Shahidullah
<p>I am trying to implement CNN using python NumPy. I searched so much, but all I found was for one filter with one channel for convolution.</p>
<p>Suppose <span class="math-container">$x$</span> is an image with the shape: <code>(N_Height, N_Width, N_Channel) = (5,5,3)</code>.</p>
<p>Let's say I have <code>16</code> filters with this shape: <code>(F_Height, F_Width, N_Channel) = (3,3,3)</code> , <code>stride=1</code> and <code>padding=0</code></p>
<p><strong>Forward</strong>:</p>
<p>The output shape after convolution 2d will be</p>
<pre><code>(
math.floor((N_Height - F_Height + 2*padding)/stride + 1 )),
math.floor((N_Width- F_Width + 2*padding)/stride + 1 )),
filter_count
)
</code></pre>
<p>So, the output of this layer will be an array with this shape: <code>(Height, Width, Channel) =  (3, 3, 16)</code></p>
<p><strong>BackPropagation</strong>:</p>
<p>Suppose <span class="math-container">$dL/dh$</span> is the input for my layer in back-propagation with this shape: <code>(3, 3, 16)</code></p>
<p>Now, I must find <span class="math-container">$dL/dw$</span> and <span class="math-container">$dL/dx$</span>: <span class="math-container">$dL/dw$</span> to update my filters parameter and <span class="math-container">$dL/dx$</span> to pass it as input to the previous layer as the loss respect to the input <span class="math-container">$x$</span>.</p>
<p>From this answer <a href="https://datascience.stackexchange.com/a/45978/89170">Error respect to filters weights</a> I found how to calculate <span class="math-container">$dL/dw$</span>.</p>
<p>The problem I have in the back-propagation is I don't know how to calculate <span class="math-container">$dL/dx$</span> having this shape: <code>(5, 5, 3)</code> and pass it to the previous layer.</p>
<p>I read lots of articles in Medium and other sites, but I don't get how to calculate it:</p>
<ul>
<li><a href="https://medium.com/@pavisj/convolutions-and-backpropagations-46026a8f5d2c" rel="nofollow noreferrer">How Backpropagation works in a CNN</a></li>
<li><a href="https://medium.com/technologymadeeasy/the-best-explanation-of-convolutional-neural-networks-on-the-internet-fbb8b1ad5df8" rel="nofollow noreferrer">The best explanation of Convolutional Neural Networks on the Internet!</a></li>
<li><a href="https://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/" rel="nofollow noreferrer">Backpropagation In Convolutional Neural Networks</a></li>
<li><a href="https://datascience.stackexchange.com/questions/28593/how-to-propagate-error-back-to-previous-layer-in-cnn?rq=1">How to propagate error back to previous layer in CNN?</a></li>
</ul>

<deep-learning><convolutional-neural-networks><backpropagation>
How do I calculate the partial derivative with respect to $x$?
A.Najafi
# How do I calculate the partial derivative with respect to $x$?

(Asked by: username_0 on 2020-05-16)

I am trying to implement CNN using python NumPy. I searched so much, but all I found was for one filter with one channel for convolution.
Suppose $x$ is an image with the shape: (N_Height, N_Width, N_Channel) = (5,5,3).
Let's say I have 16 filters with this shape: (F_Height, F_Width, N_Channel) = (3,3,3) , stride=1 and padding=0
Forward:
The output shape after convolution 2d will be
(
math.floor((N_Height - F_Height + 2*padding)/stride + 1 )),
math.floor((N_Width- F_Width + 2*padding)/stride + 1 )),
filter_count
)

So, the output of this layer will be an array with this shape: (Height, Width, Channel) =  (3, 3, 16)
BackPropagation:
Suppose $dL/dh$ is the input for my layer in back-propagation with this shape: (3, 3, 16)
Now, I must find $dL/dw$ and $dL/dx$: $dL/dw$ to update my filters parameter and $dL/dx$ to pass it as input to the previous layer as the loss respect to the input $x$.
From this answer Error respect to filters weights I found how to calculate $dL/dw$.
The problem I have in the back-propagation is I don't know how to calculate $dL/dx$ having this shape: (5, 5, 3) and pass it to the previous layer.
I read lots of articles in Medium and other sites, but I don't get how to calculate it:

How Backpropagation works in a CNN
The best explanation of Convolutional Neural Networks on the Internet!
Backpropagation In Convolutional Neural Networks
How to propagate error back to previous layer in CNN?

**Tags:** deep-learning, convolutional-neural-networks, backpropagation


---

## 1 answers

### Answer by username_1 on 2020-05-22. Score: 1

While this may not be the answer you were looking for, I hope this explanation will help you to understand applying backpropagation to a CNN. Fundamentally, convolutional layers are no different than dense layers, however there are restrictions. The key one is weight-sharing which allows a CNN to be much more efficient than a regular dense layer (as well as it being sparse due to locality). Imagine we are transforming a 4x4 image into a 2x2 image. Since we are inputting a 16-vector, and outputting a 4-vector, we need a weights matrix of 4x16:

[![enter image description here](https://i.sstatic.net/YHJad.png)](https://i.sstatic.net/YHJad.png)

This has 64 parameters. In a convolutional layer, we can accomplish this by convolving a 3x3 kernel over the image:

$$ K=
\begin{bmatrix}
k\_{1,1} & k\_{1,2} & k\_{1,3} \\
k\_{2,1} & k\_{2,2} & k\_{2,3} \\
k\_{3,1} & k\_{3,2} & k\_{3,3}
\end{bmatrix} $$

This convolution is equivalent to multiplying by the weights matrix:

[![enter image description here](https://i.sstatic.net/jyY0V.png)](https://i.sstatic.net/jyY0V.png)

As you can see, this only requires 9 parameters and backpropagation can be applied to update these parameters.

Image Source: <https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1>

> **Comments:**
> **username_0**: thanks but as you said it is not my question's answer. I finally find $dL/dX$ for each channel in my example ``16`` then I add all of them and  I get my $dL/dX$  

---

