<p>I have found that you should compute derivatives <span class="math-container">$\frac{\partial L}{\partial A}, \frac{\partial L}{\partial Z}$</span> in Flatten layer and then reshape Conv2D input shape.</p>

CC BY-SA 4.0
2021-03-01T22:45:04.403Z
Would you share an example of neural network model having Flatten --> Convolution?
Cloud Cho
CC BY-SA 4.0
2020-05-01T07:48:36.027Z
2020-05-06T10:47:26.017Z
2020-05-06T10:47:26.017Z
PentaHackedAll
<p>I'm trying to create a convolutional neural network without frameworks (such as PyTorch, TensorFlow, Keras, and so on) with Python.</p>

<p>Here's a description of CNN taken from <a href="https://en.wikipedia.org/wiki/Convolutional_neural_network" rel="nofollow noreferrer">the Wikipedia article</a></p>

<blockquote>
  <p>In deep learning, a convolutional neural network (CNN, or ConvNet) is a class of deep neural networks, most commonly applied to analyzing visual imagery. They are also known as shift invariant or space invariant artificial neural networks (SIANN), based on their shared-weights architecture and translation invariance characteristics. They have applications in image and video recognition, recommender systems, image classification, medical image analysis, natural language processing, and financial time series.</p>
</blockquote>

<p>A CNN has different types of layers, such as convolution, pooling (max or average), flatten and dense (or fully-connected) layers.</p>

<p>I have a few questions.</p>

<ol>
<li><p>Should we compute gradients (such as <span class="math-container">$\frac{\partial L}{\partial A_i}$</span>,<span class="math-container">$\frac{\partial L}{\partial Z_i}$</span>,<span class="math-container">$\frac{\partial L}{\partial A_{i-1}}$</span> and so on) in flatten layer or not?</p></li>
<li><p>If no, then how should I compute <span class="math-container">$\frac{\partial L}{\partial A_i}$</span> and <span class="math-container">$\frac{\partial L}{\partial Z_i}$</span> of first layer of convolutional layer? With <span class="math-container">$\frac{\partial L}{[\frac{\partial g(A_i)}{\partial x}]}$</span> or with 
<span class="math-container">$\frac{\partial L}{\partial dA_{i+2}}$</span>(P.S. as you know iteration of BackPropagation is reverse, so I used i+n for denote the previous layer)? </p></li>
<li><p>Or can I compute derivatives in Flatten layer with 
<span class="math-container">$$\frac{\partial J}{\partial A} = W_{i+1}^T Z_{i+1}$$</span>(i+1 denotes prev.layer in BackProp)
<span class="math-container">$$\frac{\partial L}{\partial Z} = \frac{\partial L}{\partial A} *\frac{\partial g(A_i)}{\partial x} $$</span>
and then reshape of Conv2D shape? </p></li>
</ol>

<p>P.S. I found questions like mine (names are same), but there're not answer to my question as I asking about formula.</p>

CC BY-SA 4.0
2021-12-25T12:03:48.553Z
What are $A$ and $Z$ here?
nbro
<machine-learning><convolutional-neural-networks><backpropagation>
Should I compute the gradients with respect to the flatten layer in a convolutional neural network?
PentaHackedAll
# Should I compute the gradients with respect to the flatten layer in a convolutional neural network?

(Asked by: username_0 on 2020-04-30)

I'm trying to create a convolutional neural network without frameworks (such as PyTorch, TensorFlow, Keras, and so on) with Python.

Here's a description of CNN taken from [the Wikipedia article](https://en.wikipedia.org/wiki/Convolutional_neural_network)

> In deep learning, a convolutional neural network (CNN, or ConvNet) is a class of deep neural networks, most commonly applied to analyzing visual imagery. They are also known as shift invariant or space invariant artificial neural networks (SIANN), based on their shared-weights architecture and translation invariance characteristics. They have applications in image and video recognition, recommender systems, image classification, medical image analysis, natural language processing, and financial time series.

A CNN has different types of layers, such as convolution, pooling (max or average), flatten and dense (or fully-connected) layers.

I have a few questions.

1. Should we compute gradients (such as $\frac{\partial L}{\partial A\_i}$,$\frac{\partial L}{\partial Z\_i}$,$\frac{\partial L}{\partial A\_{i-1}}$ and so on) in flatten layer or not?
2. If no, then how should I compute $\frac{\partial L}{\partial A\_i}$ and $\frac{\partial L}{\partial Z\_i}$ of first layer of convolutional layer? With $\frac{\partial L}{[\frac{\partial g(A\_i)}{\partial x}]}$ or with
   $\frac{\partial L}{\partial dA\_{i+2}}$(P.S. as you know iteration of BackPropagation is reverse, so I used i+n for denote the previous layer)?
3. Or can I compute derivatives in Flatten layer with
   $$\frac{\partial J}{\partial A} = W\_{i+1}^T Z\_{i+1}$$(i+1 denotes prev.layer in BackProp)
   $$\frac{\partial L}{\partial Z} = \frac{\partial L}{\partial A} \*\frac{\partial g(A\_i)}{\partial x} $$
   and then reshape of Conv2D shape?

P.S. I found questions like mine (names are same), but there're not answer to my question as I asking about formula.

**Tags:** machine-learning, convolutional-neural-networks, backpropagation

> **Comments:**
> **username_1**: What are $A$ and $Z$ here?  

---

## 1 answers

### Answer by username_0 on 2020-05-01. Score: 1 | Accepted answer

I have found that you should compute derivatives $\frac{\partial L}{\partial A}, \frac{\partial L}{\partial Z}$ in Flatten layer and then reshape Conv2D input shape.

> **Comments:**
> **username_2**: Would you share an example of neural network model having Flatten --> Convolution?  

---

