<p>Yes, a simple reshape would do the trick. A flattening layer is just a tool for reshaping data/activations to make them compatible with other layers/functions. The flattening layer doesn't change the activations themselves, so there is no special backpropagation handling needed other than changing back the shape.</p>

CC BY-SA 4.0
2019-04-04T19:52:23.580Z
2019-04-04T19:52:23.580Z
Philip Raeisghasem
<p>The <code>Flatten</code> layer has no learnable parameters in itself (the operation it performs is fully defined by construction); still, it has to propagate the gradient to the previous layers.</p>
<p>In general, the <code>Flatten</code> operation is well-posed, as whatever is the input shape you know what the output shape is.</p>
<p>When you backpropagate, you are supposed to do an &quot;Unflatten&quot;, which maps a flattened tensor into a tensor of a given shape, and you know what that specific shape is from the forward pass, so it is also a well-posed operation.</p>
<p>More formally</p>
<p>Say you have <code>Img1</code> in input of your <code>Flatten</code> layer</p>
<p><span class="math-container">$$
\begin{pmatrix}
f_{1,1}(x; w_{1,1}) &amp; f_{1,2}(x; w_{1,2}) \\ 
f_{2,1}(x; w_{2,1}) &amp; f_{2,2}(x; w_{2,2})
\end{pmatrix}
$$</span></p>
<p>So, in the output you have</p>
<p><span class="math-container">$$
\begin{pmatrix}
f_{1,1}(x; w_{1,1}) &amp; f_{1,2}(x; w_{1,2}) &amp; f_{2,1}(x; w_{2,1}) &amp; f_{2,2}(x; w_{2,2})
\end{pmatrix}
$$</span></p>
<p>When you compute the gradient you have</p>
<p><span class="math-container">$$
\frac{df_{i,j}(x; w_{i,j})}{dw_{i,j}}
$$</span></p>
<p>and everything in the same position as in the forward pass, so the unflatten maps from the <code>(1, 4)</code> tensor to the <code>(2, 2)</code> tensor.</p>

CC BY-SA 4.0
2020-09-06T07:10:25.853Z
2020-09-10T21:38:13.503Z
2020-09-10T21:38:13.503Z
Nicola Bernini
<p>I am making a NN library without any other external NN library, so I am implementing all layers, including the flatten layer, and algorithms (forward and backward pass) from scratch. I know the forward implementation of the flatten layer, but is the backward just reshaping it or not? If yes, can I just call a simple NumPy's <code>reshape</code> function to reshape it?</p>

<neural-networks><convolutional-neural-networks><backpropagation><implementation>
How should I implement the backward pass through a flatten layer of a CNN?
Clement
# How should I implement the backward pass through a flatten layer of a CNN?

(Asked by: username_0 on 2019-04-04)

I am making a NN library without any other external NN library, so I am implementing all layers, including the flatten layer, and algorithms (forward and backward pass) from scratch. I know the forward implementation of the flatten layer, but is the backward just reshaping it or not? If yes, can I just call a simple NumPy's `reshape` function to reshape it?

**Tags:** neural-networks, convolutional-neural-networks, backpropagation, implementation


---

## 2 answers

### Answer by username_1 on 2019-04-04. Score: 1 | Accepted answer

Yes, a simple reshape would do the trick. A flattening layer is just a tool for reshaping data/activations to make them compatible with other layers/functions. The flattening layer doesn't change the activations themselves, so there is no special backpropagation handling needed other than changing back the shape.


---

### Answer by username_2 on 2020-09-06. Score: 2

The `Flatten` layer has no learnable parameters in itself (the operation it performs is fully defined by construction); still, it has to propagate the gradient to the previous layers.

In general, the `Flatten` operation is well-posed, as whatever is the input shape you know what the output shape is.

When you backpropagate, you are supposed to do an "Unflatten", which maps a flattened tensor into a tensor of a given shape, and you know what that specific shape is from the forward pass, so it is also a well-posed operation.

More formally

Say you have `Img1` in input of your `Flatten` layer

$$
\begin{pmatrix}
f\_{1,1}(x; w\_{1,1}) & f\_{1,2}(x; w\_{1,2}) \\
f\_{2,1}(x; w\_{2,1}) & f\_{2,2}(x; w\_{2,2})
\end{pmatrix}
$$

So, in the output you have

$$
\begin{pmatrix}
f\_{1,1}(x; w\_{1,1}) & f\_{1,2}(x; w\_{1,2}) & f\_{2,1}(x; w\_{2,1}) & f\_{2,2}(x; w\_{2,2})
\end{pmatrix}
$$

When you compute the gradient you have

$$
\frac{df\_{i,j}(x; w\_{i,j})}{dw\_{i,j}}
$$

and everything in the same position as in the forward pass, so the unflatten maps from the `(1, 4)` tensor to the `(2, 2)` tensor.


---

