<p>I have been working on coding a CNN in python from scratch using numpy as a semester project and I think I have successfully implemented it up to backpropagation in the MaxPool Layers. However, my model seems to never converge whenever there is a Convolutional Layer(s) added. I am assuming there is a problem with the way I have implemented the backpropagation.</p>
<p>Most examples that I have seen for this implementation either really simplify it by using a one-channel input and a single one-channel filter, or just dive straight into the Mathematics which doesn't only not help but also confuses me more.</p>
<p>Here is the way I have tried to implement both Forward and Backward Propagation for multichannel inputs and outputs based on my own understanding and things I read online.</p>
<p><strong>Forward Prop:</strong>
<a href="https://i.sstatic.net/5QIZk.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/5QIZk.jpg" alt="Forward Propagation in Convolutional Layer" /></a></p>
<p><strong>Backward Prop for Filter Gradients:</strong>
<a href="https://i.sstatic.net/xWh8Z.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/xWh8Z.jpg" alt="enter image description here" /></a></p>
<p><strong>Backward Prop for Input Gradients:</strong>
<a href="https://i.sstatic.net/FDHlp.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/FDHlp.jpg" alt="enter image description here" /></a></p>
<p>Kindly point out anything that's wrong here. I have been working on this part for the last 2 days but there has to be a problem because my model never seems to converge.</p>
<p>Thanks!</p>

CC BY-SA 4.0
2021-05-16T15:54:26.047Z
I'm new to this, but what do you mean by not converging ? the error rates goes bigger ?
johnjerrico
CC BY-SA 4.0
2021-05-16T17:44:57.123Z
Thank you for the response @johnjerrico. The cost keeps fluctuating up and down, never drops as you would expect. Tried changing learning rates from 1 to 1e-8. No change
Ahmed Mustafa
<deep-learning><convolutional-neural-networks><backpropagation><convolution><convolutional-layers>
Convolutional Layer Multichannel Backpropagation Implementation
Ahmed Mustafa
# Convolutional Layer Multichannel Backpropagation Implementation

(Asked by: username_0 on 2021-05-16)

I have been working on coding a CNN in python from scratch using numpy as a semester project and I think I have successfully implemented it up to backpropagation in the MaxPool Layers. However, my model seems to never converge whenever there is a Convolutional Layer(s) added. I am assuming there is a problem with the way I have implemented the backpropagation.

Most examples that I have seen for this implementation either really simplify it by using a one-channel input and a single one-channel filter, or just dive straight into the Mathematics which doesn't only not help but also confuses me more.

Here is the way I have tried to implement both Forward and Backward Propagation for multichannel inputs and outputs based on my own understanding and things I read online.

**Forward Prop:**
[![Forward Propagation in Convolutional Layer](https://i.sstatic.net/5QIZk.jpg)](https://i.sstatic.net/5QIZk.jpg)

**Backward Prop for Filter Gradients:**
[![enter image description here](https://i.sstatic.net/xWh8Z.jpg)](https://i.sstatic.net/xWh8Z.jpg)

**Backward Prop for Input Gradients:**
[![enter image description here](https://i.sstatic.net/FDHlp.jpg)](https://i.sstatic.net/FDHlp.jpg)

Kindly point out anything that's wrong here. I have been working on this part for the last 2 days but there has to be a problem because my model never seems to converge.

Thanks!

**Tags:** deep-learning, convolutional-neural-networks, backpropagation, convolution, convolutional-layers

> **Comments:**
> **username_1**: I'm new to this, but what do you mean by not converging ? the error rates goes bigger ?  
> **username_0**: Thank you for the response @username_1. The cost keeps fluctuating up and down, never drops as you would expect. Tried changing learning rates from 1 to 1e-8. No change  

---

## 0 answers

