Body: <p>I have been working on coding a CNN in python from scratch using numpy as a semester project and I think I have successfully implemented it up to backpropagation in the MaxPool Layers. However, my model seems to never converge whenever there is a Convolutional Layer(s) added. I am assuming there is a problem with the way I have implemented the backpropagation.</p>
  <p>Most examples that I have seen for this implementation either really simplify it by using a one-channel input and a single one-channel filter, or just dive straight into the Mathematics which doesn't only not help but also confuses me more.</p>
  <p>Here is the way I have tried to implement both Forward and Backward Propagation for multichannel inputs and outputs based on my own understanding and things I read online.</p>
  <p><strong>Forward Prop:</strong>
  <a href="https://i.sstatic.net/5QIZk.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/5QIZk.jpg" alt="Forward Propagation in Convolutional Layer" /></a></p>
  <p><strong>Backward Prop for Filter Gradients:</strong>
  <a href="https://i.sstatic.net/xWh8Z.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/xWh8Z.jpg" alt="enter image description here" /></a></p>
  <p><strong>Backward Prop for Input Gradients:</strong>
  <a href="https://i.sstatic.net/FDHlp.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/FDHlp.jpg" alt="enter image description here" /></a></p>
  <p>Kindly point out anything that's wrong here. I have been working on this part for the last 2 days but there has to be a problem because my model never seems to converge.</p>
  <p>Thanks!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-05-16T15:54:26.047Z
Comments.Text: I'm new to this, but what do you mean by not converging ? the error rates goes bigger ?
Comments.UserDisplayName: johnjerrico
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-05-16T17:44:57.123Z
Comments.Text: Thank you for the response @johnjerrico. The cost keeps fluctuating up and down, never drops as you would expect. Tried changing learning rates from 1 to 1e-8. No change
Comments.UserDisplayName: Ahmed Mustafa
Tags: <deep-learning><convolutional-neural-networks><backpropagation><convolution><convolutional-layers>
Title: Convolutional Layer Multichannel Backpropagation Implementation
UserDisplayName: Ahmed Mustafa
ThreadText: # Convolutional Layer Multichannel Backpropagation Implementation
  
  (Asked by: username_0 on 2021-05-16)
  
  I have been working on coding a CNN in python from scratch using numpy as a semester project and I think I have successfully implemented it up to backpropagation in the MaxPool Layers. However, my model seems to never converge whenever there is a Convolutional Layer(s) added. I am assuming there is a problem with the way I have implemented the backpropagation.
  
  Most examples that I have seen for this implementation either really simplify it by using a one-channel input and a single one-channel filter, or just dive straight into the Mathematics which doesn't only not help but also confuses me more.
  
  Here is the way I have tried to implement both Forward and Backward Propagation for multichannel inputs and outputs based on my own understanding and things I read online.
  
  **Forward Prop:**
  [![Forward Propagation in Convolutional Layer](https://i.sstatic.net/5QIZk.jpg)](https://i.sstatic.net/5QIZk.jpg)
  
  **Backward Prop for Filter Gradients:**
  [![enter image description here](https://i.sstatic.net/xWh8Z.jpg)](https://i.sstatic.net/xWh8Z.jpg)
  
  **Backward Prop for Input Gradients:**
  [![enter image description here](https://i.sstatic.net/FDHlp.jpg)](https://i.sstatic.net/FDHlp.jpg)
  
  Kindly point out anything that's wrong here. I have been working on this part for the last 2 days but there has to be a problem because my model never seems to converge.
  
  Thanks!
  
  **Tags:** deep-learning, convolutional-neural-networks, backpropagation, convolution, convolutional-layers
  
  > **Comments:**
  > **username_1**: I'm new to this, but what do you mean by not converging ? the error rates goes bigger ?  
  > **username_0**: Thank you for the response @username_1. The cost keeps fluctuating up and down, never drops as you would expect. Tried changing learning rates from 1 to 1e-8. No change  
  
  ---
  
  ## 0 answers
  
  