Dear fellows,
I am wondering about the executing time for the forward and backward stages of the convolution 2d operation.
Check this notebook 1
As far as I know, the forward pass is composed by a single convolution operations. When doing the back propagation, the derivatives w.r.t each of the inputs would be computed with a convolution operation each.  Thus, I was expecting the backward pass to be slower than the forward.
But, according to the tests I did on the notebook (linked above) the backward pass is just a very small fraction of the forward pass.
From the notebook, I have:
Screen Shot 2021-01-30 at 6.08.50 PM814×136 8.68 KB
Could someone shed some light on this?