Body: <p>I'm implementing strided 2D convolution. My formula looks like this:
  <span class="math-container">$$y_{i, j} = \sum_{m=0}^{F_h - 1}\sum_{n=0}^{F_w - 1} x_{s\cdot i + m, s\cdot j + n}\,f_{m, n}, \tag{1}$$</span> where <span class="math-container">$s$</span> is the stride
  (some sources might refer to this as 'cross-correlation' but 'convolution' is consistent with PyTorch's definition)</p>
  
  <p>I have calculated the gradient with respect to the filter as:</p>
  
  <p><span class="math-container">$$\frac{\partial E}{\partial f_{m', n'}} = \sum_{i=0}^{(x_h - F_h) / s}\sum_{j=0}^{(x_w - F_w) / s} x_{s\cdot i + m', s\cdot j + n'} \frac{\partial E}{\partial y_{i, j}} \tag{2}$$</span></p>
  
  <p>and some simple dummy index relabeling leads to:
  <span class="math-container">$$\frac{\partial E}{\partial f_{i, j}} = \sum_{m=0}^{(x_h - F_h) / s}\sum_{n=0}^{(x_w - F_w) / s} x_{s\cdot m + i, s\cdot n + j} \frac{\partial E}{\partial y_{m, n}} \tag{3}$$</span> </p>
  
  <p>Equation <span class="math-container">$(3)$</span> looks similar to the first, but not exactly (the <span class="math-container">$s$</span> is on the wrong term!). My objective is to convert the second equation into 'convolutional form' so that I can calculate it using my existing, efficient convolution algorithm.</p>
  
  <p>Could someone please help me work this out, or point out any errors that I have made?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-28T16:03:47.380Z
Comments.Text: I did in fact find an answer to this and will write an answer soon
Comments.UserDisplayName: Nihar Karve
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-26T08:47:59.560Z
Comments.Text: Were you able to figure this out?
Comments.UserDisplayName: Iain
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-01T11:01:17.347Z
Comments.Text: @Iain check out this file in my repository: https://github.com/nkarve/twistml/blob/main/src/trainable.py
Comments.UserDisplayName: Nihar Karve
Tags: <convolutional-neural-networks>
Title: Conversion of strided filter gradient to convolutional form
UserDisplayName: Nihar Karve
ThreadText: # Conversion of strided filter gradient to convolutional form
  
  (Asked by: username_0 on 2020-05-15)
  
  I'm implementing strided 2D convolution. My formula looks like this:
  $$y\_{i, j} = \sum\_{m=0}^{F\_h - 1}\sum\_{n=0}^{F\_w - 1} x\_{s\cdot i + m, s\cdot j + n}\,f\_{m, n}, \tag{1}$$ where $s$ is the stride
  (some sources might refer to this as 'cross-correlation' but 'convolution' is consistent with PyTorch's definition)
  
  I have calculated the gradient with respect to the filter as:
  
  $$\frac{\partial E}{\partial f\_{m', n'}} = \sum\_{i=0}^{(x\_h - F\_h) / s}\sum\_{j=0}^{(x\_w - F\_w) / s} x\_{s\cdot i + m', s\cdot j + n'} \frac{\partial E}{\partial y\_{i, j}} \tag{2}$$
  
  and some simple dummy index relabeling leads to:
  $$\frac{\partial E}{\partial f\_{i, j}} = \sum\_{m=0}^{(x\_h - F\_h) / s}\sum\_{n=0}^{(x\_w - F\_w) / s} x\_{s\cdot m + i, s\cdot n + j} \frac{\partial E}{\partial y\_{m, n}} \tag{3}$$
  
  Equation $(3)$ looks similar to the first, but not exactly (the $s$ is on the wrong term!). My objective is to convert the second equation into 'convolutional form' so that I can calculate it using my existing, efficient convolution algorithm.
  
  Could someone please help me work this out, or point out any errors that I have made?
  
  **Tags:** convolutional-neural-networks
  
  > **Comments:**
  > **username_0**: I did in fact find an answer to this and will write an answer soon  
  > **username_1**: Were you able to figure this out?  
  > **username_0**: @username_1 check out this file in my repository: https://github.com/nkarve/twistml/blob/main/src/trainable.py  
  
  ---
  
  ## 0 answers
  
  