title: Implementing same convolution in Pytorch
body: <p>I was trying to implement same convolution. It seems (surprisingly) that such a standard layer isn't available out of the box in Pytorch. I hope this helps make sure a layer available. I was needing help generalizing it to use ALL hyper parameters. Right now it only uses kernel size (i.e. variable stride, dialation or whatever else that might affect things isn't part of this. Anyone know how to generalize it with all hyper params of a Convolution (in Conv Nets)?</p>
  
  <p>Code:</p>
  
  <pre><code>def get_cnn(C, out_filters=[14,13,12,3], kernels=[(9,7),(7,7),(5,3),(3,3)]):
      N = len(out_filters)
      print()
      print(f'C = {C}')
      model_layers = OrderedDict()
      in_channels = C
      x = torch.randn(1,C,10,10)
      out = x
      print(f'initial_size = {x.size()}')
      for i in range(1,N):
          print(f'--- building layer = {i}')
          ## create conv layer
          kernel_size = kernels[i]
          out_channels = out_filters[i]
          # to make sure its a same convolution (ignoring stride)
          print(f'in_channels = {in_channels}')
          print(f'out_channels = {out_channels}')
          padding = ((kernels[i][0]-1)//2,(kernels[i][1]-1)//2)
          conv = torch.nn.Conv2d(in_channels, out_channels, kernel_size, padding=padding)
          #conv = torch.nn.Conv2d(in_channels, out_channels, kernel_size)
          model_layers[f'Conv{i}'] = conv
          ## compute dummy data
          out = conv(out)
          print(f'x.size() = {x.size()}')
          print(f'out.size() = {out.size()}')
          ## add activation
          model_layers[f'ReLU{i}'] = torch.nn.ReLU()
          ## so that next layer works
          in_channels = out_channels
      ## make sequential model
      mdl = torch.nn.Sequential(model_layers)
      y = mdl(x)
      return mdl
  
  </code></pre>
  
  <hr>
  
  <p>References:</p>
  
  <ul>
  <li><p>Does padding ensure that the sizes of input and output of a CNN match?
  : <a href="https://stats.stackexchange.com/questions/419902/does-padding-ensure-that-the-sizes-of-input-and-output-of-a-cnn-match">https://stats.stackexchange.com/questions/419902/does-padding-ensure-that-the-sizes-of-input-and-output-of-a-cnn-match</a></p></li>
  <li><p>arithmetic of convolutions: <a href="https://arxiv.org/abs/1603.07285" rel="nofollow noreferrer">https://arxiv.org/abs/1603.07285</a></p></li>
  <li><p><a href="https://discuss.pytorch.org/t/same-convolution-in-pytorch/19937" rel="nofollow noreferrer">https://discuss.pytorch.org/t/same-convolution-in-pytorch/19937</a></p></li>
  <li><p><a href="https://discuss.pytorch.org/t/how-do-i-make-sure-the-output-of-a-cnn-never-decrease-in-size-using-padding/52103" rel="nofollow noreferrer">https://discuss.pytorch.org/t/how-do-i-make-sure-the-output-of-a-cnn-never-decrease-in-size-using-padding/52103</a></p></li>
  </ul>
  
  <hr>
  
  <p>Full running code:</p>
  
  <pre><code>import torch
  
  from collections import OrderedDict
  
  import random
  
  def do_conv():
      x = torch.randn(1,3,5,6)
      conv = torch.nn.Conv2d(in_channels=3, out_channels=5, kernel_size=(3,3))
      y = conv(x)
      print(y)
  
  def get_cnn(C, out_filters=[14,13,12,3], kernels=[(9,7),(7,7),(5,3),(3,3)]):
      N = len(out_filters)
      print()
      print(f'C = {C}')
      model_layers = OrderedDict()
      in_channels = C
      x = torch.randn(1,C,10,10)
      out = x
      print(f'initial_size = {x.size()}')
      for i in range(1,N):
          print(f'--- building layer = {i}')
          ## create conv layer
          kernel_size = kernels[i]
          out_channels = out_filters[i]
          # to make sure its a same convolution (ignoring stride)
          print(f'in_channels = {in_channels}')
          print(f'out_channels = {out_channels}')
          padding = ((kernels[i][0]-1)//2,(kernels[i][1]-1)//2)
          conv = torch.nn.Conv2d(in_channels, out_channels, kernel_size, padding=padding)
          #conv = torch.nn.Conv2d(in_channels, out_channels, kernel_size)
          model_layers[f'Conv{i}'] = conv
          ## compute dummy data
          out = conv(out)
          print(f'x.size() = {x.size()}')
          print(f'out.size() = {out.size()}')
          ## add activation
          model_layers[f'ReLU{i}'] = torch.nn.ReLU()
          ## so that next layer works
          in_channels = out_channels
      ## make sequential model
      mdl = torch.nn.Sequential(model_layers)
      y = mdl(x)
      return mdl
  
  def get_hardcoded_variable_size_input():
      C = 3
      variable_size_input = [torch.randn(1,C,15,15), torch.randn(1,C,12,16), torch.randn(1,C,15,11)]
      return variable_size_input
  
  def get_variable_size_input(N=3):
      variable_size_input = []
      for i in range(N):
          C,H,W = 3, random.randint(11,20), random.randint(14,19)
          x = torch.randn(C,H,W)
          variable_size_input.append(x)
      return variable_size_input
  
  def any_input_output_equals_input():
      '''
      test that no matter what input, the output of the network is same as the input
      '''
      X_list = get_hardcoded_variable_size_input()
      for i in range(len(X_list)):
          print(f'data point i = {i}')
          ## get data
          x = X_list[i]
          print(x.size())
          ## create cnn
          _,C,_,_ = x.size()
          cnn = get_cnn(C)
          # pass data
          y = cnn(x)
          ## make sure input and output have same size
          assert x.size() == y.size(), f'Error: {x.size()} != {y.size()}'
      ##
  
  if __name__ == '__main__':
      #do_conv()
      print()
      print('start main')
      any_input_output_equals_input()
      print('Done \a')
  </code></pre>