So now I have the input is a tensor of size (1,1,60,60). I pad the image by 1 for every side, divide the image into 3x3 patches by using unfold() with stride 1, process them and then stitch them back together into a new one using view(). Then the new image will be 3xtimes bigger than the input image.
For the process, I have a dictionary having the positions according to each position of the center pixel of the 3x3 patches. And then for each patch, I will replace it’s 8 outer pixels with the pixels from the input image that have the coordinates stored in the dictionary.
x = torch.randn(1, 1, 60, 60)

coor = {(0,0) : np.matrix([[59, 7, 14, 22, 29, 36, 44, 51], [1, 1, 1, 1, 1, 1, 1, 1]])
        ...
       }
kc, kh, kw = 1, 3, 3  # kernel size
dc, dh, dw = 1, 1, 1  # stride
pad = (1,1,1,1)
paddedimage = F.pad(x, pad).to(DEVICE)
patches = paddedimage.unfold(1, kc, dc).unfold(2, kh, dh).unfold(3, kw, dw)
unfold_shape = patches.size()
print(unfold_shape)
patches = patches.contiguous().view(-1, kc, kh, kw)
print(patches.shape)

a = 0
for i in range(60):
    for j in range(60):
        coordinate = [i,j]
        indices = np.matrix(coor[(coordinate[0],coordinate[1])])
        patches[a,0,2,1] = image[0,0,indices[1,0],indices[0,0]]
        patches[a,0,2,2] = image[0,0,indices[1,1],indices[0,1]]
        patches[a,0,1,2] = image[0,0,indices[1,2],indices[0,2]]
        patches[a,0,0,2] = image[0,0,indices[1,3],indices[0,3]]
        patches[a,0,0,1] = image[0,0,indices[1,4],indices[0,4]]
        patches[a,0,0,0] = image[0,0,indices[1,5],indices[0,5]]
        patches[a,0,1,0] = image[0,0,indices[1,6],indices[0,6]]
        patches[a,0,2,0] = image[0,0,indices[1,7],indices[0,7]]
        a += 1
# Reshape back
patches_orig = patches.view(unfold_shape)
output_c = unfold_shape[1] * unfold_shape[4]
output_h = unfold_shape[2] * unfold_shape[5]
output_w = unfold_shape[3] * unfold_shape[6]
patches_orig = patches_orig.permute(0, 1, 4, 2, 5, 3, 6).contiguous()
patches_orig = patches_orig.view(1, output_c, output_h, output_w)


After this process, the output can become the input for the next convolutional layer…
What I want is to apply this process after every conv2d() layer which means the example with 1 channel will extend to cases with many channels input.
Is there anyway to optimize this process? Thank you Mr. @albanD!!!