Hi everyone,
I need to reconstruct an image tensor from its patches and a one-hot tensor by transpose convolution.
Let’s say the original image tensor, I, is 1x1x4x4 large:
(0 ,0 ,.,.) =
0   1   2   3
4   5   6   7
8   9  10  11
12  13  14  15
Its patches P (size 3x3, stride 1):
(0 ,0 ,.,.) =
0   1   2
4   5   6
8   9  10
(1 ,0 ,.,.) =
1   2   3
5   6   7
9  10  11
(2 ,0 ,.,.) =
4   5   6
8   9  10
12  13  14
(3 ,0 ,.,.) =
5   6   7
9  10  11
13  14  15
And I have a one-hot tensor K corresponding to P:
(0 ,0 ,.,.) =
1  0
0  0
(0 ,1 ,.,.) =
0  1
0  0
(0 ,2 ,.,.) =
0  0
1  0
(0 ,3 ,.,.) =
0  0
0  1
Transpose convolving K and P (as filter) reconstructs I’, but it also sums up values from overlapping patches:
(0 ,0 ,.,.) =
0   2   4   3
8  20  24  14
16  36  40  22
12  26  28  15
To work around this, I record the overlapping times in a tensor O with the same size of I when making patches using the helper function below, and at the end perform elementwise division I’ = I’/O.
def compute_overlaps(tensor, patch_size=(3, 3), patch_stride=(1, 1)):
    n, c, h, w = tensor.size()
    px, py = patch_size
    sx, sy = patch_stride
    nx = ((w-px)//sx)+1
    ny = ((h-py)//sy)+1

    overlaps = torch.zeros(tensor.size()).type_as(tensor.data)
    for i in range(ny):
        for j in range(nx):
            overlaps[:, :, i*sy:i*sy+py, j*sx:j*sx+px] += 1
    overlaps = Variable(overlaps)
    return overlaps

But I am wondering if averaging could be done in an efficient way?