Hi,
Here is the issue



 soulslicer:

kernel = kernel.repeat((B, 64, 1, 1))


If your input size is [64, h, w] (batch_size does not matter), and if you want to have outputs with size [64, new_h, new_w], then you need 64 filters which each one is [64, 3, 3]. Literally a [64, 64, 3, 3] filter is needed.

b = 2
c = 64
h, w = 15, 15

x = torch.randn(b, c, h, w)
kernel = (1/9)*torch.tensor([[1, 1, 1], 
                             [1, 1, 1], 
                             [1, 1, 1]])
kernel = kernel.repeat((64, 64, 1, 1))
print(kernel.shape)  # torch.Size([64, 64, 3, 3])
res = F.conv2d(input=x, weight=kernel, padding=1)
print(res.shape)  # torch.Size([2, 64, 15, 15])

Bests