I have very large kernels (from 63 x 63 to 255 x 255) and would like to perform convolutions on an image of size 512 x 512. Currently, I get OOM errors because I think that PyTorch performs an nn.Unfold operation to treat the convolution as a matrix-vector product.
Is there a way to perform such large convolutional operations, for example using a distributed or a sequential approach?