My understanding is that torch.nn.unfold does the equivalent of tf.extract_image_patches. The extract_image_patches in TF doesn’t support gradients and backprop to my knowledge.
I was wondering what the status of unfold in PyTorch is - does it support gradients / autodiff right now? Has anyone used this successfully as part of an architecture?
In terms of speedup, how does it compare to explicitly extracting local windows in a DWise Conv like fashion? Example, Scott Gray has these kernels in his blocksparse repo, where I believe the idea is to run the convolutions at a block level and simulate the channel wise index gathering through a convolution primitive (which I guess implicitly calls im2col or whatever’s faster under the hood)- blocksparse/spatial_conv.py at master · openai/blocksparse · GitHub
Edit: More specifically, for the speedup part,  if anyone has benchmarked implementing a regular conv2d with unfold + matmul + fold and compared the two on a GPU, that would be great to know!
Thanks!