You could wrap the filt in an nn.Parameter, which would create gradients in it after a backward() call:
filt = nn.Parameter(torch.randn(channels, kh, kw))

patches = image.unfold(...)
...

patches.sum().backward()
print(filt.grad)




 Hmrishav_Bandyopadhy:

Also, does implementing this whole in a forward function of an nn.Module class ensures proper flow of gradients and proper update of weights?


This should be the case. The small example shows, that the computation graph is properly created.