tensor.unfold will give you the image patches, so I assume you are applying some operation on each patch and reshape it to this activation shape afterwards?
What operation would you like to apply? Maybe a standard nn.Conv2d module will also do the trick?