Thanks a lot for the answer!
Basically, I have a tensor of B,2,H,W and set of B,N*N pairs of (X,Y), which are the middle of the patch, do you see a way to do it efficiently? With indices we should do a for loop, right?