@albanD  Doing that by customing cuda layer is possible. However it’s not device friendly…  Therefore I am trying to figure out whether it can be done by Pytorch api. To generate all the rectangular, I can use unfold, but I am not sure whether the task can be accomplished by some calculation, without resorting to cuda layer…