Why don’t you just slice the tensor and then stack it?
Something like:
orange = X[0,:,:X.size(2)//2,:X.size(3)//2]
blue = X[0,:,:X.size(2)//2,X.size(3)//2:]
grey = X[0,:,X.size(2)//2:,:X.size(3)//2]
green = X[0,:,X.size(2)//2:,X.size(3)//2:]
output = torch.stack([orange,blue,grey,green])
This should preserve the gradients I believe.