But in order to keep track of the gradient I did this :
dy = int(X.shape[2] / pool_stripe)  # dimension of Y
Y = X[:, :, :dy, :dy] * 1

Can you see a more elegant way to do so? Or does it look fine for you ?