Convolutional layer utilizes the shift-equivalent prior of images which makes it a great success for image processing. However, commonly used down sampling methods in convolutional neural networks (CNNs), such as max-pooling, averagepooling, and stride convolution, are not shift-equivalent. This destroys the shiftequivalent property of CNNs and degrades their performance. In this paper, we provide a formal definition of shift-equivalent when down sampling involved. We propose a strict shift equivalent and anti-aliasing pooling method. This is achieved by (inverse) Discrete Fourier Transform and we call our method frequency pooling. Experiments on image classifications show that frequency pooling improves accuracy and robustness w.r.t shifts of CNNs.
This paper researches the pooling operation, which is an important component in convolutional neural networks (CNN) for image classification. Taking the perspective from signal processing, this paper proposes a pooling operation called frequency pooling (F-pooling). The key motivation is to make the pooling operation shift-equivalent and anti-aliasing. This paper gives an improved definition on shift-equivalent functions and shows that the proposed F-pooling is optimal in the sense of reconstructing the orignal signal. The F-pooling is then implemented with matrix multiplications and tested with recent convolutional neural networks for image classifiation on CIFAR-100 and a subset of ImageNet dataset. 
It is interesting to take the perspective from signal processing to give pooling operation in CNN a formal treatment. As indicated in the recent literature, enforcing shift-invariance does help to improve the performance of a CNN on classification accuracy and the robustness with respect to image shift. At the same time, this work can be further enhanced at the following aspects:
1. This work can make it clearer in principle how anti-aliasing contributes to improving the classification performance and robustness. This will help to make this paper more self-contained. 
2. When showing the optimality of F-pooling in Section 2.3, the criterion is to reconstruct the original signal x. Considering that the ultimate goal is classification, the information to be maximally preserved through each operation through the layers shall be the information that relates to the class label y. In light of this, some justification and explanation shall be provided for using this criterion for optimality.
3. The experimental study is weak. Experiments could be conducted on more benchmark datasets with more CNN architectures to convincingly show the effectiveness of the proposed F-pooling. Also, from the three Tables in the experimental part, the improvement of F-pooling over AA-pooling (developed by the main reference of this work) does not seem to be significant or consistent. For example, in Table 2, the F-pooling only wins at either accuracy (marginally) or consistency, but not both. In Table 3, the F-pooling consistently shows inferior classification performance, although obtaining slightly higher consistency. This makes the advantage of F-pooling over the existing AA-pooling unclear.