Do Deep Neural Networks Suffer from Crowding?
Crowding is a visual effect suffered by humans, in which an object that can be recognized in isolation can no longer be recognized when other objects, called flankers, are placed close to it. In this work, we study the effect of crowding in artificial Deep Neural Networks (DNNs) for object recognition. We analyze both deep convolutional neural networks (DCNNs) as well as an extension of DCNNs that are multi-scale and that change the receptive field size of the convolution filters with their position in the image. The latter networks, that we call eccentricitydependent, have been proposed for modeling the feedforward path of the primate visual cortex. Our results reveal that the eccentricity-dependent model, trained on target objects in isolation, can recognize such targets in the presence of flankers, if the targets are near the center of the image, whereas DCNNs cannot. Also, for all tested networks, when trained on targets in isolation, we find that recognition accuracy of the networks decreases the closer the flankers are to the target and the more flankers there are. We find that visual similarity between the target and flankers also plays a role and that pooling in early layers of the network leads to more crowding. Additionally, we show that incorporating flankers into the images of the training set for learning the DNNs does not lead to robustness against configurations not seen at training.
This paper studies if crowding, a visual effect suffered by human visual systems, happens to deep neural network as well. The paper systematically analyzes the performance difference when (1) clutter/flankers is present; (2) the similarity and proximity to the target; (3) when different architectures of the network is used. 
Pros:
There are very few papers to study if various visual perceptual phenomenon exists in deep neural nets, or in vision algorithms in general. This paper studies the effect of crowding in DNN/DCNN image classification problem, and presents some interesting results which seems to suggest similar effect exists in DNN because of pooling layers merges nearby responses. And this is related to the theories of crowding in humans, which is also interesting. The paper also suggests what we should not do prematurely pooling if when designing architectures. In my opinion such papers should be encouraged.
Cons:
My main criticism to the paper is that it solely studies crowding in the context of image classification. However, if crowding is studied in the context of object detection, where the task is localize the object and recognize its category, the effect may be significantly lessened. For example, R-CNN proposes high saliency regions where the object might be and perform classification on that masked region. Because targets are usually centered in such proposed region and background clutters are excluded from the proposed region, the accuracy can potentially be much higher. 
After all, the extent to which crowding is present in DNN depends a lot on the chosen architecture. And the architecture in this paper is very primitive compare to what researchers consider state-of-the-art these days, and the accuracy of the MNIST tasks reported by the paper are way lower than what most researchers would expect from a practical system. For example, [1] performs digit OCR which has much more clutters but with very high accuracy. It is not obvious architectures like that also suffer from crowding.
Suggestion:
The paper is overall easy to follow. I feel the experimental setup can be more clear if some more example images (non-cropped, like the ones in the Fig1 of supplementary material).
Overall, this paper has an interesting topic and is a nice read. The conclusions are not too strong because it uses simplistic architecture/datasets. But I think it is nonetheless this is a helpful paper to (re-)generate some interest on drawing relation between theories of human visual recognition and neural nets.
[1] Goodfellow, Ian J., et al. "Multi-digit number recognition from street view imagery using deep convolutional neural networks." arXiv preprint arXiv:1312.6082 (2013).