In contrast to fully connected networks, Convolutional Neural Networks (CNNs) achieve efficiency by learning weights associated with local filters with a finite spatial extent. An implication of this is that a filter may know what it is looking at, but not where it is positioned in the image. Information concerning absolute position is inherently useful, and it is reasonable to assume that deep CNNs may implicitly learn to encode this information if there is a means to do so. In this paper, we test this hypothesis revealing the surprising degree of absolute position information that is encoded in commonly used neural networks. A comprehensive set of experiments show the validity of this hypothesis and shed light on how and where this information is represented while offering clues to where positional information is derived from in deep CNNs.
This paper studies whether and how position information is encoded in CNNs. On top of VGG and ResNet, it constructs an additional PosENet to recover position information. By analyzing how well PosENet recovers position information, this paper provides several interesting findings: CNNs indeeds encode position information and zero-padding is surprisingly important here.
[Pros]
1. I enjoy reading this paper: probing CNNs is not easy, but it designs experiments in an intuitive way and rigorously performs ablation studies and analysis.
2. The observations and findings are interesting and helpful to the community.
[Cons]
1. A weakness of this paper is that it ignores the impact of training process while probing PosENet: In Table 1, VGG/ResNet perform much better than PosENet, but it could be because VGG/ResNet is easier to train (kind of fine-tuning PosENet only) than PosENet. Would be nice to show the training curve and train PosENet longer.
2. Zero-padding seems to play a surprisingly important role in encoding position information (Table 5), but it is still unclear why it is so important and how it helps.
Overall, I think this is a good paper.