  Neural network classifiers have become the de-facto choice for current
"pre-train then fine-tune" paradigms of visual classification. In this paper,
we investigate k-Nearest-Neighbor (k-NN) classifiers, a classical model-free
learning method from the pre-deep learning era, as an augmentation to modern
neural network based approaches. As a lazy learning method, k-NN simply
aggregates the distance between the test image and top-k neighbors in a
training set. We adopt k-NN with pre-trained visual representations produced by
either supervised or self-supervised methods in two steps: (1) Leverage k-NN
predicted probabilities as indications for easy vs. hard examples during
training. (2) Linearly interpolate the k-NN predicted distribution with that of
the augmented classifier. Via extensive experiments on a wide range of
classification tasks, our study reveals the generality and flexibility of k-NN
integration with additional insights: (1) k-NN achieves competitive results,
sometimes even outperforming a standard linear classifier. (2) Incorporating
k-NN is especially beneficial for tasks where parametric classifiers perform
poorly and / or in low-data regimes. We hope these discoveries will encourage
people to rethink the role of pre-deep learning, classical methods in computer
vision. Our code is available at: https://github.com/KMnP/nn-revisit.

Computer Vision and Pattern Recognition