Convolutional Kernel Networks
An important goal in visual recognition is to devise image representations that are invariant to particular transformations. In this paper, we address this goal with a new type of convolutional neural network (CNN) whose invariance is encoded by a reproducing kernel. Unlike traditional approaches where neural networks are learned either to represent data or for solving a classification task, our network learns to approximate the kernel feature map on training data. Such an approach enjoys several benefits over classical ones. First, by teaching CNNs to be invariant, we obtain simple network architectures that achieve a similar accuracy to more complex ones, while being easy to train and robust to overfitting. Second, we bridge a gap between the neural network literature and kernels, which are natural tools to model invariance. We evaluate our methodology on visual recognition tasks where CNNs have proven to perform well, e.g., digit recognition with the MNIST dataset, and the more challenging CIFAR-10 and STL-10 datasets, where our accuracy is competitive with the state of the art.
This paper presents a kernel based approach to learning a model that is heavily based on convolutional networks. The standard types of layers of a convolutional network is replicated by a kernel that produces an equivalent or similar response to that of original. I think the main aspect of the paper is the approximation of the gaussian kernel by a learned formulation given in equation 4, however the exact procedure for training is not clear from the paper. The results presented on widely used datasets suggest that despite unsupervised training of the hierarchy of feature extractor, the system achieves competitive results.The method presented in the paper is algorithmically quite heavy compared to the simplicity of convolutional networks. On the other hand, it might provide useful insights into new models or better understanding of current models. I think the main drawback of this paper is that, it does not contain a clean explanation of an end-to-end process of training the model. Convolutional nets are popular since they achieve state of the art results on challenging datasets, but at the same time, the training procedure is very simple and scales to large datasets. I think it would be very helpful if the paper included a more clean explanation of the overall algorithm and computational requirements of training a CKN.