We propose a novel architecture for k-shot classification on the Omniglot dataset. Building on prototypical networks, we extend their architecture to what we call Gaussian prototypical networks. Prototypical networks learn a map between images and embedding vectors, and use their clustering for classification. In our model, a part of the encoder output is interpreted as a confidence region estimate about the embedding point, and expressed as a Gaussian covariance matrix. Our network then constructs a direction and class dependent distance metric on the embedding space, using uncertainties of individual data points as weights. We show that Gaussian prototypical networks are a preferred architecture over vanilla prototypical networks with an equivalent number of parameters. We report results consistent with state-of-the-art performance in 1-shot and 5-shot classification both in 5-way and 20-way regime on the Omniglot dataset. We explore artificially down-sampling a fraction of images in the training set, which improves our performance. Our experiments therefore lead us to hypothesize that Gaussian prototypical networks might perform better in less homogeneous, noisier datasets, which are commonplace in real world applications.
The paper extends the prototypical networks of Snell et al, NIPS 2017 for one shot learning. Snell et al use a soft kNN classification rule, typically used in standard metric learning work (e.g. NCA, MCML), over learned instance projections, i.e. distances are computed over the learned projections. Each class is represented by a class prototype which is given by the average of the projections of the class instances. Classification is done with soft k-NN on the class prototypes. The distance that is used is the Euclidean distance over the learned representations, i.e. (z-c)^T(z-c), where z is the projection of the x instance to be classified and c is a class prototype, computed as the average of the projections of the support instances of a given class.
The present paper extends the above work to include the learning of a Mahalanobis matrix, S, for each instance, in addition to learning its projection. Thus now the classification is based on the Mahalanobis distance: (z-c)^T S_c (z-c). On a conceptual level since S_c should be a PSD matrix it can be written as the square of some matrix, i.e. S_c = A_c^TA_c, then the Mahanalobis distance becomes (A_c z - A_c c)^T ( A_c z-A_c c), i.e. in addition to learning a projection as it is done in Snell et al, the authors now learn also a linear transformation matrix which is a function of the support points (i.e. the ones which give rise to the class prototypes). The interesting part here is that the linear projection is a function of the support points. I wonder though if such a transformation could not be learned by the vanilla prototypical networks simply by learning now a projection matrix A_z as a function of the query point z. I am not sure I see any reason why the vanilla prototypical networks cannot learn to project x directly to A_z z and why one would need to do this indirectly through the use of the Mahalanobis distance as proposed in this paper.
On a more technical level the properties of the learned Mahalanobis matrix, i.e. the fact that it should be PSD, are not really discussed neither how this can be enforced especially in the case where S is a full matrix (even though the authors state that this method was not further explored). If S is diagonal then the S generation methods a) b) c) in the end of section 3.1 will make sure that S is PSD, I do not think that this is the case with d) though.
In the definition of the prototypes the component wise weigthing (eq. 5) works when the Mahalanobis matrix is diagonal (even though the weighting should be done by the \sqrt of it), how would it work if it was a full matrix is not clear.
On the experiments side the authors could have also experimented with miniImageNet and not only omniglot as is the standard practice in one shot learning papers. 
I am not sure I understand figure 3 in which the authors try to see what happens if instead of learning the Mahalanobis matrix one would learn a projection that would have as many additional dimensions as free elements in the Mahalanobis matrix. I would expect to see a comparison of the vanilla prototypical nets against their method for each one of the different scenarios of the free parameters of the S matrix, something like a ratio of accuracies of the two methods in order to establish whether learning the Mahalanobis matrix brings an improvement over the prototypical nets with an equal number of output parameters.