MODELING UNCERTAINTY WITH HEDGED INSTANCE EMBEDDING
Instance embeddings are an efficient and versatile image representation that facilitates applications like recognition, verification, retrieval, and clustering. Many metric learning methods represent the input as a single point in the embedding space. Often the distance between points is used as a proxy for match confidence. However, this can fail to represent uncertainty which can arise when the input is ambiguous, e.g., due to occlusion or blurriness. This work addresses this issue and explicitly models the uncertainty by "hedging" the location of each input in the embedding space. We introduce the hedged instance embedding (HIB) in which embeddings are modeled as random variables and the model is trained under the variational information bottleneck principle (Alemi et al., 2016;Achille & Soatto, 2018). Empirical results on our new N-digit MNIST dataset show that our method leads to the desired behavior of "hedging its bets" across the embedding space upon encountering ambiguous inputs. This results in improved performance for image matching and classification tasks, more structure in the learned embedding space, and an ability to compute a per-exemplar uncertainty measure which is correlated with downstream performance.
While most works consider embedding as the problem of mapping an input into a point in an embedding space, paper 1341 considers the problem of mapping an input into a distribution in an embedding space. Computing the matching score of two inputs (e.g. two images) involves the following steps: (i) assuming a Gaussian distribution in the embedding space, computing the mean and standard deviation for each input, (ii) drawing a set of samples from each distribution, (3) computing the normalized distances between the samples and (iv) averaging to obtain a global score.
The proposed approach is validated on a new benchmark built on MNIST.
On the positive side:
-	The topic of injecting uncertainty in neural networks should be of broad interest to the ICLR community.
-	The paper is generally clear.
-	The qualitative evaluation provides intuitive results.
On the negative side:
-	The whole idea of drawing samples to compute the distance between two Gaussian distributions seems unnecessarily complicated. Why not computing directly a distance between distributions? There exist kernels between distributions, such as the Probability Product Kernel (PPK). See Jebara, Kondor, Howard “Probability product kernels”, JMLR’04. The PPK between two distributions p(x) and q(x) writes as: \int_x p^a(x) q^a(x) dx, where a is a parameter. When a=1, it is known as the Expected Likelihood Kernel (ELK). When a=1/2, this is known as the Hellinger or Bhattacharyya kernel (BK). In p and q are Gaussian distributions, then the PPK can be computed in closed form. If p and q are mixtures of Gaussians, then the ELK can be computed in closed form. 
-	The Mixture of Gaussians embedding extension is lacking in details. How does the network generate C Gaussian distributions? By having 2C output branches generating C means and C standard deviation vectors? 
-	It might be useful to provide more details about why the self-similarity measure makes sense as an uncertainty measure. In its current state, the paper does not provide much intuition and it took me some time to understand (I actually understood when I made the connection with the ELK). Also, why not using a simpler measure of uncertainty such as the trace of the covariance matrix?
-	The experiments are lacking in some respects:
o	It would be useful to report results without the VIB regularization.
o	The focus on the cases D=2 and D=3 (embedding in a 2D or 3D space) shades some doubt on the practical usefulness of this framework in a higher-dimensional case.
Miscellaneous:
-	It seems there is a typo between equations (4) and (5). It should write z_1^{(k_1)} \sim p(z_1|x_1)
--- 
In their rebuttal, the authors satisfyingly addressed my concerns. Hence, I am upgrading my overall rating.