Contrastive Learning from Pairwise Measurements
Learning from pairwise measurements naturally arises from many applications, such as rank aggregation, ordinal embedding, and crowdsourcing. However, most existing models and algorithms are susceptible to potential model misspecification. In this paper, we study a semiparametric model where the pairwise measurements follow a natural exponential family distribution with an unknown base measure. Such a semiparametric model includes various popular parametric models, such as the Bradley-Terry-Luce model and the paired cardinal model, as special cases. To estimate this semiparametric model without specifying the base measure, we propose a data augmentation technique to create virtual examples, which enables us to define a contrastive estimator. In particular, we prove that such a contrastive estimator is invariant to model misspecification within the natural exponential family, and moreover, attains the optimal statistical rate of convergence up to a logarithmic factor. We provide numerical experiments to corroborate our theory.
The paper considers a family of statistical models for pairwise measurements, in which we observe scores that come from triplets of the form (user, item1, item2). The problem is to estimate the underlying score function of user-item pair, which is assumed to be low-rank, and the observation to be generated by a natural exponential family.   The authors address the problem from a novel angle as a semiparametric estimation problem, which avoids the need to specify the log-partition function in the natural exponential family.  A penalized contrastive estimator is derived and its consistency and error bound are analyzed.  Numerical simulation supports the rate established in the bound.  I find this a nice addition to the learning from pairwise comparison literature. I recommend accept.    Detailed Comments:   1. Equation (3.2) (3.4)  effectively reduces the problem to a logistic loss matrix sensing problem with a specific randomly generated sensing matrix.  Can a similar robustness to model misspecification be stated for other sensing matrices and other link function? 2. The exponential family distribution of p(y|Theta) really needs to be a natural exp. family.   In general, one still needs to specify the model class in order to reparameterize a given exp family to a natural one. In that sense, the semiparametric view is not as strong as it seems.  3. Assumption 4.3 is somewhat strong as it rules out Gaussian, Poisson and many others. Perhaps it suffices to use sub-gaussian or sub-exponential?