  In many recommender systems and search problems, presenting a well balanced
set of results can be an important goal in addition to serving highly relevant
content. For example, in a movie recommendation system, it may be helpful to
achieve a certain balance of different genres, likewise, it may be important to
balance between highly popular versus highly personalized shows. Such balances
could be thought across many categories and may be required for enhanced user
experience, business considerations, fairness objectives etc. In this paper, we
consider the problem of calibrating with respect to any given categories over
items. We propose a way to balance a trade-off between relevance and
calibration via a Linear Programming optimization problem where we learn a
doubly stochastic matrix to achieve optimal balance in expectation. We then
realize the learned policy using the Birkhoff-von Neumann decomposition of a
doubly stochastic matrix. Several optimizations are considered over the
proposed basic approach to make it fast. The experiments show that the proposed
formulation can achieve a much better trade-off compared to many other
baselines. This paper does not prescribe the exact categories to calibrate over
(such as genres) universally for applications. This is likely dependent on the
particular task or business objective. The main contribution of the paper is
that it proposes a framework that can be applied to a variety of problems and
demonstrates the efficacy of the proposed method using a few use-cases.

Information Retrieval;Machine Learning