Collaboratively Learning Preferences from Ordinal Data
In personalized recommendation systems, it is important to predict preferences of a user on items that have not been seen by that user yet. Similarly, in revenue management, it is important to predict outcomes of comparisons among those items that have never been compared so far. The MultiNomial Logit model, a popular discrete choice model, captures the structure of the hidden preferences with a low-rank matrix. In order to predict the preferences, we want to learn the underlying model from noisy observations of the low-rank matrix, collected as revealed preferences in various forms of ordinal data. A natural approach to learn such a model is to solve a convex relaxation of nuclear norm minimization. We present the convex relaxation approach in two contexts of interest: collaborative ranking and bundled choice modeling. In both cases, we show that the convex relaxation is minimax optimal. We prove an upper bound on the resulting error with finite samples, and provide a matching information-theoretic lower bound.
summary of paper: This paper proposes a new approach to inference for a low-rank matrix factorization model with ordinal data and another application.It is not clear if MultiNomialLogit (MNL) models have already been applied to these particular problems and the contribution is just the inference approach (nuclear norm minimization), or if the models are contributions as well.Regardless, the authors give performance guarantees for their approach, but do not demonstrate it on real or even synthetic data.quality: good - The theory appears to be thorough.The contributions obviously required substantial work. clarity: poor - Needs significant editing to address both minor issues and also major flow of ideas within sections.Should have more discussion of implications of contributions. originality: mediocre - Low rank matrix factorization has already been addressed from many, many angles; there, the contribution appears incremental.I am less familiar with the other application. significance: poor - The authors do not compare to any other methods (there are many well-established ones for matrix factorization), either empirically or theoretically.Even a small improvement can have a large impact, but the reader is left to assume that this paper would have no practical and only minor theoretical impact on the problems described.some detailed points - line 20 (abstract), Awkward/unclear/editing mistake?: "A popular discrete choice model of multinomial logit model captures..."- line 18, we're obviously talking about personalized user preferences, but that could be more clear-the user isn't mentioned until after the problem of preference prediction, which seems backward; then, we're talking about comparisons without talking about what we're comparing.This sequence of ideas is very convoluted. - abstract starts off with motivating applications, but then the contribution bears to only be theoretical (presenting an approach and proving a bound), with no justification as to why the theory is important, aside from it being a "natural approach" to for inference on "a popular model."No hint of how the theory impacts application, nor mention of experiments on real or even synthetic data. - line 33: should be comma after applications: "applications, such as..." - 1st line of intro and abstract are identical; while not strictly prohibited, it disengages the reader - line 44, 53: elements in a list (i.e., (a) something, (b) something) should have some kind of parallel structure.Here, (b) is the goal and (a) is the means.Again, the sequences of ideas seem convoluted. - line 57: MultiNomialLogit model not capitalized consistently with abstract; needs citation - line 61: "The" should be capitalized at start of sentence - line 62: it might have been better to only use one example in the intro (the bundling is more novel, so I'd pick that); you can always add the other in the experiments/results - line 70: it's great to have your contribution highlighted - line 70: why is it natural to take this approach? - contribution: I think you're saying that MNL is already applied to ordinal low-rank matrix factorization, but that you're doing a new kind of inference for the model and applying that to two cases.Is that right?Or has the MNL only been applied to non-ordinal MF problems and you are applying the MNL to ordinal data for the first time? - line 76: RUM should also have a citation here - line 94: spacing issues - Sections 3 and 4, generally: the practical implications of the theorems and their corollaries are unclear.Perhaps I need to study them in greater detail, but the authors could highlight in words, or empirically, the take home message.The single figure does not immediately contribute to greater understanding. - line 289: RMSE comes out of nowhere.Further, while RMSE is commonly used, it's not actually a good metric for rank-based item recommendation. - Discussion shouldn't just be a to-do list for the future.What should the reader take home?