Ranking is a central task in machine learning and information retrieval. In this task, it is especially important to present the user with a slate of items that is appealing as a whole. This in turn requires taking into account interactions between items, since intuitively, placing an item on the slate affects the decision of which other items should be placed alongside it. In this work, we propose a sequence-to-sequence model for ranking called seq2slate. At each step, the model predicts the next item to place on the slate given the items already selected. The recurrent nature of the model allows complex dependencies between items to be captured directly in a flexible and scalable way. We show how to learn the model end-to-end from weak supervision in the form of easily obtained click-through data. We further demonstrate the usefulness of our approach in experiments on standard ranking benchmarks as well as in a real-world recommendation system.
The authors consider the problem of re-ranking an initial ranker that doesn’t consider interactions between items (e.g., a point-wise ranker) with a pointer-network approach that considers these interactions when re-ordering the input ranking. Notably, this is performed during decoding as opposed to {pairwise, list-wise} learning to rank approaches that consider interactions during training, but emit an item-wise score during inference. Operationally in practice, this has to be trained from click-through data for which the authors consider both a RL approach (Reinforce) and supervised training (a sequence-level hinge loss function) and decoded either with a single-step greedy policy or a sampling procedure. Experiments are conducted on learning-to-rank benchmarks where interactions are introduced to test the validity of the method and on a real-world, large-scale recommendation engine — showing solid improvements in both cases.
From a high-level perspective, the methodological innovation (a pointer-network trained on sequence loss from logged data), setting (re-ranking a slate to consider interactions), and empirical analyses are largely ‘incremental’ — although I think non-trivial to put together and the paper itself is well-written and fairly convincing. In framing the paper this way, I would have expected some comparison to sub-modular methods on the ‘diverse-clicks’ generated data for completeness, although I would be surprised if the Seq2Slate method doesn’t perform better (but all the more reason to conduct). In addition to reporting how to resolve some of the details of applying this, the most interesting results may very well be the real-world experiments as the result improvements are fairly impressive (such that I intend to play with this myself). Thus, as the focus is on details and empirical results over methodological innovation, this paper reads a bit like an industry-track paper — but I find the results interesting overall and am marginally inclined to accept.
Evaluating the paper along the requested dimensions:
= Quality: The paper clearly states its motivation, proposes a model, discusses practical issues, and provides convincing experiments (given the constraints of proprietary data, etc.). I didn’t observe any technical flaws and everything was relatively self-contained and easy to read. I could think of a few more experiments regarding submodular-based models, possibly different settings of the ‘diverse-click’ data for a sensitivity analysis, and a more direct comparison to [Ai, et al., SIGIR18], but this isn’t required to make the results sufficiently convincing. (6/10)
= Clarity: The paper is very clearly written. (7/10)
= Originality: This is the weakest aspect of the paper from a methodological perspective. It is a fairly straightforward application of pointer-networks. Even the path forward is fairly straightforward as outlined in the conclusion. One additional pointer that is methodologically similar, but for ‘discrete choice’ as opposed to re-ranking is [Mottini & Acuna-Agost, Deep Choice Model Using Pointer Networks for Airline Itinerary Prediction; KDD17] (which honestly, is probably a better venue for this specific work). Non-trivial and complete, but not particularly innovative. (5/10)
= Significance: Methodologically, probably will influence some work regarding re-ranking methodologically. From a practical perspective, seems very promising. A few more experiments would make this case stronger, but has real-world data. (6/10)
=== Pros ===
+ extends a widely used model (pointer-networks) to the re-ranking setting
+ discusses practical issues in getting this to work at scale
+ shows that it works in a real-world setting 
+ contextualization within existing research shows good understanding of related work
=== Cons ===
- is a fairly direct application of pointer-networks with the innovation being in the details (i.e., is more of an ‘industry’ paper)
- additional experiments around ‘diverse-clicks’ settings (to see how smooth the performance curve) and submodular comparisons may have been interesting
In summary, I think there is room for improvement (some outlined in the conclusion), but is an interesting finding with promise that I plan to try myself. Thus, I lean toward an accept.