BEYOND GREEDY RANKING: SLATE OPTIMIZATION VIA LIST-CVAE
The conventional solution to the recommendation problem greedily ranks individual document candidates by prediction scores. However, this method fails to optimize the slate as a whole, and hence, often struggles to capture biases caused by the page layout and document interdepedencies. The slate recommendation problem aims to directly find the optimally ordered subset of documents (i.e. slates) that best serve users' interests. Solving this problem is hard due to the combinatorial explosion in all combinations of document candidates and their display positions on the page. Therefore we propose a paradigm shift from the traditional viewpoint of solving a ranking problem to a direct slate generation framework. In this paper, we introduce List Conditional Variational Auto-Encoders (List-CVAE), which learns the joint distribution of documents on the slate conditioned on user responses, and directly generates full slates. Experiments on simulated and real-world data show that List-CVAE outperforms popular comparable ranking methods consistently on various scales of documents corpora.
The latest revision is a substantially improved version of the paper. The comment about generalization still feels unsatisfying ("our model requires choosing c* in the support of P(c) seen during training") but could spur follow-up work attempting a precise characterization.
I remain wary of using a neural net reward function in the simulated environment, and prefer a direct simulation of Eqn5. With a non-transparent metric, it is much harder to diagnose whether the observed improvement in List-CVAE indeed corresponds to improved user engagement; or whether the slate recommender has gamed the neural reward function. Transparent metrics (that encourage non-greedy scoring) also have user studies showing they correlate with user engagement in some scenarios.
In summary, I think the paper is borderline leaning towards accept -- there is a novel idea here for framing slate recommendation problems in an auto-encoder framework that can spur some follow-up works.
---
The paper proposes using a variational auto-encoder to learn how to map a user context, and a desired user response to a slate of item recommendations. During training, data collected from an existing recommender policy (user contexts, displayed slate, recorded user response) can be used to train the encoder and decoder of the auto-encoder to map from contexts to a latent variable and decode the latent variable to a slate. Once trained, we can invoke the encoder with a new user context and the desired user response, and decode the resulting latent variable to construct an optimal slate.
A basic question for such an approach is: [Fig2] Why do we expect generalization from the user responses c(r) seen in training to c(r*) that we construct for testing? At an extreme, suppose our slate recommendation policy always picks the same k items and never gets a click. We can optimize Eqn3 very well on any dataset collected from our policy; but I don't expect deploying the VAE to production with c(r*) as the desired user response will give us anything meaningful.
The generalization test on RecSys2015-medium (Fig6d) confirms this intuition. Under what conditions can we hope for reliable generalization?
The comment about ranking evaluation metrics being unsuitable (because they favor greedy approaches) needs to be justified. There are several metrics that favor diversity (e.g. BPR, ERR) where a pointwise greedy scoring function will perform very sub-optimally.  Such metrics are more transparent than a neural network trained to predict Eqn6. See comment above for why I don't expect the neural net trained to predict Eqn6 on training data will not necessarily generalize to the testing regime we care about (at the core, finding a slate recommendation policy is asking how best to change the distribution P(s), which introduces a distribution shift between training and test regimes for this neural net).
There are 2 central claims in the paper: that this approach can scale to many more candidate items (and hence, we don't need candidate generation followed by a ranking step); and that this approach can reason about interaction-effects within a slate to go beyond greedy scoring. For the second claim, there are many other approaches that go beyond greedy (one of the most recent is Ai et al, SIGIR2018 https://arxiv.org/pdf/1804.05936.pdf; the references there should point to a long history of beyond-greedy scoring) These approaches should also be benchmarked in the synthetic and semi-synthetic experiments. At a glance, many neural rankers (DSSM-based approaches) use a nearly identical decoder to CVAE (one of the most recent is Zamani et al, CIKM2018 https://dl.acm.org/citation.cfm?id=3271800; the references there should point to many other neural rankers) These approaches should also be benchmarked in the expts.
This way, we have a more representative picture of the gain of CVAE from a more flexible (slate-level) encoder-decoder; and the gain from using item embeddings to achieve scalability.