Towards Deep Conversational Recommendations
There has been growing interest in using neural networks and deep learning techniques to create dialogue systems. Conversational recommendation is an interesting setting for the scientific exploration of dialogue with natural language as the associated discourse involves goal-driven dialogue that often transforms naturally into more free-form chat. This paper provides two contributions. First, until now there has been no publicly available large-scale dataset consisting of real-world dialogues centered around recommendations. To address this issue and to facilitate our exploration here, we have collected REDIAL, a dataset consisting of over 10,000 conversations centered around the theme of providing movie recommendations. We make this data available to the community for further research. Second, we use this dataset to explore multiple facets of conversational recommendations. In particular we explore new neural architectures, mechanisms, and methods suitable for composing conversational recommendation systems. Our dataset allows us to systematically probe model sub-components addressing different parts of the overall problem domain ranging from: sentiment analysis and cold-start recommendation generation to detailed aspects of how natural language is used in this setting in the real world. We combine such sub-components into a full-blown dialogue system and examine its behavior.
Summary: This paper provides a large-scale data set of 10,000 conversations for the domain of movie recommendation.  Strengths: + The authors will release the first large-scale dataset for conversational movie recommendations, which can be significant for follow-up work on conversational recommendation.   + The authors propose an end-to-end trainable architecture, with sub-components of new neural-based models for sentiment analysis and cold-start recommendations, along with natural language.   Weaknesses: - As this work has the perspective of task-oriented recommendation, it seems that works such as  []  Li, Xiujun, et al. "End-to-end task-completion neural dialogue systems." arXiv preprint arXiv:1703.01008 (2017). are important to include, and compare to, at least conceptually. Also, discussion in general on how their work differs from other chatbox research works e.g.  []  He, Ji, et al. "Deep reinforcement learning with a natural language action space." arXiv preprint arXiv:1511.04636(2015). would be very useful.  -  It is important that the authors highlight the strengths as well as the weaknesses of their released dataset: e.g. what are scenarios under which such a dataset would not work well? are 10,000 conversations enough for proper training?  Similarly, a discussion on their approaches, in terms of things to further improve would be useful for the research community to extend -- e.g.  a discussion on how the domain of movie recommendation can differ from other tasks, or a discussion on the exploration-exploitation trade-off. Particularly, it seems that this paper envisions conversational recommendations as a goal oriented chat dialogue. However, conversational recommendations could be more ambiguous..  - Although it is great that the authors have included these different modules capturing recommendations, sentiment analysis and natural language, more clear motivation on why each component is needed would help the reader. For example, the cold-start setting, and the sampling aspect of it, is not really explained. The specific choices for the models for each module are not explained in detail (why were they chosen? Why is a sentiment analysis model even needed -- can't we translate the like/dislike as ratings for the recommender?)  - Evaluation -- since one of the contributions argued in the paper is "deep conversational recommenders", evaluation-wise, a quantitative analysis is needed, apart from user study results provided (currently the other quantitative results evaluate the different sub-modules independently). Also, the authors should make clearer the setup of how exactly the dataset is used to train/evaluate on the Amazon Turk conversations -- is beam-search used as in other neural language models?  Overall, although I think that this paper is a nice contribution in the domain of movie conversational recommendation, I believe that the authors should better  position their paper, highlighting also the weaknesses/ things to improve in their work, relating it to work on neural dialogue systems, and expanding on the motivation and details of their sub-modules and overall architecture. Some discussion also on how quantitative evaluation of the overall dialogue quality should happen would be very useful.  ==  I've read the authors' rebuttal. It would be great if the authors add some of their comments from the rebuttal in the revised paper regarding the size of the dataset, comparison with goal-oriented chatbots and potential for quantitative evaluation.