An objective of pro-activity in dialog systems is to enhance the usability of conversational agents by enabling them to initiate conversation on their own. While dialog systems have become increasingly popular during the last couple of years, current task oriented dialog systems are still mainly reactive and users tend to initiate conversations. In this paper, we propose to introduce the paradigm of contextual bandits as framework for pro-active dialog systems. Contextual bandits have been the model of choice for the problem of reward maximization with partial feedback since they fit well to the task description. As a second contribution, we introduce and explore the notion of memory into this paradigm. We propose two differentiable memory models that act as parts of the parametric reward estimation function. The first one, Convolutional Selective Memory Networks, uses a selection of past interactions as part of the decision support. The second model, called Contextual Attentive Memory Network, implements a differentiable attention mechanism over the past interactions of the agent. The goal is to generalize the classic model of contextual bandits to settings where temporal information needs to be incorporated and leveraged in a learnable manner. Finally, we illustrate the usability and performance of our model for building a pro-active mobile assistant through an extensive set of experiments.
The paper "CONTEXTUAL MEMORY BANDIT FOR PRO-ACTIVE DIALOG ENGAGEMENT" proposes to address the problem of pro-active dialog engagement by the mean of a bandit framework that selects dialog situations w.r.t. to the context of the system. Authors define a neural archiecture managing memory with the mean of a contextual attention mechanism.
My main concern about this paper is that the proposal is not enough well described. A very large amount of technical details are missing for allowing the reader to understand the model (and reproduce the experiments). The most important ones are about the exploration policies which are not described at all, while it is a very central point of the paper. The only discussion given w.r.t. the exploration policy is a very general overview about Thompson Sampling. But nothing is said about how it is implemented in the case of the proposed model. How is estimated p(\Theta|D) ? Ok  authors give p(\Theta|D) as a product between prior and likelihood. But it is not sufficient to get p(\Theta|D), the evidence should also been considered (for instance by using variational inference). Also, what is the prior of the parameters ? How is distributed r given a,x and \Theta ? 
Also, not enough justification is given about the general idea of the model. Authors should give more intuitions about the mechanism they propose. Figure 2 should be able to help, but no reference to this figure is given in the text, so it is very difficult to extract any information from it. Authors only (roughly) describe  the architecture without justifying their choices.
At last, the experiments really fail at demonstrating the relevance of the approach, as only questionable artificial data is used. On the first hand it appears mandatory to me to consider some (even minimal) experiments on real data for such proposal. On the other one, the simulated data used there cannot correspond to cues to validate the approach since they appear very far from real scenarios: the trajectories do not depend on what is recommended. Ok only the recommended places reveal some reward but it appears not as a sufficiently realistic scenario to me. Also, very too few baselines are considered: only different versions of the proposal and a random baseline are considered. A classical contextual bandit instance (such as LinUCB) would have been a minimum.
Other remarks:
      - the definition of q is not given
      - user is part of the context x  in the bandit section but not after where it is denoted as u.
      - the notion of time window should be more formally described
      - How is built the context is not clear in the experiments section