To acquire a new skill, humans learn better and faster if a tutor, based on their current knowledge level, informs them of how much attention they should pay to particular content or practice problems. Similarly, a machine learning model could potentially be trained better with a scorer that "adapts" to its current learning state and estimates the importance of each training data instance. Training such an adaptive scorer efficiently is a challenging problem; in order to precisely quantify the effect of a data instance at a given time during the training, it is typically necessary to first complete the entire training process. To efficiently optimize data usage, we propose a reinforcement learning approach called Differentiable Data Selection (DDS). In DDS, we formulate a scorer network as a learnable function of the training data, which can be efficiently updated along with the main model being trained. Specifically, DDS updates the scorer with an intuitive reward signal: it should up-weigh the data that has a similar gradient with a dev set upon which we would finally like to perform well. Without significant computing overhead, DDS delivers strong and consistent improvements over several strong baselines on two very different tasks of machine translation and image classification.
The paper proposes an iterative method that jointly trains the model and a scorer network that places a non-uniform distribution over data sets.  The paper proposes a gradient method to learn the scorer network based on reinforcement learning, which is novel as to what the reviewer knows.
There are several concerns/questions:
1) The paper doesn’t define the D_{dev} clearly. How is D_{dev} chosen? Is it a subset of D_{train}? 
2) In section 2.1, why “smaller development set D_{dev} is much closer to the P_{test}(X,Y)”? P_{test}(X,Y) is supposed to be not observed during training?
3) In Eq (5), if D_{dev} is s subset of D_{train}, if \theta* is the minimal of J, it means the gradient 
at  \theta* is 0. To calculate the gradient of J with respect to \psi, by chain rule, it need to calculate gradient to \theta* first then \theta* to \psi. If gradient of \theta* is 0, the product is also 0? So the \psi will not be updated if D_{dev}  is sufficiently similar to D_{train} ?
4) In Section 2.3, it omits the second order Hessian term. How does that influence the performance? 
5) it mentions “without significant computing overhead“ in abstract, which is not demonstrated elsewhere.
6) In the experiments, table 1, it seems the major improvement comes from retrain and TCS rather than DDS? In figure 3, it is better to show the weights of an image without DDS and comparing that with DDS.
7) The paper contains many typos such as Eqn.11 is not defined in the main paper, the “Eqn ??” Appears in the appendix, “tha minimizes” etc.
In general, the idea of the paper is natural and the results seem promising. I am looking forward to the reply to my questions/concerns. 
#############
I have read the author's feedback. I think the clarity of both methodology and experiment does not reach the acceptance level and would maintain my current rating.