In this paper we develop an approach based on deep reinforcement learning (DRL) to address dynamic pricing problem on E-commerce platform. We models realworld E-commerce dynamic pricing problem as Markov Decision Process. Environment state are defined with four groups of different business data. We make several main improvements on the state-of-the-art DRL-based dynamic pricing approaches: 1. We first extend the application of dynamic pricing to a continuous pricing action space. 2. We solve the unknown demand function problem by designing different reward functions. 3. The cold-start problem is addressed by introducing pre-training and evaluation using the historical sales data. Field experiments are designed and conducted on real-world E-commerce platform, pricing thousands of SKUs of products lasting for months. The experiment results shows that, on E-commerce platform, the difference of the revenue conversion rates (DRCR) is a more suitable reward function than the revenue only, which is different from the conclusion from previous researches. Meanwhile, the proposed continuous action model performs better than the discrete one.
This paper presents a reinforcement learning approach to dynamically set the price of items on sale on an e-commerce website, based on a state description consisting of several kinds of features: price, sales, customer traffic and competitiveness. The actions (possible prices) are constrained to lie in item-specific lower and upper bounds. Against a proposed method relying a continuous action space (an implementation of the Deep Deterministic Policy Gradient of Lillicrap et al., 2015), alternatives relying on multi-armed bandits and Deep Q Networks are evaluated. Experimental results from an online deployment are presented.
The paper is at times hard to understand. In particular, it would benefit from a thorough review of English grammar and style. For instance, the alternating descriptions between the past and present tenses (e.g. the abstract and the introduction) are quite non-natural and somewhat irritating [see specific instances in the detailed comments, below]. 
From an algorithmic standpoint, the paper does not introduce new methods and relies on well-known reinforcement learning techniques. The proposed methodology of applying specific RL techniques such as DDPG to pricing appears novel. However, there is an abundant literature on optimal pricing and discounting in operations research, much of it based on dynamic programming techniques, and more links to this literature could be provided.
One could question the choice of the reward function chosen (eq. 1): since it is a ratio, it is extremely sensitive to variations in the denominator, and this could severely impact convergence. 
The primary weakness of the paper lies in the experimental evaluation: at least the following informations are missing to truly understand the methodological impact and economic benefits of the proposed approach:
1. A complete description of the experimental setting, i.e. how many SKUs (stock keeping units) were evaluated, over which product categories, over what time horizon. Of those, how are they distributed in the fast-mover / slow-mover plane (Syntetos et al., 2005)? What special events were material during the time period? How many of those were predicted and incorporated into the state representation?
2. One or more tables of results giving expected utility gains over baseline of the proposed methods, along with confidence intervals.
3. For an ICLR submission, there should be additional investigations as to the structure of the learned representations, at least by the DQN and the DDPG models — is the state embedding learned by the networks somewhat suggestive of economically meaningful properties of the items or the sales environment?
As such, even though the paper is interesting, it is in too early a state to recommend acceptance at ICLR.
Detailed comments:
* p. 1: many past tenses that should be in the present tense, e.g. (just in the abstract): modeled ==> models, defined ==> defines, then it introduced ==> it then introduces, were designed ==> are designed. Many other cases in the rest of the paper.
* p. 1: has draw ==> has drawn
* p. 2: Forth ==> Fourth
* p. 2: overtime ==> over time
* p. 2: is assumed, to ==> is assumed to
* p. 2: described ==> describe
* p. 3: “looks deep inside learning while earning approaches” ==> sentence not clear
* p. 3: this is not clear: “since the number of page visitors may ﬂuctuate dramatically and this could lead to non-concavity“ ==> why would it lead to non-concavity?
* p. 4: The whole paragraph before eq. (1) is not clear
* p. 5: The D in eq. (5) should be explained immediately, not after eq. (6).
* p. 5: In eq. (5), $\theta’$ is not explained: how does it differ from $\theta$ ?
* p. 5: In eq. (6), how is $\theta^{Q’}$ different from $\theta^{Q}$ ?
* p. 6: in eq. (8), the denominator r_t could be a small number, leading to a noisy error; this should be discussed.
* p. 6: Below eq. (12), these sentences are not clear: “For dynamic pricing problem, we particularly concern the outcome from changing between prices. To have a well knowledge between two prices before and after pricing.”