Building robust online content recommendation systems requires learning complex interactions between user preferences and content features. The field has evolved rapidly in recent years from traditional multi-arm bandit and collaborative filtering techniques, with new methods integrating Deep Learning models that enable to capture non-linear feature interactions. Despite progress, the dynamic nature of online recommendations still poses great challenges, such as finding the delicate balance between exploration and exploitation. In this paper we provide a novel method, Deep Density Networks (DDN) which deconvolves measurement and data uncertainties and predicts probability density of CTR (Click Through Rate), enabling us to perform more efficient exploration of the feature space. We show the usefulness of using DDN online in a real world content recommendation system that serves billions of recommendations per day, and present online and offline results to evaluate the benefit of using DDN.
The paper adresses a very interesting question about the handling of the dynamics of a recommender systems at scale (here for linking to some articles).
The defended idea is to use the context to fit a mixture of Gaussian with a NN and to assume that the noise could be additively split into two terms. One depend only on the number of observations of the given context and the average reward in this situation and the second term begin the noise. This is equivalent to separate a local estimation error from the noise. 
The idea is interesting but maybe not pushed far enough in the paper:
*At fixed context x, assuming that the error is a function of the average reward u and of the number of displays r of the context could be a constant could be a little bit more supported (this is a variance explanation that could be tested statistically, or the shape of this 2D function f(u,r) could be plot to exhibit its regularity). 
* None of the experiments is done on public data which lead to an impossible to reproduce paper
* The proposed baselines are not really the state of the art (Factorization Machines, GBDT features,...) and the used loss is MSE which is strange in the context of CTR prediction (logistic loss would be a more natural choice)
* I'm not confident with the proposed surrogate metrics. In the paper, the work of Lihong Li &al on offline evaluation on contextual bandits is mentioned and considered as infeasible here because of the renewal of the set of recommendation. Actually this work can be adapted to handle theses situations (possibly requiring to bootstrap if the set is actually regenerating too fast). Also note that Yahoo Research R6a - R6b  datasets where used in ICML'12 Exploration and Exploitation 3 challenge where about pushing some news in a given context and could be reused to support the proposed approach. An other option would be to use some counterfactual estimates (See Leon Bottou &all and Thorsten Joachims &all)
* If the claim is about a better exploration,  I'd like to have an idea of the influence of the tuning parameters and possibly a discussion/comparison over alternatives strategies (including an epsilon-n greedy algorithm)
Besides theses core concerns, the papers suffers of some imprecisions on the notations which should be clarified. 
* As an example using O(1000) and O(1M) in the figure one. Everyone understands what is meant but O notation are made to eliminate constant terms and O(1) = O(1000).
* For eqn (1) it would be better to refer to and "optimistic strategy" rather to UCB because the name is already taken by an algorithm which is not this one. Moreover the given strategy would achieve a linear regret if used as described in the paper which is not desirable for bandits algorithms (smallest counter example with two arms following a Bernouilli with different parameters if the best arms generates two zero in a row at the beginning, it is now stuck with a zero mean and zero variance estimate). This is why bandits bounds include a term which increase with the total number of plays. I agree that in practice this effect can be mitigated at that the strategy can be correct in the contextual case (but then I'd like to the dependancies on x to be clear) 
* The papers never mentions whats is a scalar, a vector or a matrix. This creates confusion: as an example eqn (3) can have several different meaning depending if the values are scalars, scalars depending on x or having a diagonal \sigma matrix
* In the paragraph above (2) I unsure of what is a "binomial noise error distribution" for epsilon, but a few lines later epsilon becomes a gaussian why not just mention that you assume the presence of a gaussian noise on the parameters of a Bernouilli distribution ?