Unsupervised Learning of Disentangled Representations from Video
We present a new model DRNET that learns disentangled image representations from video. Our approach leverages the temporal coherence of video and a novel adversarial loss to learn a representation that factorizes each frame into a stationary part and a temporally varying component. The disentangled representation can be used for a range of tasks. For example, applying a standard LSTM to the time-vary components enables prediction of future frames. We evaluate our approach on a range of synthetic and real videos, demonstrating the ability to coherently generate hundreds of steps into the future.

This paper presents a neural network architecture and video-based objective function formulation for the disentanglement of pose and content features in each frame. The proposed neural network consists of encoder CNNs and a decoder CNN. The encoder CNNs are trained to decompose frame inputs into contents features (i.e. background, colors of present structures, etc), and pose features (i.e. current configuration of the structures in the video). A decoder CNN is then used to combine the content features of one frame and pose features of a different frame to reconstruct the frame that corresponds to the pose features. This paper presents a new loss for the disentanglement of pose features from input images using a discriminator network (similar to adversarial loss). They train a discriminator network find similarities between features from the pose encoder of images from the same video but different time steps. The pose encoder is then trained to find features that the discriminator cannot distinguish as being similar. At convergence, the pose encoder features will only contain difference between the frames which should reflect the pose change of the person in time. After the encoder-decoder CNN has been trained to identify the content and pose features, an LSTM is trained to take the content features of the last frame and previous pose features to predict the next pose features. The disentangled content and pose features result in high quality video prediction. The experimental results backup the advantage of this method in terms of feature quality for classification tasks, and shows quantitative and qualitative performance boost over the state-of-the-art in video prediction.
Pros:
Novel video-based loss functions for feature disentanglement.
Excellent results on video prediction using the learned pose and content features.
Cons:
Although this paper claims to be an unsupervised method, the proposed objective function actually needs some weak supervision that different videos should have different content, e.g. object instances; otherwise the second term in equation 3 could be negatively affected (classifying images from different videos as different when they may look the same).
The pose LSTM is trained separately from the encoder-decoder CNN. Can the authors comment on why joint end-to-end training was not performed? In fact, similar networks for end-to-end video prediction have been experimented in Oh et al, NIPS 2015.
Some typos in equations and text.
Should equation 3 be L_{adversarial}(C) instead of L_{adversarial}(D)?
h^t_c in LSTM input could be denoted as h^n where n is the last observed frame index, t makes it seem as the current step in generation.
Lines 204 and 219 point to Fig. 4.3 which is not in the main paper.
Figure 4 should have matching test sequences since it expect us to compare the “left”, “middle”, and “right” results.
Action classification experiments are a little confusing:
It is not clear what features are used to determine action class and how they are used (i.e. concatenated features? Single features and average for each video? Classification on clips?). The paper states: “In either case, we train a two layer classifier network S on top of either hc or hp, with its output predicting the class label y.”
In KTH, the content is not a good indicator about the action that is happening in the video since the same people perform all actions in similar backgrounds. Thus it’s not clear how the content features can be useful for action classification.
Classification results of network trained without pose disentanglement loss can make classification results stronger if previous issues are clarified.
Pose estimation results using the disentangled pose features are strongly recommended that will significantly strengthen the paper.
For the SUNCG dataset (3D object renderings), comparison with the weakly-supervised pose disentangling work by Yang et al, NIPS 2015 is suggested to highlight the performance of the proposed method.
The use of the first term in L_{adversarial}(E_p) is not clear to me, it seems that the first term and second term in this loss are pulling the features in completely opposite directions which, to my understanding, is helping D instead of confusing it. Isn’t confusing D the whole purpose of  L_{adversarial}(E_p) to confuse D?
Title can be more specific to the contribution (adversarial-like loss for content-pose feature disentangling)


Efficient Sublinear-Regret Algorithms for Online Sparse Linear Regression with Limited Observation
Online sparse linear regression is the task of applying linear regression analysis to examples arriving sequentially subject to a resource constraint that a limited number of features of examples can be observed. Despite its importance in many practical applications, it has been recently shown that there is no polynomialtime sublinear-regret algorithm unless NP⊆BPP, and only an exponential-time sublinear-regret algorithm has been found. In this paper, we introduce mild assumptions to solve the problem. Under these assumptions, we present polynomialtime sublinear-regret algorithms for the online sparse linear regression. In addition, thorough experiments with publicly available data demonstrate that our algorithms outperform other known algorithms.

The paper considers the online sparse regression problem introduced by Kale (COLT'14), in which the online algorithm can only observe a subset of k features of each data point and has to sequentially predict a label based only on this limited observation (it can thus only use a sparse predictor for each prediction). Without further assumptions, this problem has been recently shown to be computationally hard by Foster et al (ALT'16). To circumvent this hardness, the authors assume a stochastic i.i.d. setting, and under two additional distributional assumptions they give efficient algorithms that achieve sqrt(T) regret.
The results are not particularly exciting, but they do give a nice counter to the recent computational impossibility of Foster et al, in a setting where the data is i.i.d. and well-specified by a k-sparse vector. One of the main things I was missing in the paper is a proper discussion relating its setup, assumptions and results to the literature on sparse recovery / compressed sensing / sparse linear regression. Currently, the paper only discusses the literature and previous work on attribute efficient learning.
In particular, I was not entirely convinced that the main results of the paper are not simple consequences of known results in sparse recovery: since the data is stochastic i.i.d., one could implement a simple follow-the-leader style algorithm by solving the ERM problem at each step; this could be done efficiently by applying a standard sparse recovery/regression algorithm under a RIP condition. It seems that this approach might give sqrt(T)-regret bounds similar to the one established in the paper, perhaps up to log factors, under the authors' Assumption (a) (which is much stronger than RIP: it implies that the data covariance matrix is well conditioned, so the unique linear regression solution is the sparse one).
While the authors do consider an additional assumption which is somewhat weaker than RIP (Assumption b), it is again not properly discussed and related to more standard assumptions in sparse recovery (e.g, RIP), and it was not clear to me how reasonable and meaningful this assumption is.
Questions / comments:
* The "Online" in the title of the paper is somewhat misleading: the setting of the paper is actually a stochastic i.i.d. setting (where the objective is still the regret), whereas "online" typically refers to the worst-case non-statistical online setting.
* The description of Alg 1 is quite messy and has to be cleaned up. For example:
	- line 192, "computing S_t": the entire paragraph was not clear to me. What are the "largest features with respect to w_t"?
	- line 196: what are i,j?
	- line 199, "computing g_t": many t subscripts are missing in the text.
* Thm 2: Why k_1 is arbitrary? Why not simply set it to k'-2 (which seems to give the best regret bound)?
* Can you give a simpler algorithm under Assumption (b) for the easier case where k <= k'+2 (analogous to Alg 1)?
Minors / typos:
* line 14: the references to [11,10] seems out of context.
* line 156, Assumption (a): use an explicit constant in the assumption, rather than O(1); in the analysis you actually implicitly assume this constant is 1.
* line 188: "dual average" => "dual averaging".
* Lem 3 is central to the paper and should be proven in the main text.
* Lem 4 is a standard bound for dual averaging / online gradient descent, and a proper reference should be given. Also, it seems that a simpler, fixed step-size version of the bound suffices for your analysis.
* line 226: "form" => "from".
* line 246: "Rougly" => "Roughly".


PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
Few prior works study deep learning on point sets. PointNet [20]  is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.

As clearly indicated in the title, this paper submission is an extension of the PointNet work of [19], to appear at CVPR 2017. The goal is to classify and segment (3D) point clouds. Novel contributions over [19] are the use of a hierarchical network, leveraging neighbourhoods at different scales, and a mechanism to deal with varying sampling densities, effectively generating receptive fields that vary in a data dependent manner. All this leads to state-of-the-art results.
PointNet++ seems an important extension over PointNet, in that it allows to properly exploit local spatial information. Yet the impact on the overall performance is just 1-2 percent.
Some more experimental or theoretical analysis would have been appreciated. For instance:
- A number of alternative sampling options, apart from most distant point, are mentioned in the text, but not compared against in the experiments.
- In the conclusions, it is mentioned that MSG is more robust than MRG but worse in terms of computational efficiency. I'd like to see such a claim validated by a more in-depth analysis of the computational load, or at least some indicative numbers.


Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
We present a new approach to learn compressible representations in deep architectures with an end-to-end training strategy. Our method is based on a soft (continuous) relaxation of quantization and entropy, which we anneal to their discrete counterparts throughout training. We showcase this method for two challenging applications: Image compression and neural network compression. While these tasks have typically been approached with different methods, our soft-to-hard quantization approach gives results competitive with the state-of-the-art for both.

This paper proposed a unified end-to-end framework for training neural networks to get compressible representations or weights.
The proposed method encourages compressibility by minimizing the entropy of representation or weight.  Because the original entropy-based objective cannot be minimized directly, the authors instead relax the objective and use discretized approximations.
The proposed method also use an annealing schedule to start from a soft discretization to gradually transform to a hard discretization.
My main concerns with this paper are on experiments and comparisons. The authors performed two sets of experiments, one on compressing representations of image and the other on compressing classifier weights.
For the image compression experiment, the proposed method offers visually appealing results but it is only compared to BPG and JPEG. I suggest the authors also include comparisons against other neural network based approaches [1].
In the weight compression experiment, the proposed method does not show advantage over previous state-of-the-art. The authors also noted this but argued that the proposed method is trained end-to-end while the previous methods have many hand designed steps. However, I would expect integrated end-to-end training to improve performance instead of hindering it. I suggest the authors investigate more and improve their method to fully realize the benefit of end-to-end training.
[1] Full Resolution Image Compression with Recurrent Neural Networks


One-Sided Unsupervised Domain Mapping
In unsupervised domain mapping, the learner is given two unmatched datasets A and B. The goal is to learn a mapping G AB that translates a sample in A to the analog sample in B. Recent approaches have shown that when learning simultaneously both G AB and the inverse mapping G BA , convincing mappings are obtained. In this work, we present a method of learning G AB without learning G BA . This is done by learning a mapping that maintains the distance between a pair of samples. Moreover, good mappings are obtained, even by maintaining the distance between different parts of the same sample before and after mapping. We present experimental results that the new method not only allows for one sided mapping learning, but also leads to preferable numerical results over the existing circularity-based constraint. Our entire code is made publicly available at https://github.com/sagiebenaim/DistanceGAN.

This paper tackles the problem of unsupervised domain adaptation. The paper introduces a new constraint, which compares samples and enforces high cross-domain correlation between the matching distances computed in each domain. An alternative to pairwise distance is provided, for cases in which we only have access to one data sample at a time. In this case, the same rationale can be applied by splitting the images and comparing the distances between their left/right or up/down halves in both domains. The final unsupervised domain adaptation model is trained by combining previously introduced losses (adversarial loss and circularity loss) with the new distance loss, showing that the new constraint is effective and allows for one directional mapping.
The paper is well structured and easy to follow. The main contribution is the introduction of a distance constraint to recent state-of-the-art unsupervised domain adaptation pipeline. The introduction of the distance constraint and the assumption held by this constraint, i.e. there is a high degree of correlation between the pairwise distances in the 2 domains, are well motivated with experimental evidence.
Although literature review is quite extensive, [a-c] might be relevant to discuss.
[a] https://arxiv.org/pdf/1608.06019.pdf
[b] https://arxiv.org/pdf/1612.05424.pdf
[c] https://arxiv.org/pdf/1612.02649.pdf
The influence of having a distance loss is extensively evaluated on a variety of datasets for models based on DiscoGAN, by changing the loss function (either maintaining all its components, or switching off some of them).  Qualitative and quantitative results show the potential of the distance loss, both in combination with the cycle loss and on its own, highlighting the possibility of learning only one-sided systems. However, given that the quantitative evaluation pipeline is not robust (i.e. it depends on training a regressor), it is difficult to make any strong claims on the performance of the method.
The authors qualitatively and quantitatively assess their contributions for models based on CycleGAN as well. Quantitative results are showed for SVHN to MNIST mapping. However, among the large variety of mappings shown in the CycleGAN paper, authors only pick the horse to zebra mapping to make the qualitative comparison. Given the subjectivity of this kind of comparison, it would be more compelling if the authors could show some other mappings such as season transfer, style transfer, other object transfigurations or labels2photo/photo2label task. Quantitative evaluation of CycleGAN-based models could be further improved by following the FCN-scores of the CycleGAN paper on the task labels/photo or photo/labels.
Finally, in the conclusions, the claim “which empirically outperforms the circularity loss” (lines 293-294) seems a bit too strong given the experimental evidence.


Practical Locally Private Heavy Hitters
We present new practical local differentially private heavy hitters algorithms achieving optimal or near-optimal worst-case error -TreeHist and Bitstogram. In both algorithms, server running time isÕ(n) and user running time isÕ(1), hence improving on the prior state-of-the-art result of Bassily and Smith [STOC  2015] requiringÕ(n 5/2 ) server time andÕ(n 3/2 ) user time. With a typically large number of participants in local algorithms (n in the millions), this reduction in time complexity, in particular at the user side, is crucial for the use of such algorithms in practice. We implemented Algorithm TreeHist to verify our theoretical analysis and compared its performance with the performance of Google's RAPPOR code.

Private Heavy Hitters is the following problem: a set of n users each has an
element from a universe of size d. The goal is to find the elements that occur
(approximately)
most frequently, but do so under a local differential privacy constraint.
This is immensely useful subroutine, which can be used, e.g. to estimate the
common home pages of users of a browswer, or words commonly used by users of a
phone. In fact, this is the functionality implemented by the RAPPOR system in
chrome, and more recently in use in iphones.
To make things a bit more formal, there is an a desired privacy level that
determines how well one can solve this problem. A \gamma-approximate heavy
hitters algorithm outputs a set of elements that contains most elements with
true relative frequency at least 2\gamma and a negligible number of elements with
relative frequency less than \gamma. The best achievable accuracy is \sqrt{n log
d}/eps, and Bassily and Smith showed matching lower and upper bounds. In that
work, this was achieved at a server processing cost of n^{5/2}. This work gives
two more practical algorithms that achieve the same gamma with a server
processing cost of approximately n^{3/2}.
On the positive side, this is an important problem and algorithms for this
problem are already in largescale use. Thus advances in this direction are
valuable.
On the negative side, in most applications, the target gamma is a separate
parameter (see more detailed comment below) and it would be much more accurate
to treat it as such and present and compare bounds in terms of n, d and gamma.
Clarifying these dependencies better would make the work more relevant and
usable.
Detailed comment:
In practice, the goal is usually to collect several statistics at a small
privacy cost. So a target eps, of say 1, is split between several different
statstics, so that the target gamma is not \sqrt{n log d}/eps, but \sqrt{n log
d}/ k\eps, when we want to collect k statistics. Often this k itself is
determined based on what may be achievable, for a fixed constant gamma, of say
0.01. In short then, the gamma used in practice is almost never \sqrt{n log
d}/eps, but a separate parameter. I believe therefore that the correct
formulation of the problem should treat gamma as a separate parameter, instead
of fixing it at ~\sqrt{n}.


A Linear-Time Kernel Goodness-of-Fit Test
We propose a novel adaptive test of goodness-of-fit, with computational cost linear in the number of samples. We learn the test features that best indicate the differences between observed samples and a reference model, by minimizing the false negative rate. These features are constructed via Stein's method, meaning that it is not necessary to compute the normalising constant of the model. We analyse the asymptotic Bahadur efficiency of the new test, and prove that under a mean-shift alternative, our test always has greater relative efficiency than a previous linear-time kernel test, regardless of the choice of parameters for that test. In experiments, the performance of our method exceeds that of the earlier linear-time test, and matches or exceeds the power of a quadratic-time kernel test. In high dimensions and where model structure may be exploited, our goodness of fit test performs far better than a quadratic-time two-sample test based on the Maximum Mean Discrepancy, with samples drawn from the model.

The kernel Stein discrepancy goodness-of-fit test is based on a kernelized Stein operator T_p such that under the distribution p (and only under the distribution p), (T_p f) has mean 0 for all test functions f in an RKHS unit ball. The maximum mean of (T_p f) over the class of test functions under a distribution q thus provides a measure of the discrepancy between q and p. This quantity is known as the kernel Stein discrepancy and can also be expressed as the RKHS norm of a witness function g.
In this paper, instead of estimating the RKHS norm of g using a full-sample U-statistic (which requires quadratic time to compute) or an incomplete U-statistic (which takes linear time but suffers from low power), the authors compute the empirical norm of g at a finite set of locations, which are either generated from a multivariate normal distribution fitted to the data or are chosen to approximately maximize test power. This produces a linear-time test statistic whose power is comparable to that of the quadratic-time kernel Stein discrepancy in simulations. The authors also analyze the test's approximate Bahadur efficiency.
This paper is well-written and well-organized, and the proposed test dramatically improves upon the computational burden of other tests, such as the maximum mean discrepancy and kernel Stein discrepancy tests, without sacrificing too much power in simulations. I certainly believe this paper is worthy of inclusion in NIPS.
Although this is not often a salient concern in machine learning, in more traditional statistical applications of hypothesis testing, randomized tests engender quite a bit of nervousness; we don't want a rejection decision in a costly clinical trial or program evaluation to hinge on having gotten a lucky RNG seed. For permutation tests or bootstrapped test statistics, we know that the rejection decision is stable as long as enough permutations or bootstrap iterations are performed. For the authors' proposed test, on the other hand, randomization is involved in choosing the test locations v_1 through v_J (or in choosing the initial locations for gradient ascent), and the simulations use a very small value of J (namely, J=5). Furthermore, in order to estimate parameters of the null distribution of the test statistic, a train-test split is needed. Which raises the question: are the test statistic and rejection threshold stable (for fixed data, across different randomizations)? It would be helpful to have additional simulations regarding the stability of the rejection decision for a fixed set of data, and guidelines as to how many train-test splits are required / how large J needs to be to achieve stability.
A minor comment: line 70, switch the order of "if and only if" and "for all f".


Improved Graph Laplacian via Geometric Consistency
In all manifold learning algorithms and tasks setting the kernel bandwidth used construct the graph Laplacian is critical. We address this problem by choosing a quality criterion for the Laplacian, that measures its ability to preserve the geometry of the data. For this, we exploit the connection between manifold geometry, represented by the Riemannian metric, and the Laplace-Beltrami operator. Experiments show that this principled approach is effective and robust.

This paper proposes a method for improving the graph Laplacian by choosing the bandwidth parameter in the heat kernel for the neighborhood graph.  The method uses geometric consistency in the sense that the Riemannian metric estimated with the graph Laplacian should be close to another estimate of the Riemannian metric based on the estimation of the tangent space.  Some experiments with synthetic and real data demonstrate promising results of the proposed method.
The proposed method for choosing bandwidth seems novel and interesting.  It is based on the very natural geometric idea for capturing the Riemannian geometry of the manifold.  There are, however, some concerns also as described below.
- From the experimental results in Table 1, it is not very clear whether the proposed method outperforms the other methods.  In comparison with Rec, the advantage of GC are often minor, and GC^{-1} sometimes gives worse results.  More careful comparison including statistical significance is necessary.
- The authors say that d=1 is a good choice in the method.  The arguments to support this claim is not convincing, however.  It is not clear whether Proposition 3.1 tells that d=1 is sufficient.  More detailed explanations and discussions would be preferable.
- The paper is easy to read, but not necessarily well written.  There are many typos in the paper.  To list a few,
-- l.36: methods are duplicated.
-- eq.(2): it should be minus in the exponential.
-- l. 179: on is duplicated.
-- Caption of Figure 1: have are.


Runtime Neural Pruning
In this paper, we propose a Runtime Neural Pruning (RNP) framework which prunes the deep neural network dynamically at the runtime. Unlike existing neural pruning methods which produce a fixed pruned model for deployment, our method preserves the full ability of the original network and conducts pruning according to the input image and current feature maps adaptively. The pruning is performed in a bottom-up, layer-by-layer manner, which we model as a Markov decision process and use reinforcement learning for training. The agent judges the importance of each convolutional kernel and conducts channel-wise pruning conditioned on different samples, where the network is pruned more when the image is easier for the task. Since the ability of network is fully preserved, the balance point is easily adjustable according to the available resources. Our method can be applied to off-the-shelf network structures and reach a better tradeoff between speed and accuracy, especially with a large pruning rate.

Update: I updated my score after RNN overhead clarification for more recent networks and their remark on training complexity.
The authors propose a deep RL based method to choose a subset of convolutional kernels in runtime leading to faster evaluation speed for CNNs. I really like the idea of combining an RNN and using it to guide network structure. I have some doubts on the overhead of the decision network (see below) would like a comment on that before making my final decision.
-Do your computation results include RNN runtime (decision network), can you comment on the overhead?
The experiments focus on VGG16 which is a very heavy network. In comparison, more efficient networks like GoogleNet may make the RNN overhead significant. Do you have time or flops measurements for system overhead? Finally, as depth increases (e.g. Resnet151) your decision overhead increases as well since the decider network runs at every step.
- Do you train a different RNP for each p? Is 0.1 a magic number?
-The reward structure and formulating feature (in this case kernel) selection as an MDP is not new, check Timely Object Recognition Karayev, Imitation Learning by Coaching, Hal Daume III. However, application of it to internal kernel selection is interesting.
line 33 "Some other samples are more difficult, which require more computational resources. This property is not exploited in previous neural pruning methods, where input samples are treated equally" This is not exactly true check missing references below.
Missing references on sample adaptive inference in DNNs:
-The cascading neural network: building the Internet of Smart Things Leroux et al.
-Adaptive Neural Networks for Fast Test-Time Prediction Bolukbasi et al.
-Changing Model Behavior at Test-Time Using Reinforcement Learning Odena et al.
misc:
table 1 3x column should have Jaderberg as bold (2.3 vs 2.32)
line 158 feature -> future
line 198 p=-0.1 -> p=0.1 ?


Noise-Tolerant Interactive Learning Using Pairwise Comparisons
We study the problem of interactively learning a binary classifier using noisy labeling and pairwise comparison oracles, where the comparison oracle answers which one in the given two instances is more likely to be positive. Learning from such oracles has multiple applications where obtaining direct labels is harder but pairwise comparisons are easier, and the algorithm can leverage both types of oracles. In this paper, we attempt to characterize how the access to an easier comparison oracle helps in improving the label and total query complexity. We show that the comparison oracle reduces the learning problem to that of learning a threshold function. We then present an algorithm that interactively queries the label and comparison oracles and we characterize its query complexity under Tsybakov and adversarial noise conditions for the comparison and labeling oracles. Our lower bounds show that our label and total query complexity is almost optimal.

he authors study the active binary classification setting where
+ The learner can ask the label of an instance. The labelling oracle will provide a noisy answer (with either adversarial noise or Tsybakov)
+ In addition to the labeling oracle, there is a comparison oracle. The leaner then can ask which of two instances is more likely to be positive. This oracle's answers is also noisy.
The authors provide results which show that if the learner uses comparison queries, then it can reduce the label-query complexity. The problem that the authors study seem to be a valid and interesting one. The presentation of the result however should be improved. In the current state, accurately parsing the arguments is hard. Also, it would have been better if a better proof sketch was included in the main paper (to give the reviewer an intuition about the correctness of the results without having to read the supplements).
The core idea is to use the usual active-type algorithm, but then replace the label-request queries with a sub-method that uses mostly comparison queries (along with a few label queries). This is possible by first sorting the instances based on comparison queries and then doing binary search (using label queries) to find a good threshold.
One thing that that concerning is the assumption on the noise level. On one hand, the authors argue that the major contribution of the paper compared to previous work is is handling noise. However, their positive result is not applicable when the amount of noise is more than some threshold (which depends on epsilon and even delta).
Also, the authors do not discuss if the learner is allowed to perform repeated comparison queries about the same pair of instances. This seems to be helpful. [Furthermore, the authors assume that the outcome of the queries are iid, even when they have been asked about overlapping pairs]


Experimental Design for Learning Causal Graphs with Latent Variables
We consider the problem of learning causal structures with latent variables using interventions. Our objective is not only to learn the causal graph between the observed variables, but to locate unobserved variables that could confound the relationship between observables. Our approach is stage-wise: We first learn the observable graph, i.e., the induced graph between observable variables. Next we learn the existence and location of the latent variables given the observable graph. We propose an efficient randomized algorithm that can learn the observable graph using O(d log 2 n) interventions where d is the degree of the graph. We further propose an efficient deterministic variant which uses O(log n + l) interventions, where l is the longest directed path in the graph. Next, we propose an algorithm that uses only O(d 2 log n) interventions that can learn the latents between both nonadjacent and adjacent variables. While a naive baseline approach would require O(n 2 ) interventions, our combined algorithm can learn the causal graph with latents using O(d log 2 n + d 2 log (n)) interventions.

The authors propose theory and algorithms for identifying ancestral relations, causal edges and latent confounders using hard interventions. Their algorithms assume that it is possible to perform multiple interventions on any set of variables of interest, and the existence of an independence oracle, and thus is mostly of theoretical value. In contrast to previous methods, the proposed algorithms do not assume causal sufficiency, and thus can handle confounded systems. The writing quality of the paper is good, but some parts of it could be changed to improve clarity.
General Comments
Several parts of the paper are hard to follow (see below). Addressing the below comments should improve clarity.  In addition, it would be very helpful if further examples are included, especially for Algorithms 1-3. Furthermore, results are stated but no proof is given. Due to lack of space, this may be challenging. For the latter, it would suffice to point to the appendix provided in the supplementary material. Some suggestions to save space: (1) remove the end if/for/function parts of the algorithms, as using appropriate indentation suffices, (2) the “results and outline of the paper” section could be moved to the end of the introduction, and several parts in the introduction and background could be removed or reduced.
Although existing work is mentioned, the connections to previous methods are not always clear. For instance, how do the proposed methods relate to existing methods that assume causal sufficiency? How about methods that do not assume causal sufficiency, but instead make other assumptions? Also, the paper [1] (and references therein) are related to the work but not mentioned.
In the abstract, the authors mention that some experiments may not be technically feasible or unethical, yet this is not addressed by the proposed algorithms. In contrast, the algorithms assume that any set of interventions is possible. This should be mentioned as a limitation of the algorithms. Furthermore, some discussion regarding this should be included. An important question is if and how such restrictions (i.e. some variables can’t be intervened upon) affect the soundness/completeness of the algorithms.
Lines 47-50: The statement regarding identifiability of ancestral relations is incorrect. For some edges in MAGs it can be shown that they are direct, or whether the direct relation is also confounded; see visible edges in [2] and direct causal edges in [3]. If shown to be direct, they are so only in the context of the measured variables; naturally, multiple variables may mediate the direct relation.
Line 58: [4] is able to handle interventional data with latent variables. Other algorithms that are able to handle experimental data with latent variables also exist [5-9].
Definition 1: How is the strong separating system computed? (point to appendix from the supplementary material)
Lemma 3: Shouldn’t also X_i \not\in S?
Algorithm 2, line 7: What is Y? This point is crucial for a correct understanding of the algorithm. I assume there should be an iteration over all Y that are potential children of X, but are not indirect descendants of X.
Lemma 4: For sparse graphs, the transitive closure can be trivially computed in O(n * (n + m)) time (n: number of nodes, m: number of edges) which is faster than O(n^\omega). I mention this as the graphs considered in this paper are sparse (of constant degree).
Line 179: The fact that Tr(D) = Tr(D_{tc}) is key to understanding Lemma 5 and Algorithm 3 and should be further emphasized.
Algorithm 3: It is not clear from the algorithm how S is used, especially in conjunction with Algorithm 1. One possibility is for Algorithm 1 to also always intervene on S whenever intervening on set S_i (that is, intervene on S_i \cup S). Is this correct? This part should be improved.
Algorithm 3: Using S to intervene on a random variable set, and then performing additional interventions using Algorithm 1 raises the question whether a more efficient approach exists which can take into consideration the fact that S has been intervened upon. Any comments on this?
Algorithm 3: It would be clearer if the purpose of c is mentioned (i.e. that it is a parameter which controls the worst case probability of recovering the observable graph).
Line 258: Induced matching is not defined. Although this is known in graph theory, it should be mentioned in the paper as it may confuse the reader.
Do-see test: Although not necessary, some comment regarding the practical application of the do-see test would be helpful, if space permits.
Typos / Corrections / Minor Suggestions
Pa_i is used multiple times but not defined
The longest directed path is sometimes denoted as r and sometimes as l. Use only one of them.
Abstract: O(log^2(n)) -> O(d*log^2(n))
Line 30: … recovering these relations … -> … recovering some relations … (or something along that lines, as they do not identify all causal relations)
Line 59: international -> interventional
Line 105: Did you mean directed acyclic graph?
Line 114: (a) -> (b)
Line 114: … computed only transitive closures … -> … computing using transitive closured …
Line 120: (a) -> (c)
Line 159: Define T_i … -> something wrong here
Line 161: Using a different font type for T_i is confusing. I would recommend using different notation.
Lemma 5, Line 185: V_i \in S^c would be clearer if written as V_i \not in S (unless this is not the same and I missed something)
Line 191: We will show in Lemma 5 -> Shouldn’t it be Theorem 3?
Algorithm 3: \hat{D}(Tr(\hat{D}_S)) looks weird. Consider removing Tr(\hat{D}_S) (unless I misunderstood something).
Lines 226-227: To see the effect of latent path
Line 233: Figure 4.2 -> Figure 2
Line 240: … which shows that when … -> remove that
[1] Hyttinen et al, Experiment Selection for Causal Discovery, JMLR 2013
[2] Zhang, Causal Reasoning with Ancestral Graphs, JMLR 2008
[3] Borboudakis et al, Tools and Algorithms for Causally Interpreting Directed Edges in Maximal Ancestral Graphs, PGM 2012
[4] Triantafillou and Tsamardinos. Constraint-based causal discovery from multiple interventions over overlapping variable sets. JMLR 2015
[5] Hyttinen et al, Causal Discovery of Linear Cyclic Models from Multiple Experimental Data Sets with Overlapping Variables, UAI 2012
[6] Hyttinen et al, Discovering Cyclic Causal Models with Latent Variables: A General SAT-Based Procedure, UAI 2013
[7] Borboudakis and Tsamardinos, Towards Robust and Versatile Causal Discovery for Business Applications, KDD 2016
[8] Magliacane et al, Ancestral Causal Inference, NIPS 2016
[9] Magliacane et al, Joint Causal Inference from Observational and Experimental Datasets, Arxiv


Adversarial Surrogate Losses for Ordinal Regression
Ordinal regression seeks class label predictions when the penalty incurred for mistakes increases according to an ordering over the labels. The absolute error is a canonical example. Many existing methods for this task reduce to binary classification problems and employ surrogate losses, such as the hinge loss. We instead derive uniquely defined surrogate ordinal regression loss functions by seeking the predictor that is robust to the worst-case approximations of training data labels, subject to matching certain provided training data statistics. We demonstrate the advantages of our approach over other surrogate losses based on hinge loss approximations using UCI ordinal prediction tasks.

The paper proposes an adversarial approach to ordinal regression, building upon recent works along these lines for cost-sensitive losses. The proposed method is shown to be consistent, and to have favourable empirical performance compared to existing methods.
The basic idea of the paper is simple yet interesting: since ordinal regression can be viewed as a type of multiclass classification, and the latter has recently been attacked by adversarial learning approaches with some success, one can combine the two to derive adversarial ordinal regression approaches. By itself this would make the contribution a little narrow, but it is further shown that the adversarial loss in this particular problem admits a tractable form (Thm 1), which allows for efficient optimisation. Fisher-consistency of the approach also follows as a consequence of existing results for the cost-sensitive case, which is a salient feature of the approach.
The idea proves effective as seen in the good empirical results of the proposed method. Strictly, the performance isn't significantly better than existing methods, but rather is competitive; it would be ideal if taking the adversarial route led to improvements, but they at least illustrate that the idea is conceptually sound.
Overall, I thus think the paper makes a good contribution. My only two suggestions are:
	o It seems prudent to give some more intuition for why the proposed adversarial approach is expected to result in a surrogate that is more appealing than standard ones (beyond automatically having consistency) -- this is mentioned in the Introduction, but could be reiterated in Sec 2.5.
	o As the basis of the paper is that ordinal regression with the absolute error loss can be viewed as a cost-sensitive loss, it might help to spell out concretely this connection, i.e. specify what exactly the cost matrix is. This would make transparent e.g. Eqn 3 in terms of their connection to the work of (Asif et al., 2015).
Other comments:
- notation E_{P(x, y) \hat{P}(\hat{y} | x)}[ L_{\hat{Y}, Y} ] is a bit confusing -- Y, \hat{Y} are presumably random variables with specified distribution? And the apparent lack of x on the RHS may confuse.
- the sentence "using training samples distributed according to P̃(x, y), which are drawn from P(x, y)" is hard to understand
- Sec 3.2 and elsewhere, may as well define \hat{f} = w \cdot x, not w \cdot x_i; dependence on i doesn't add much here
- Certainly consistency compared to Crammer-Singer SVM is nice, but there are other multi-class formulations which are consistent, so I'm not sure this is the strongest argument for the proposed approach?
- what is the relative runtime of the proposed methods in the experiments?
- it wasn't immediately clear why one couldn't use the existing baselines, e.g. RED^th, in the case of a Gaussian kernel (possibly by learning with the empirical kernel map).


Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Deep neural networks (NNs) are powerful black box predictors that have recently achieved impressive performance on a wide spectrum of tasks. Quantifying predictive uncertainty in NNs is a challenging and yet unsolved problem. Bayesian NNs, which learn a distribution over weights, are currently the state-of-the-art for estimating predictive uncertainty; however these require significant modifications to the training procedure and are computationally expensive compared to standard (non-Bayesian) NNs. We propose an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates. Through a series of experiments on classification and regression benchmarks, we demonstrate that our method produces well-calibrated uncertainty estimates which are as good or better than approximate Bayesian NNs. To assess robustness to dataset shift, we evaluate the predictive uncertainty on test examples from known and unknown distributions, and show that our method is able to express higher uncertainty on out-of-distribution examples. We demonstrate the scalability of our method by evaluating predictive uncertainty estimates on ImageNet.

The paper proposes a simple non-Bayesian baseline method for estimating predictive uncertainty. This is achieved by using ensembles of NNs, where M distinct NNs, from different random initializations, are trained independently through optimizing a scoring rule. The authors simply used a conventional loss function (i.e. softmax cross entropy) as the score rule for classification task, while for regression problems the authors replaced a regular loss function (i.e. MSE) by negative log-likelihood (NLL), which should capture predictive uncertainty estimates. In addition to this, an adversarial training (AT) schema is proposed to promote smoothness over the predictive distribution. To demonstrate the capacity of ensemble and AT to estimate predictive uncertainty, the authors evaluated the proposed method on a large number of datasets across different tasks, both for regression and classification (vision) tasks. Their method outperforms MC dropout for classification tasks while being competitive on regression datasets in term of NLL. Furthermore evaluations demonstrate the ability of the proposed approach to provide uniform predictive distributions for out-of-distribution (unknown) samples, for an object classification task (ImageNet).
The paper proposed the use of NNs ensemble, which trained independently on ordinary training samples and their corresponding adversary, in order to estimate the predictive uncertainty. Although the proposed method is not significantly novel, nor very sophisticated, the paper is contributing to the domain by expliciting an idea that have not been very much exploited so far. However, the results obtained from the experimentations are not able to support well the proposal.
Training NNs by adversaries is one of the contributions of the paper, but the relationship between smoothness and obtaining a well-calibrated predictive uncertainty estimates is a bit vague. How smoothness obtained using AT can help to achieve a well-calibrated predictive uncertainty estimates? In other words, the effectiveness of AT on the predictive uncertainty estimates is not clearly justified, and the experiments are not supporting this well. For examples, the authors showed AT has no considerable effect for regression problems (Table 2 in Appendix), while it has a significant effect for vision classification problems. Why? Moreover, if we consider AT as data augmentation, can we demonstrate the positive effects of AT on the predictive uncertainty over other simple data augmentation approach, such as random crop? I think that having another experiment with simple data augmentation for training NNs and ConvNets can highlight whether the smoothness has a definite effect on the predictive uncertainty or not, and whether AT is truly key to achieve this.
Recognizing erroneous samples, which are confidently misclassified by NNs, is also another concern for AI safety. The high confidence of misclassified samples prohibit a human intervention then some major deficiencies (accidents) can occur. So, the question can be asked that whether this ensemble can recognize the misclassified samples by providing uniform distributions? The experiments should support that.
Estimating the predictive uncertainty is a key for recognizing out-of-distribution and misclassified samples.  However, a few non-Bayesian papers are missed from the literature review of the paper. For example, Hendrycks and Gimpel (2016) propose a baseline method by using softmax statistics to estimate the probability of error and probability of out-of-distribution samples. Also, ensemble of DNNs have been evaluated by Abbasi and Gagné (2017) for computing the uncertainty of the predictions for adversaries cases, a hard case of out-of-distribution examples.
References:
Hendrycks, Dan, and Kevin Gimpel (2016). "A baseline for detecting misclassified and out-of-distribution examples in neural networks." arXiv preprint arXiv:1610.02136.
Abbasi, Mahdieh, and Christian Gagné (2017). "Robustness to Adversarial Examples through an Ensemble of Specialists." arXiv preprint arXiv:1702.06856.
** Update following reviewers discussions **
I likely underappreciated the paper, re-reading it in the light of other reviews and discussions with other reviewers, I increased my score to 6. I still maintain my comments on AT, which I think is the weakest part of the paper.


Discriminative State-Space Models
We introduce and analyze Discriminative State-Space Models for forecasting nonstationary time series. We provide data-dependent generalization guarantees for learning these models based on the recently introduced notion of discrepancy. We provide an in-depth analysis of the complexity of such models. We also study the generalization guarantees for several structural risk minimization approaches to this problem and provide an efficient implementation for one of them which is based on a convex objective.

Synopsis -------------------
* This paper follows in a line of work from Kuznetsov and Mohri on developing agnostic learning guarantees for time series prediction using sequential complexity measures. Ostensibly, the setup in past works and the present work is that one will fit a hypothesis to observations (x_t,y_t) from time 1 through T, then use this hypothesis to predict y_{T+1} from x_{T+1} (the generalization guarantees provided are slightly more general than this). Where previous works (e.g. "Learning Theory and Algorithms for Forecasting Non-Stationary Time Series", NIPS 2015) considered the case where the hypothesis class is a fixed mapping from context space to outcome space, the present work extends this setup to the case where the setting where one learns a "state space" model. Here, the each hypothesis is a pair (h,g), where g generates an internal "state" variable and h predicts using both a context and state variable (see the paper for more detail). This captures, for example, HMMs. The authors give basic generalization guarantees for this setup, generalization for the setup under structural risk minimization, and an algorithm for learning a restricted class of state space models.
Review  -------------------
* I found the concept for the paper exciting and believe it is well-written but I feel it has two major shortcomings:
I) Technical novelty
* The main results Theorem 1 and 2 should be compared with Theorem 1/Corollary 2 of the NIPS 2015 paper mentioned above (call it KM15 for short). The authors mention that the main advantage of the present work is that the discrepancy term in Theorem 2 does not include a sup over the class G, which one would get by applying Theorem 1 from KM15 naively. Yet, if we look at the proof of Theorem 1 Theorem 2 of the present work, we see that this is achieved by being careful with where one takes the sup over G (line 418), then immediately applying to Lemma 1, whose proof is essentially identical to that of KM15 Theorem 1. To give some more examples: Appendix B largely consists of standard facts about sequential rademacher complexity. It is unclear to me whether the structural risk minimization results represent a substantive improvement on the basic generalization bounds. While I admit that others may find these results practically useful, I feel that the applications and experiments should have been fleshed out further if this was the goal.
 II) General setting is not instantiated sufficiently.
* I was not satisfied with the depth to which the general bounds (eg Theorem 2) were instantiated. I do not feel that the basic promise of the paper (learning state space models) has been fully explored. Namely
** The algorithms section (5) only applies in the setting where the state space mapping class G is a singleton, and so there is no real "learning" of the state space dynamics going on. While I acknowledge that developing efficient algorithms for learning G may be challenging, this bothers me from a conceptual perspective because Theorem 2 bears no improvement over the KM15 results mentioned above when G only contains a single element.
** In line with this comment, the discrepancy term never seems to be instantiated.
** While Theorem 2 depends on the expected sequential cover of the class H x G, which one expects should exhibit some rich dependence on the dynamics G and the distribution over (X,Y), Appendix B seems to handle this term by immediately bounding it by the *worst case* sequential rademacher complexity, which includes a sup over all state space paths, not just those that could be induced by G.
Conclusion -------------------
* In summary, I found the basic concept fairly interesting and think that the tools introduced (eg sequential covering for state space models) will be useful, but I don't think they were explored to enough depth. I also found Assumption 1 and the concrete cases where the discrepancy can be removed in the SRM setting to form a nice set of observations.
* One last comment: While you allude that some notion of discrepancy is necessary for learning based on some results of Long, it would be nice to see a formal lower bound whose functional form depends on this quantity in the general case.
Misc comments:
* Line 123: Definition of \mathcal{R}_{s} doesn't exactly match the formal specification for the class \mathcal{R} used in the definition of the sequential covering number.


A New Theory for Matrix Completion
Prevalent matrix completion theories reply on an assumption that the locations of the missing data are distributed uniformly and randomly (i.e., uniform sampling). Nevertheless, the reason for observations being missing often depends on the unseen observations themselves, and thus the missing data in practice usually occurs in a nonuniform and deterministic fashion rather than randomly. To break through the limits of random sampling, this paper introduces a new hypothesis called isomeric condition, which is provably weaker than the assumption of uniform sampling and arguably holds even when the missing data is placed irregularly. Equipped with this new tool, we prove a series of theorems for missing data recovery and matrix completion. In particular, we prove that the exact solutions that identify the target matrix are included as critical points by the commonly used nonconvex programs. Unlike the existing theories for nonconvex matrix completion, which are built upon the same condition as convex programs, our theory shows that nonconvex programs have the potential to work with a much weaker condition. Comparing to the existing studies on nonuniform sampling, our setup is more general.

The authors study matrix completion from a few entries.  They propose a new
proprty of (matrix,sampling pattern) which they term "Isomery". They show that
the well known (incoherence,low rank,random sampling) assumption implies this
condition. The authors then consider a nonconvex bilinear program for matrix
completion. Under the Isomery assumption, they prove that the exact solution is
a critical point of this program (with no assumptions on rank, it seems).  Some
empirical evidence are presented, whereby the nonconvex program is superior to
the convex Nuclear Norm minimization method for a certain pattern of nonuniform
sampling.
The paper is relatively well written (see comments on English below). The
results are novel as far as I can tell, and interesting. This work touches on
fundamental aspects of matrix recovery, and can certainly lead to new research.
There are a few major comments I'd be happy to see addressed. That said, I vote
 accept.
Major comments:
1. The authors say nothing about the natural questions: Can the Isometry
condition be verified in practice? Is it a reasonable assumption to make?
2. The authors discuss in the introduction reasons why random sampling may not
be a reasonable assumption, with some real data examples. It would be nice if,
for a real full data matrix, and a sampling pattern of the kind shown, the
    authors can verify the Isomry assumption.
3. The matrix recovery problem, as well as all the algorithms discussed in this
paper, are invariant under row and col permutations. The authors discuss
sampling patterns which are "concentrated around the main diagonal" - this
condition is not invariant under permutations. Only invariant conditions make
sense.
4. The main contributions claimed (l.79) are not backed up by the paper itself.
Specifically,
    - it's unclear what is meant by "our theories are more flexible and
powerful"
    - the second bullet point in the main contribution list (l.83) referes
 to "nonconvex programs" in general
    - The third bullet point (l.87) is not actuallty proved, and should not be
      advertised as a main contribution.  The authors
  proved Theorem 3.1, but its connection to matrix completion is only vageuly described
  in the paragraph above Theorem 3.1.
minor comments:
- English throughout
- Theorem 3.2: formally define "with high probability"
- The word "theory" and "theories" in the title and throughout: I'm a non-native English speaker, yet I don't see
why you use the word.
- l.56: "incoherent assumption" -> "incoherence assumption"
- l.75 "by the commonly used" -> "of the commonly used"
- l.79 "hypothesis" -> condition
- l.92, l.129 "proof processes" -> "proofs"
- l.188 "where it denotes" -> "where we denote"
- l.208 "that" -??
- Eq. (8) and Eq. (9) - maybe missing =0 on the right hand side?
- Figure 2: label vertical axis
- what do you mean by "proof of theory" (eg line 92)?


Affine-Invariant Online Optimization and the Low-rank Experts Problem
We present a new affine-invariant optimization algorithm called Online Lazy Newton. The regret of Online Lazy Newton is independent of conditioning: the algorithm's performance depends on the best possible preconditioning of the problem in retrospect and on its intrinsic dimensionality. As an application, we show how Online Lazy Newton can be used to achieve an optimal regret of order √ rT for the low-rank experts problem, improving by a √ r factor over the previously best known bound and resolving an open problem posed by Hazan et al. [15].

Summary:
This paper proposes a new optimization algorithm, Online Lazy Newton (OLN), based on Online Newton Step (ONS) algorithm. Unlike ONS which tries to utilize curvature information within convex functions, OLN aims at optimizing general convex functions with no curvature.  Additionally, by making use of low rank structure of the conditioning matrix, the authors showed that OLN yields better regret bound under certain conditions. Overall, the problem is well-motivated and the paper is easy to follow.

Major Comments:
1. The major difference between OLN and ONS is that ONS introduces a lazy evaluation step, which accumulates the negative gradients at each round.  The authors claimed in Lines 155-157 that this helps in decoupling between past and future conditioning and projections and it is better in the case when transformation matrix is changing between rounds.  It would be better to provide some explanations.

2. Lines 158-161, it is claimed that ONS is not invariant to affine transformation due to its initialization.  In my understanding, the regularization term is added partly because it allows for an invertible matrix and can be omitted if Moore-Penrose pseudo-inverse is used as in the FTAL.

3. Line 80 and Line 180, it is claimed that  O(\sqrt(r T logT)) is improved upon O(r \sqrt(T)). The statement will hold under the condition that r/logT = O(1), is this always true?

Minor:
1. Lines 77-79, actually both ONS and OLN utilizes first-order information to approximate second-order statistics.

2. Line 95, there is no definition of the derivative on the right-hand-side of the equation prior to the equation.

3. Line 148-149, on the improved regret bound.  Assuming that there is a low rank structure for ONS (similar to what is assumed in OLS), would the regret bound for OLS still be better than ONS?

4. Line 166, 'In The...' -> 'In the...'

5. Line 181, better to add a reference for Hedge algorithm.

6. Line 199,  what is 'h' in '... if h is convex ...'?

7. Line 211-212,  '...all eigenvalues are equal to 1, except for r of them...', why it is the case?  For D = I + B defined in Line 209, the rank for B is at most r, then at least r of the eigenvalues of D are equal to 1.

The regret bound depends on the low rank structure of matrix A as in Theorem 3, another direction that would be interesting to explore is to consider the low rank approximation to the matrix A and check if similar regret bound can be derived, or under which conditions similar regret bound can be derived.  I believe the proposed methods will be applicable to more general cases along this direction.


An Empirical Bayes Approach to Optimizing Machine Learning Algorithms
There is rapidly growing interest in using Bayesian optimization to tune model and inference hyperparameters for machine learning algorithms that take a long time to run. For example, Spearmint is a popular software package for selecting the optimal number of layers and learning rate in neural networks. But given that there is uncertainty about which hyperparameters give the best predictive performance, and given that fitting a model for each choice of hyperparameters is costly, it is arguably wasteful to "throw away" all but the best result, as per Bayesian optimization. A related issue is the danger of overfitting the validation data when optimizing many hyperparameters. In this paper, we consider an alternative approach that uses more samples from the hyperparameter selection procedure to average over the uncertainty in model hyperparameters. The resulting approach, empirical Bayes for hyperparameter averaging (EB-Hyp) predicts held-out data better than Bayesian optimization in two experiments on latent Dirichlet allocation and deep latent Gaussian models. EB-Hyp suggests a simpler approach to evaluating and deploying machine learning algorithms that does not require a separate validation data set and hyperparameter selection procedure.

# Update after author feedback
I maintain my assessment of the paper. I strongly urge the authors to improve the paper on the basis of the reviewers' input; this could turn out to become a very nice paper.
Specially important IMHO
- retrain on validation+train (will come as an obvious counter-argument)
- clarify the use of $\lambda$; R2 and R3 got confused
- variance in experiments table 2+3 (again, would be an easy target for critique once published)
# Summary of paper
Bayesian optimisation (BO) discards all but the best performance evaluations over hyperparameter sets; this exposes to overfitting the validation set, and seems wasteful. The proposed method "averages" over choices hyperparameters $\eta$ in a Bayesian way, by treating them as samples of a posterior and integrating over them.
This requires introducing an extra layer of hyper-hyperparameters (introducing $\lambda$, subjected to empirical Bayes optimisation).
An algorithmic choice is required to define an "acquisition strategy" in BO terms, ie a model for $r^{(s)}$, as well as a model for hyperprior $p(\eta | \lambda)$.
The proposed method does not require a separate validation set.
# Evaluation
The paper is very clearly written, without any language issues. It represents the literature and the context of the methods well. The problem statement and resolution process is clear, as is the description of the adopted solutions and approximations. The authors take care in preventing misunderstandings in several places.
The discussion of convergence is welcome.
The experiments are non-trivial and help the argumentation, with the caveat of the significance of results mentioned above.
The method is novel, it applies the idea of EB and hierarchical Bayes to the BO method. I believe the approach is a good, interesting idea.
# Discussion
Discussion lines 44-47 and 252 sqq: it may sound like the case presented here was cherry-picked to exhibit a "bad luck" situation in which the leftmost (in fig 1) hyperparameter set which is selected by BO also exhibits bad test error; and in a sense, it is indeed "bad luck", since only a minority of hyperparameter choices plotted in fig 1 seem to exhibit this issue. However, dealing with this situation by averaging over $\eta$'s seems to be the central argument in favour of the proposed EB-Hyp method. I buy the argument, but I also recognise that the evidence presented by fig 1 is weak.
The following advantage does not seem to be discussed in the paper: since a separate validation set is not needed, the training set can be made larger. Does this explain the observed performance increase? An analytical experiment could help answer this. Does the comparison between BO with/without validation in tables 2 and 3 help?
In algo 1, I am not clear about the dependence on $\hat{\lambda}$. I am assuming it is used in the line "return approximation to ..." (I suggest you write $p(X^*|X_\textrm{train}, \hat{\lambda})$ to clarify the dependence.)
I assume that in line "find optimal...", we actually use, per lines 200-208, $\hat{\lambda} = \arg \max \frac{1}{T} \sum_s p(\eta ^{(s)} | \lambda) f^{(s)}_X $. This is worth clarifying.
But then, I'm confused what $p(\eta)$ in line "calculate hyperparameter posterior..." and eq 5 is? Is this an input to the algorithm?
EB-Hyp seems to carry the inconvenient that we must consider, as the "metric of interest" in BO terms, the predictive likelihood $f^{(s)}_X$, while BO can use other metrics to carry out its optimisation. This restriction should be discussed.
## Experiments
How many steps/ evaluations of $A$ do we use in either experiment? We are using the same number for all configurations, I suppose? How does the performance increase with the number of steps? Does EB-Hyp overtake only after several steps, or is its performance above other methods from the start?
I find it hard to assign significance to the improvements presented in table 2 and 3, for lack of a sense of performance variance. The improvements are treated as statistically significant, but eg are they immune against different random data splits ? As it stands, I am not convinced that the performance improvement is real.
# Suggestions for improvement, typos
## Major
Fig 2 is incomplete: it is missing a contour colour scale. In the caption, what is a "negative relationship"? Please clarify. What is the point of fig 2 ? I can't see where it is discussed?
## Minor
fig 1a caption: valiation
fig 1: make scales equal in a vs b
algo 1: increment counter $s$. Number lines.
line 228: into into
line 239: hyerparameters


Beyond Worst-case: A Probabilistic Analysis of Affine Policies in Dynamic Optimization
Affine policies (or control) are widely used as a solution approach in dynamic optimization where computing an optimal adjustable solution is usually intractable. While the worst case performance of affine policies can be significantly bad, the empirical performance is observed to be near-optimal for a large class of problem instances. For instance, in the two-stage dynamic robust optimization problem with linear covering constraints and uncertain right hand side, the worst-case approximation bound for affine policies is O( p m) that is also tight (see Bertsimas and   Goyal [8]), whereas observed empirical performance is near-optimal. In this paper, we aim to address this stark-contrast between the worst-case and the empirical performance of affine policies. In particular, we show that affine policies give a good approximation for the two-stage adjustable robust optimization problem with high probability on random instances where the constraint coefficients are generated i.i.d. from a large class of distributions; thereby, providing a theoretical justification of the observed empirical performance. On the other hand, we also present a distribution such that the performance bound for affine policies on instances generated according to that distribution is ⌦( p m) with high probability; however, the constraint coefficients are not i.i.d.. This demonstrates that the empirical performance of affine policies can depend on the generative model for instances.

Review 2 after authors' comments:
I believe the authors gave a very good rebuttal to the comments made which leads me to believe that there is no problem in accepting this paper, see updated rating.
------------
This paper addresses the challenging question of giving bounds for affine policies for adjustable robust optimization models. I like the fact that the authors (in some probabilistic sense) reduced the large gap of sqrt(m) to 2 for some specific instances. The authors have combined various different techniques combining probabilistic bounds to the structure of a set, which is itself derived using a dual-formulation from the original problem. However, I believe the contributions given in this paper and the impact is not high enough to allow for acceptance for NIPS. I came to this conclusion due to the following reasons:
- Bounds on the performance of affine policies have been described in a series of earlier papers. This paper does not significantly close the gap in my opinion.
- The results strongly depend on the generative model for the instances. However, adjustable robust optimization is solely used in environments that have highly structured models, such as the ones mentioned in the paper on page 2, line 51: set cover, facility location and network design problems. It is also explained that the performance differs if the generative model, or the i.i.d. assumption is slightly changed. Therefore, I am not convinced about the insights these results give for researchers that are thinking of applying adjustable robust optimization to solve their (structured) problem.
- Empirical results are much better than the bounds presented here. In particular, it appears that affine policies are near optimal. This was known and has been shown in various other papers before.
- On page 2, lines 48-49 the authors say that "...without loss of generality that c=e and d=\bar{d}e (by appropriately scaling A and B)." However, I believe you also have to scale the right-hand side h (or the size/shape of the uncertainty set). And the columns of B have to be scaled, making the entries no longer I.I.D. in the distribution required in Section 2?
- Also on page 2, line 69 the authors describe the model with affine policies. The variables P and q are still in an inner minimization model. I think they should be together with the minimization over x?
- On page 6, Theorem 2.6. The inequality z_AR <= z_AFF does always hold I believe? So not only with probability 1-1/m?
- There are a posteriori methods to describe the optimality gap of affine policies that are much tighter for many applications, such as the methods described by [Hadjiyiannis, Michael J., Paul J. Goulart, and Daniel Kuhn. "A scenario approach for estimating the suboptimality of linear decision rules in two-stage robust optimization." Decision and Control and European Control Conference (CDC-ECC), 2011 50th IEEE Conference on. IEEE, 2011.] and [Kuhn, Daniel, Wolfram Wiesemann, and Angelos Georghiou. "Primal and dual linear decision rules in stochastic and robust optimization." Mathematical Programming 130.1 (2011): 177-209.].
-------
I was positively surprised by the digitized formulation on page 8. Is this approach used before in the literature, and if so where? The authors describe that the size can depend on desired accuracy. With the given accuracy, is the resulting solution a lower bound or an upper bound? If it is an upper bound, is the solution feasible? Of course, because it is a MIP it can probably only solve small problems as also illustrated by the authors.
-------
I have some minor comments concerning the very few types found on page 5 (this might not even be worth mentioning here):
- line 166. "peliminary"
- line 184. "affinly independant"
- line 186. "The proof proceeds along similar lines as in 2.4." (<-- what does 2.4 refer to?)


Doubly Accelerated Stochastic Variance Reduced Dual Averaging Method for Regularized Empirical Risk Minimization
We develop a new accelerated stochastic gradient method for efficiently solving the convex regularized empirical risk minimization problem in mini-batch settings. The use of mini-batches has become a golden standard in the machine learning community, because the mini-batch techniques stabilize the gradient estimate and can easily make good use of parallel computing. The core of our proposed method is the incorporation of our new "double acceleration" technique and variance reduction technique. We theoretically analyze our proposed method and show that our method much improves the mini-batch efficiencies of previous accelerated stochastic methods, and essentially only needs size √ n mini-batches for achieving the optimal iteration complexities for both non-strongly and strongly convex objectives, where n is the training set size. Further, we show that even in non-mini-batch settings, our method achieves the best known convergence rate for non-strongly convex and strongly convex objectives.

The paper proposes a novel doubly accelerated variance reduced dual averaging method for solving the convex regularized empirical risk minimization problem in mini batch settings. The method essentially can be interpreted as replacing the proximal gradient update of APG method with the inner SVRG loop and then introducing momentum updates in inner SVRG loops. Finally to allow lazy updated, primal SVRG is replaced with variance reduce dual averaging. The main difference from AccProxSVRG is the introduction of momentum term at the outer iteration level also. The method requires only O(sqrt{n}) sized mini batches to achieve optimal iteration complexities for both convex and non-convex functions when the problem is badly conditioned or require high accuracy. Experimental results show substantial improvements over state of the art under the above scenario.
Overall, given the theoretical complexity of the paper, it is very well written and explained. Relation with the previous work and differences are clear and elaborative. The reviewer thinks that the paper makes substantial theoretical and algorithmic advances. However, I am giving paper relatively lower rating due to following reasons:
a)  I would like to see more experiments on different datasets especially the effect of conditioning and regularizer.
b)  The accuracy curves are not shown in the experiments. This is important in this case because it seems that major differences between methods are visible only at lower values of the gap and it would be interesting to see if accuracy has saturated by then which will make the practical utility limited.
c) Instead of x axis as gradient evaluations, I would like to see the actual wall clock time there because the constants or O(1) computation in inner loop matters a lot. Especially, the double accelerated equations look computationally more tedious and hence I would like to see the time axis there.
d) Looking at lazy updates in supplementary, I am not sure why accelerated SVRG cannot have lazy updates. Can authors exactly point out the step in detail which will have the issues.


Generalization Properties of Learning with Random Features
We study the generalization properties of ridge regression with random features in the statistical learning framework. We show for the first time that O(1/ √ n) learning bounds can be achieved with only O( √ n log n) random features rather than O(n) as suggested by previous results. Further, we prove faster learning rates and show that they might require more random features, unless they are sampled according to a possibly problem dependent distribution. Our results shed light on the statistical computational trade-offs in large scale kernelized learning, showing the potential effectiveness of random features in reducing the computational complexity while keeping optimal generalization properties.

This is in my opinion an excellent paper, a significant theoretical contribution to understanding the role of the well established random feature trick in kernel methods. The authors prove that for a wide range of optimization tasks in machine learning random feature based methods provide algorithms giving results competitive (in terms of accuracy) to standard kernel methods with only \sqrt{n} random features (instead of linear number; this provides scalability). This is according to my knowledge, one of the first result where it is rigorously proven that for downstream applications (such as kernel ridge regression) one can use random feature based kernel methods with relatively small number of random features (the whole point of using the random feature approach is to use significantly fewer random features than the dimensionality of a data). So far most guarantees were of point-wise flavor (there are several papers giving upper bounds on the number of random features needed to approximate the value of the kernel accurately for a given pair of feature vectors x and y but it is not clear at all how these guarantees translate for instance to risk guarantees for downstream applications). The authors however miss one paper with very relevant results that it would be worth to compare with theirs. The paper I talk about is: "Random Fourier Features for Kernel Ridge Regression: Approxiamation Bounds and Statistica Guarantees" (ICML'17).
In this paper the authors work on exactly the same problem and derive certain bounds on the number of random features needed, but it is not clear for me how to obtain from these bounds (that rely on certain spectral guarantees derived in that paper) the \sqrt{n} guarantees obtained in this submission. Thus definitely I strongly advice relating to that paper in the final version of this draft. Anyway, the contribution is still very novel. Furthermore, this paper is very nicely written, well-organized and gives an excellent introduction to the problem. What is most important, the theoretical contribution is to my opinion huge. To sum it up, very good work with new theoretical advances in this important field.


Near Optimal Sketching of Low-Rank Tensor Regression
We study the least squares regression problem
where ⇥ is a low-rank tensor, defined as
. Here, denotes the outer product of vectors, and A(⇥) is a linear function on ⇥. This problem is motivated by the fact that the number of parameters in ⇥ is only R · P
p d number of parameters in ordinary least squares regression. We consider the above CP decomposition model of tensors ⇥, as well as the Tucker decomposition. For both models we show how to apply data dimensionality reduction techniques based on sparse random projections 2 R m⇥n , with m ⌧ n, to reduce the problem to a much smaller problem min ⇥ k A(⇥) bk 2 2 , for which k A(⇥) bk 2 2 = (1± ")kA(⇥) bk 2 2 holds simultaneously for all ⇥. We obtain a significantly smaller dimension and sparsity in the randomized linear mapping than is possible for ordinary least squares regression. Finally, we give a number of numerical simulations supporting our theory.
where ⇥ is a low-rank tensor, defined as ⇥ = P p d number of parameters in ordinary least squares regression. We consider the above CP decomposition model of tensors ⇥, as well as the Tucker decomposition. For both models we show how to apply data dimensionality reduction techniques based on sparse random projections 2 R m⇥n , with m ⌧ n, to reduce the problem to a much smaller problem min ⇥ k A(⇥) bk holds simultaneously for all ⇥. We obtain a significantly smaller dimension and sparsity in the randomized linear mapping than is possible for ordinary least squares regression. Finally, we give a number of numerical simulations supporting our theory.

*Summary*
This paper studies the tensor L2 regression problem and proposed to use matrix sketching to reduce computation. 1+epsilon bound is established to guarantee the error incurred by sketching.
I don't like the writing. First, there is little discussion of and comparison with related work. Second, the proof in the main body of the paper is messy, and it takes much space which could have been used for discussion and experiments. Third, the use of notation is confusing.
Some claims in the paper seems false or suspicious. I'd like to see the authors' response. I'll adjust my rating accordingly.
*Details*
I'd like to see the authors' response to Questions 2, 4, 9, 10.
1. I don't find the description of your algorithm anywhere. I don't find the time complexity analysis anywhere. I want to see a comparison of the computational complexities w/ and w/o sketching.
2. How is this paper related to and different from previous work, e.g., [32], Drineas's work on sketched OLS, and tensor sketch? For example, if A is matrix, which is a special case of tensor, then how does your bounds compare to Drineas's and Clarkson&Woodruff's?
3. The notation is not well described. I look back and forth for the definition of D, p, r, R, m, s, etc.
4. In Line 67, it is claimed that no assumption on incoherence is made. However, in Line 169 and 231, there are requirements on the max leverage scores. How do you explain this?
5. There are incompleteness and errors in the citations.
* There are numerous works on sketched OLS. These works should be cited telow Eqn (6).
* In Definition 5: SJLT is well known as count sketch. Count sketch has been studied in numerous papers, e.g,
Charikar et al: Finding frequent items in data streams.
Clarkson & Woodruff: Low rank approximation and regression in input sparsity time
Meng and Mahoney: Low-distortion subspace embeddings in input- sparsity time and applications to robust linear regression
Nelson & Nguyen: Osnap: Faster numerical linear algebra algorithms via sparser subspace embeddings.
Pham and Pagh. Fast and scalable polynomial kernels via explicit feature maps
Thorup and Zhang. Tabulation-based 5-independent hashing with appli- cations to linear probing and second moment estimation
I don't think [11] is relevant.
6. How does this work extend to regularized regression?
7. Line 182: How is the assumption mild? The denominator is actually big. Matrix completion is not a good reference; it's a different problem.
8. The synthetic data in the experiments are not interesting. Sampling from normal distribution ensures incoherence. You'd better generate data from t-distribution according to the paper
* Ma et al: A statistical perspective on algorithmic leveraging.
I'd like to see real data  experiments to demonstrate the usefulness of this work.
9. The title claims "near optimal". Why is this true? Where's your lower bound?
10. Tensors are rarely low-rank; but sometimes they are approximately low-rank. How does your theory and algorithm apply in this case?
=== after feedback ===
I appreciate the authors' patient reply. I think my evaluation of the technical quality is fair. I won't change my rating.


Tractability in Structured Probability Spaces
Recently, the Probabilistic Sentential Decision Diagram (PSDD) has been proposed as a framework for systematically inducing and learning distributions over structured objects, including combinatorial objects such as permutations and rankings, paths and matchings on a graph, etc. In this paper, we study the scalability of such models in the context of representing and learning distributions over routes on a map. In particular, we introduce the notion of a hierarchical route distribution and show how they can be leveraged to construct tractable PSDDs over route distributions, allowing them to scale to larger maps. We illustrate the utility of our model empirically, in a route prediction task, showing how accuracy can be increased significantly compared to Markov models.

This paper looks at the problem of representing simple routes on a graph as a probability distribution using Probabilistic Sentential Decision Diagrams (PSDDs). Representing a complex structure such as a graph is difficult, and the authors transform the problem by turning a graph into a Boolean circuit where it is straightforward to perform inference, and as an experiment, use their method on a route prediction method for San Francisco taxi cabs, where it beats two baselines.
PSDDs refer to a framework that represents probability distributions over structured objects through Boolean circuits. Once the object is depicted as a Boolean circuit, it becomes straightforward to parameterize it. More formally, PSDD’s are parameterized by including a distribution over each or-gate, and PSDD’s can represent any distribution (and under some conditions, this distribution is unique). The authors focus on the specific problem of learning distributions over simple routes — those that are connected and without cycles. The advantage of SDD circuits is that they have been shown to work on graphs that would be computationally expensive to model with Bayesian nets. With large maps, tractability is still a problem with PSDDs, and the authors do this by considering hierarchical approximations; they break up the map into hierarchies so that representing distributions is polynomial when constraining the size of each region to be a certain size.
The paper is well-written, and the authors define PSDD's clearly and succinctly.
To me, the results of the paper seems incremental -- the authors apply an existing representation method to graphs, and after some approximations, apply the existing inference techniques. Additionally, the paper spends a couple of pages listing theorems, which appear to be basic algorithmic results.
Additionally, I'm not convinced by the baselines chosen for the experiment. For a task of predicting edges given a set of previous edges, the baselines are a naive model which only looks at frequencies and a Markov model that only considers the last edge used. I imagine a deep learning approach or even a simple probabilistic diagram could've yielded more interesting and comparable results. Even if these would require more time to train, it would be valuable to compare model complexity in addition to accuracy. I would need to see more experiments to be convinced by the approach.


Shallow Updates for Deep Reinforcement Learning
Deep reinforcement learning (DRL) methods such as the Deep Q-Network (DQN) have achieved state-of-the-art results in a variety of challenging, high-dimensional domains. This success is mainly attributed to the power of deep neural networks to learn rich domain representations for approximating the value function or policy. Batch reinforcement learning methods with linear representations, on the other hand, are more stable and require less hyper parameter tuning. Yet, substantial feature engineering is necessary to achieve good results. In this work we propose a hybrid approach -the Least Squares Deep Q-Network (LS-DQN), which combines rich feature representations learned by a DRL algorithm with the stability of a linear least squares method. We do this by periodically re-training the last hidden layer of a DRL network with a batch least squares update. Key to our approach is a Bayesian regularization term for the least squares update, which prevents over-fitting to the more recent data. We tested LS-DQN on five Atari games and demonstrate significant improvement over vanilla DQN and Double-DQN. We also investigated the reasons for the superior performance of our method. Interestingly, we found that the performance improvement can be attributed to the large batch size used by the LS method when optimizing the last layer.

The authors propose to augment value-based methods for deep reinforcement learning (DRL) with batch methods for linear approximation function (SRL). The idea is motivated by interpreting the output of the second-to-last layer of a neural network as linear features. In order to make this combination work, the authors argue that regularization is needed. Experimental results are provided for 5 Atari games on combinations of DQN/Double DQN and LSTD-Q/FQI.
Strengths:
I find the proposition of combining DRL and SRL with Bayesian regularization original and promising.
The explanation provided for the improved performance seems reasonable, but it could have been better validated (see below).
Weaknesses:
The presentation of Algorithm 1 and in particular line 7, is a bit strange to me, given that in Section 4, the authors mention that generating D with the current weights results in poor performance. Why not present line 7 as using ER only? Besides, would it be interesting to use p% of trajectories from ER and (1-p)% of trajectories generated from the current weights?
The experiments could be more complete. For instance, in Table 1, the results for LS-DDQN_{LSTD-Q} are missing and in Fig.2, the curves for Qbert and Bowling are not provided.
For a fairer comparison between DQN and LS-DQN, I think the authors should compare Algorithm 1 against DQN given an equal time-budget to check that their algorithm indeed provides at the end a better performance.
Besides, the ablative analysis was just performed on Breakout. Would the same conclusions hold on different games?
Detailed comments:
l.41: the comment on data-efficiency of SRL is only valid in the batch setting, isn’t it?
l.114, l.235: it’s
l.148: in the equation above, \mu should be w_k. I find the notation w_k for the last hidden layer to be unsatisfying, as w_k also denotes the weights of the whole network at iteration k
l.205, l.245: a LS -> an LS
l.222: algortihm


Collapsed variational Bayes for Markov jump processes
Markov jump processes are continuous-time stochastic processes widely used in statistical applications in the natural sciences, and more recently in machine learning. Inference for these models typically proceeds via Markov chain Monte Carlo, and can suffer from various computational challenges. In this work, we propose a novel collapsed variational inference algorithm to address this issue. Our work leverages ideas from discrete-time Markov chains, and exploits a connection between these two through an idea called uniformization. Our algorithm proceeds by marginalizing out the parameters of the Markov jump process, and then approximating the distribution over the trajectory with a factored distribution over segments of a piecewise-constant function. Unlike MCMC schemes that marginalize out transition times of a piecewise-constant process, our scheme optimizes the discretization of time, resulting in significant computational savings. We apply our ideas to synthetic data as well as a dataset of check-in recordings, where we demonstrate superior performance over state-of-the-art MCMC methods.

The authors present a variational inference algorithm for continuous time Markov jump processes. Following previous work, they use "uniformization" to produce a discrete time skeleton at which they infer the latent states.  Unlike previous work, however, the authors propose to learn this skeleton (a point estimate, via random search) and to integrate out, or collapse, the transition matrix during latent state inference.  They compare their algorithm to existing MCMC schemes, which also use uniformization, but which do not collapse out the transition matrix.
While this work is well motivated, I found it difficult to tease out which elements of the inference algorithm led to the observed improvement. Specifically, there are at least four dimensions along which the proposed method differs from previous work (e.g. Rao and Teh, 2014):  (i) per-state \Omega_i vs shared \Omega for all states; (ii) learned point estimate of discretization vs sampling of discretization;  (iii) variational approximation to posterior over latent states vs sampling of latent state sequence; and (iv) collapsing out transition matrix vs maintaining sample / variational factor for it.  That the "Improved MCMC" method does not perform better than "MCMC" suggests that (i) does not explain the performance gap.  It seems difficult to test (ii) explicitly, but one could imagine a similar approach of optimizing the discretization using sample based estimates of the marginal likelihood, though this would clearly be expensive.
Dimensions (iii) and (iv) suggest two natural alternatives worth exploring. First, an "uncollapsed" variational inference algorithm with a factor for the transition matrix and a structured variational factor on the complete set of latent states U. Indeed, given the discretization, the problem essentially reduces to inference in an HMM, and uncollapsed, structured variational approximation have fared well here (e.g. Paisley and Carin, 2009; Johnson and Willsky, 2014).  Second, it seems you could also collapse out the transition matrix in the MCMC scheme of Rao and Teh, 2014, though it would require coordinate-wise Gibbs updates of the latent states u_t | u_{\neg t}, just as your proposed scheme requires q(U) to factorize over time.  These two alternatives would fill in the gaps in the space of inference algorithms and shed some light on what is leading to the observed performance improvements. In general, collapsing introduces additional dependencies in the model and precludes block updates of the latent states, and I am curious as to whether the gains of collapsing truly outweigh these costs. Without the comparisons suggested above, the paper cannot clearly answer this question.
Minor comments:
- I found the presentation of MCMC a bit unclear.  In some places (e.g. line 252) you say MCMC "requires a homogeneously dense Poisson distributed trajectory discretization at every iteration."  but then you introduce the "Improved MCMC" algorithm which has an \Omega_i for each state, and presumably has a coarser discretization.
References:
Paisley, John, and Lawrence Carin. "Hidden Markov models with stick-breaking priors." IEEE Transactions on Signal Processing 57.10 (2009): 3905-3917.
Johnson, Matthew, and Alan Willsky. "Stochastic variational inference for Bayesian time series models." International Conference on Machine Learning. 2014.


Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space
This paper explores image caption generation using conditional variational autoencoders (CVAEs). Standard CVAEs with a fixed Gaussian prior yield descriptions with too little variability. Instead, we propose two models that explicitly structure the latent space around K components corresponding to different types of image content, and combine components to create priors for images that contain multiple types of content simultaneously (e.g., several kinds of objects). Our first model uses a Gaussian Mixture model (GMM) prior, while the second one defines a novel Additive Gaussian (AG) prior that linearly combines component means. We show that both models produce captions that are more diverse and more accurate than a strong LSTM baseline or a "vanilla" CVAE with a fixed Gaussian prior, with AG-CVAE showing particular promise.

This paper investigated the task of image-conditioned caption generation using deep generative models. Compared to existing methods with pure LSTM pipeline, the proposed approach augments the representation with an additional data dependent latent variable. This paper formulated the problem under variational auto-encoder (VAE) framework by maximizing the variational lowerbound as objective during training. A data-dependent additive Gaussian prior was introduced to address the issue of limited representation power when applying VAEs to caption generation. Empirical results demonstrate the proposed method is able to generate diverse and accurate sentences compared to pure LSTM baseline.
== Qualitative Assessment ==
I like the motivation that adding stochastic latent variable to the caption generation framework. Augmenting the prior of VAE is not a novel contribution but I see the novelty applied to caption generation task. Performance-wise, the proposed AG-CVAE achieved more accurate performance compared to both LSTM baseline and other CVAE baselines (see Table 2). The paper also analyzed the diversity of the generated captions in comparison with the pure LSTM based approach (see Figure 5). Overall, the paper is generally well-written with sufficient experimental details.
Considering the additive Gaussian prior as major contribution, the current version does not seem to be very convincing to me. I am happy to raise score if my concerns can be addressed in the rebuttal.
* Any strong evidence showing the advantages of AG-CVAE over CVAE/GMM-CVAE?
The improvements in Table 2 are not very significant. For qualitative results (Figure 5 and other figures in supplementary materials), side-by-side comparisons between AG-CVAE and CVAE/GMM-CVAE are missing. It is not crystal clear to me whether AG-CVAE actually adds more representation power compared to CVAE/GMM-CVAE. Please comment on this in the rebuttal and include such comparisons in the final version of the paper (or supplementary materials).
* Diversity evaluation: it is not clear why does AG-CVAE perform worse than CVAE. Also, there is no explanation about performance gap from different variations of AG-CVAE. Since CVAE is a stochastic generative model, I wonder whether top 10 sentences are sufficient for diversity evaluations? The results will be much more convincing if the authors provide a curve (y-axis is the diversity measure and x-axis is the number of sentences).
Additional comments:
* Pre-defined means of clusters for GMM-CVAE and AG-CVAE (Line 183)
It is not surprising that the authors failed to obtain better results when means of clusters u_k are free variables. In principle, it is possible to learn duplicate representations without any constraints (e.g., sparsity or orthogonality) on u_k or c_k. I would encourage the authors to explore this direction a bit in the future. Hopefully, learnable data-dependent prior can boost the performance to some extent.