We propose DeepV2D, an end-to-end deep learning architecture for predicting depth from video. DeepV2D combines the representation ability of neural networks with the geometric principles governing image formation. We compose a collection of classical geometric algorithms, which are converted into trainable modules and combined into an end-to-end differentiable architecture. DeepV2D interleaves two stages: motion estimation and depth estimation. During inference, motion and depth estimation are alternated and converge to accurate depth.

This paper pushes forward the research of deep learning based video 3D reconstruction by decomposing the problem in two-stages:
1. Depth estimation from multi-view stereo
2. Camera pose estimation from optical flow estimation and PnP SE3 pose,
which turns out to achieve state-of-the-art performance on public datasets.
In general, the iterative procedure of this paper is similar to DeepTAM (Zhou et al., 2018), the major difference is that the camera pose is estimated by PnP with estimated Flow but not directly predicted from sub-network structures, which contributes to the higher tracking accuracy and further improves the accuracy of multi-view stereo depth estimation.
A multi-view camera pose estimation (the global pose optimization in the paper) is also proposed to utilize the relative relation between all video frame pairs, which is difficult for fully CNN based pipeline and maximizes the use of the explicit PnP optimization.
After that, a residual flow field is predicted using CNNs and camera poses is further refined by minimizing the geometric re-projection error.
So this paper focuses more on the camera pose estimation than the depth, which is a good start point to achieve better multi-view capabilities in a CNN framework.
However, I still have several concerns for this paper:
1. The full system is highly engineered and complicated. For example, the feature map is extracted from two hour-glass networks, which seems over-complicated for feature extraction, and the multi-view stereo network using four 3D hour-glass networks, which consumes a large amount of memories. So I would like to see the authors demonstrate the performance from in simpler settings, e.g. the feature map can be a single encoder-decoder and the multi-view stereo is done by correlation and 2D convolution. So I would like to see the inference time, peak memory consumption and the model size. Ablation studies will also help the readers to understand whether performance gain is from the pose estimation or the network capacities, which is unclear in the current paper (Appendix.D).
2. The comparison with state-of-the-art conventional system is missing. For example, is the camera pose estimation better than initializing the DSO (Engel et al., 2018) with a monocular depth estimation? In real applications, if the performance gain is insignificant, the conventional method will still be a better choice because the CNN based methods are computationally expensive on platforms without powerful GPUs. I will not downgrade the rating if the performance gain is insignificant, but it is necessary to see the comparison.
3. Even the difference is ignorable for performance, I hope the author could use adjoint when deriving the derivatives in Eq.(11) of Appendix A.1. The author can refer to this tutorial http://ethaneade.com/lie.pdf or the text book https://www.eecis.udel.edu/~cer/arv/readings/old_mkss.pdf.
PS: For the current version with 10 pages, I lean to a borderline score, but I select 6 because 5 is not an option. I will keep or raise the current score if my concerns are addressed during rebuttal.


Even though recent works have brought some insight into the performance improvement of techniques used in state-of-the-art deep-learning models, more work is needed to understand the generalization properties of over-parameterized deep neural networks. We shed light on this matter by linking the loss function to the output's sensitivity to its input. We find a rather strong empirical relation between the output sensitivity and the variance in the bias-variance decomposition of the loss function, which hints on using sensitivity as a metric for comparing generalization performance of networks, without requiring labeled data. We find that sensitivity is decreased by applying popular methods which improve the generalization performance of the model, such as (1) using a deep network rather than a wide one, (2) adding convolutional layers to baseline classifiers instead of adding fully connected layers, (3) using batch normalization, dropout and max-pooling, and (4) applying parameter initialization techniques.

This paper examines generalization performance of various neural network architectures in terms of a sensitivity metric that approximates how the error responds to perturbations of the input. A crude argument is presented for how the proposed sensitivity metric captures the variance term in the standard bias-variance decomposition of the loss. A number of experimental results are presented that show strong correlation between the sensitivity metric and the empirical test loss.
Understanding the distinguishing characteristics of networks that generalize well versus networks that generalize poorly is a central challenge in modern deep learning research, so the topic and analyses presented in this paper are salient and will be of interest to most of the community. The experimental results are intriguing and the presentation is clear and easy to read. While some may object to the egregious simplifications utilized in "deriving" the sensitivity metric, I believe this kind of analysis should be welcomed if it produces new insights and helps explain otherwise opaque empirical phenomena. All told, if taken in isolation from prior work, I think the insights and empirical results presented in this paper are quite interesting and certainly sufficient for acceptance to ICLR.
However, there is significant overlap with prior work that severely detracts from the novelty of the results presented here, and I think the community is already familiar with the paper's main conclusions. From the empirical viewpoint, [1] performs a very similar (and actually quite a bit more thorough) analysis, and reaches very similar conclusions. The authors do cite [1], but unless I missed something, their main argument for uniqueness is basically "in experiments, we prefer S to the Jacobian, because in order to compute S it is enough to look at the network as a black box that given an input, generates an output, without requiring further knowledge of the model." While this may be useful from the practical standpoint for some non-differentiable models, I'm not convinced that this distinction is really significant in terms of building insights or new understanding.
One additional way this paper is distinct from [1] is that it includes a theoretical "derivation" for the sensitivity metric. While I found the argument interesting, from the theoretical perspective, [2] gives much more rigorous and insightful arguments that help explain the observed phenomena.
Overall, I'm just not convinced this paper is novel enough to merit publication. But perhaps I've overlooked something, in which case I hope the author's response can highlight their unique contributions relative to prior work.
[1] Novak, Roman, et al. "Sensitivity and generalization in neural networks: an empirical study." arXiv preprint arXiv:1802.08760 (2018).
[2] Arora, Sanjeev, et al. "Stronger Generalization Bounds for Deep Nets via a Compression Approach." International Conference on Machine Learning. 2018.


We study the convergence of gradient descent (GD) and stochastic gradient descent (SGD) for training L-hidden-layer linear residual networks (ResNets). We prove that for training deep residual networks with certain linear transformations at input and output layers, which are fixed throughout training, both GD and SGD with zero initialization on all hidden weights can converge to the global minimum of the training loss. Moreover, when specializing to appropriate Gaussian random linear transformations, GD and SGD provably optimize wide enough deep linear ResNets. Compared with the global convergence result of GD for training standard deep linear networks (Du & Hu, 2019), our condition on the neural network width is sharper by a factor of OpκLq, where κ denotes the condition number of the covariance matrix of the training data. We further propose a modified identity input and output transformations, and show that a pd`kq-wide neural network is sufficient to guarantee the global convergence of GD/SGD, where d, k are the input and output dimensions respectively.

This paper studies the convergence properties of GD and SGD on deep linear resnets. The authors prove that, under certain conditions on the input and output transformations and with zero initialization, GD and SGD converges to global minima. The results derived in this paper show that the condition on the width of a deep linear resnet is less strict (by a factor O(L kappa) where L is the depth and kappa the condition number of the training data) that a network without residual connection. Overall, the paper is well-written and the analysis is fairly standard and easy to follow (I checked most of the theorems except for Lemma A.2 and the results for stochastic gradients). The authors do not clearly contrast their results to prior work (especially Bartlett et al. (2019) and Arora et al. (2019a)) and I’m therefore not convinced the final result brings any new insight. Please address this issue in your rebuttal. I will reconsider my score if the authors can provide a satisfactory answer.
Comparison to Du & Hu
The authors claim that their bound shows an improvement by a factor of O(kappa L) over the deep linear network (without residual connections) analyzed in Du & Hu.
1) I don’t think this is explicitly stated in the paper but are the initialization conditions and the assumed distance to the optimum the same in both papers?
2) These results are obviously worst-case bounds and the analysis used in both papers is different to some extent. Couldn’t you re-derived this result by adapting your own analysis?
3) Is there any theoretical proof or empirical evidence showing that resnets do indeed scale better w.r.t. to the condition number of the data?
Comparison to prior work
The authors mention the work of Bartlett et al. (2019) and Arora et al. (2019a) but it is never very clear what the real differences are.
1) Regarding Bartlett et al. (2019), you say that “Theorem 3.1 can imply the convergence result in Bartlett et al. (2019).”. Does the result of Theorem 3.1 provides a tighter bound in terms of L or kappa (when using the same initialization conditions)?
2) Arora et al. (2019a): You explain that they showed that “GD converges under substantially weaker conditions”. How much weaker are these conditions? How do they compare to the “modified identity transformation” that allows you to obtain a global convergence rate. How does your final result compare to Arora et al. (2019a)? The analysis of Arora et al. (2019a) requires a balanced-ness condition, is this substituted by a different condition in your analysis?
3) Finally, Allen-Zhu et al. (2019) already has some results on deep resnets although their analysis is for a heavily over-parametrized regime. Can you still comment on how their results differ from yours?
A & B are fixed matrices initialized from a Gaussian distribution. How essential is this condition? Assuming for simplicity that A and B are square matrices. Can I not set A=B=identity and still satisfy the condition in Theorem 1? The term on the RHS would be 1 so then the initialization would need to scale as a function of the spectrum of the data matrix X.
ResNet vs LinearNet
Where does your proof break down for a linear net without residual connections, i.e. where does the proof absolutely need the identify matrices. Looking at the proof, it seems to me, one could change the following in order to still obtain a similar result:
1) If we consider a network B \prod_l W_l AX, the proof of proposition 3.3 could be unchanged if the W matrices are initialized to identify instead of zero.
2) The lower bound on \sigma^2_min(I + \tau W_l) would instead be replaced by a lower on \sigma^2_min(W_l)=\lambda_min(W_l). Is this where  one can see the benefit of having residual connections?
Proposition 3.3
The bound on the singular values of the matrices A and B is vacuous for square matrices, in which case I believe the theorem does not hold. Can you comment on this?
Extension stochastic setting
I only skimmed at the proof but the extension looks fairly straightforward. Can you comment on how difficult this derivation is compared to the deterministic setting?
Extensions
You claim "can potentially provide meaningful insights to the convergence analysis of deep non-linear ResNets." although this is not obvious as the landscape of such networks can be very different. Can you elaborate?


Learning disentangled representations is considered a cornerstone problem in representation learning. Recently, Locatello et al. (2019)  demonstrated that unsupervised disentanglement learning without inductive biases is theoretically impossible and that existing inductive biases and unsupervised methods do not allow to consistently learn disentangled representations. However, in many practical settings, one might have access to a limited amount of supervision, for example through manual labeling of (some) factors of variation in a few training examples. In this paper, we investigate the impact of such supervision on state-of-the-art disentanglement methods and perform a large scale study, training over 52 000 models under well-defined and reproducible experimental conditions. We observe that a small number of labeled examples (0.01-0.5% of the data set), with potentially imprecise and incomplete labels, is sufficient to perform model selection on state-of-the-art unsupervised models. Further, we investigate the benefit of incorporating supervision into the training process. Overall, we empirically validate that with little and imprecise supervision it is possible to reliably learn disentangled representations.

This paper considers the challenge of learning disentangled representations---i.e. learning representations of data points x, r(x), that capture the factors of variation in an underlying latent variable z that controls the generating process of x---and studies two approaches for integrated a small number of data points manually labeled with z (or noisier variants thereof): one using these to perform model selection, and another incorporating them into unsupervised representation learning via an additional supervised loss term.  This investigation is motivated by recent results concluding that inductive biases are needed otherwise learning disentangled representations in an unsupervised fashion is impossible.  The paper poses its overall goal as making this injection of inductive biases explicit via a small number (~100 even) of (potentially noisy) labels, and reports on exhaustive experiments on four datasets.
I think that this paper merits acceptance because (a) the motivation of taking a necessity in practice (somehow selecting models / injecting inductive biases) and making it more explicit in the approach is a good one, and because the thorough empirical survey (and simple, but novel, contribution of a new semi-supervised representation learning objective) are likely valuable contributions to this community.
One negative comment overall would be that the results are not that surprising: that is, the fact that using labels either (a) to do model validation or (b) in a semi-supervised fashion would help is not too surprising.  However, I believe in the context of (a) making more explicit a practical (and theoretically) necessary step in the pipeline of learning representations, and (b) contributing a comprehensive empirical study, this is a worthwhile contribution.
Minor notes:
- Fig. 1 isn't the most intuitive- ideally would be better explained for the headlining figure
- 8.57 P100 GPU years is ~= $75k based on a cursory glance at cloud instance pricing at monthly rates... this is a lot to reproduce these experiments...


Decisions made by machine learning systems have increasing influence on the world. Yet it is common for machine learning algorithms to assume that no such influence exists. An example is the use of the i.i.d. assumption in online learning for applications such as content recommendation, where the (choice of) content displayed can change users' perceptions and preferences, or even drive them away, causing a shift in the distribution of users. Generally speaking, it is possible for an algorithm to change the distribution of its own inputs. We introduce the term self-induced distributional shift (SIDS) to describe this phenomenon. A large body of work in reinforcement learning and causal machine learning aims to deal with distributional shift caused by deploying learning systems previously trained offline. Our goal is similar, but distinct: we point out that changes to the learning algorithm, such as the introduction of meta-learning, can reveal hidden incentives for distributional shift (HIDS), and aim to diagnose and prevent problems associated with hidden incentives. We design a simple environment as a 'unit test' for HIDS, as well as a content recommendation environment which allows us to disentangle different types of SIDS. We demonstrate the potential for HIDS to cause unexpected or undesirable behavior in these environments, and propose and test a mitigation strategy.

The authors study the phenomena of self-introduced distributional shift. They define the term along with the term hidden incentives for distributional shift. The latter describes factors that motivate the learner to change the distribution in order to achieve a higher performance. The authors study both phenomena in two domains (one being a prisoner dilemma and the other a recommender system) and show how meta-learning reveals the hidden incentives for distributional shift. They then propose an approach based on swapping learners between environments to reduce self introduced distributional shift.
In my opinion this paper should be (weak) rejected for several reasons.
-	The paper is poorly written. Some sentences are hard to comprehend, even after repeated reading. It feels hastily written and should be carefully proofread with the help of a proficient English speaker.
-	The very first paragraph is not a helpful example. While the authors provide an example that that illustrates a distributional shift, they don’t describe what the shift is and why that shift in a negative consequence for learning. Therefore, the example is not helpful, but rather confusing.
-	In point 3 of the contributions, the authors state ‘that meta-learning reveals HIDS in these environments’. Is this true for all meta-learning approaches? The current statement overclaims their findings. If the authors decide to keep it, they should provide a description of the types of meta-learning approaches that reveal HIDS and which don’t or evaluate all different meta-learning approaches.
-	Section 4.1 is hard to follow. The authors state that the meta-learner is used to tune the learning rate – they fail to clearly explain how exactly the learning rate tuning results in a non-myopic behavior for a myopic learner.
-	In the Q-learning example in Section 4.1, is the effect of \epsilon considered? The discovery of non-myopic strategies might simply be based on chance. I would like the authors to include an investigation into the effect of this parameter.
Generally speaking, I appreciate the author’s thoughts on SIDS and how they approach revealing hidden incentives. Although I vote to reject this paper, I strongly encourage the authors to rewrite the paper, address all other issues that are noted during this peer review and resubmit.


In this work we introduce the DP-auto-GAN framework for synthetic data generation, which combines the low dimensional representation of autoencoders with the flexibility of Generative Adversarial Networks (GANs). This framework can be used to take in raw sensitive data, and privately train a model for generating synthetic data that will satisfy the same statistical properties as the original data. This learned model can be used to generate arbitrary amounts of publicly available synthetic data, which can then be freely shared due to the post-processing guarantees of differential privacy. Our framework is applicable to unlabeled mixed-type data, that may include binary, categorical, and real-valued data. We implement this framework on both unlabeled binary data (MIMIC-III) and unlabeled mixedtype data (ADULT). We also introduce new metrics for evaluating the quality of synthetic mixed-type data, particularly in unsupervised settings.

The paper considers synthetic data generation using deep GANs and autoencoders that can be shared for model training.
The authors can generate mixed-type data and consider some additional metrics that allow the evaluation of the quality of synthetic data.
While the problem raised in the paper is interesting, and there some insights on what kind of metrics one should use, the article now lacks conclusions and discussion of obtained results.
In particular, there is a significant number of misprints and inconsistencies here and there (see more on this below).
Moreover, the experiments are irreproducible e.g. I could not found information about the value of reduced dimension q in the description of the experiments.
Also, there is no comparison with previous approaches (e.g. [1, 2]), only results about the proposed one are presented.
The paper will also benefit from additional rounds of proofreading:
1. The algorithm $\mathcal{M}$ is not defined. The range $Range(\mathcal{M})$ is not defined.
1. a mixture of Gaussian distribution -> a
mixture of Gaussian distributions
2. comepare -> compare, matrics -> metrics, deceases -> decreases
4. should to minimize -> should minimize
5. In the formula "(true) loss function" subscript "i" should be dropped, as we talk about $x \sim Z$, not $x_i$ here
6. Articles in many places can be improved (finding good autoencoder -> finding a good autoencoder)
7. It is possible, that in the paragraph after the formula (2) "encoder" should be replaced by "decoder".
8.  the total number of samples the real
data - > the total number of available real data samples
9. The axis labels are too small for Figure 2
10. No reference to Figure 3 in the text of the paper. For Figure 3 the most left plot has for some reason a smaller number of points. Why?
11. The selection of classifiers is not discussed. I.e. why in some cases authors use random forests (5.2), but in other logistic regression (5.1)? Also in my opinion mixing of R2 and F1 scores in one plot can be confusing.
12. No conclusion in the end
[1.] Xu et al. Modeling tabular data using conditional GAN. NeurIPS 2019
[2.] S.K.Lim et al. DOPING: Generative Data Augmentation for Unsupervised Anomaly Detection with GAN. IEEE ICDM 2019.


Improving the accuracy of numerical methods remains a central challenge in many disciplines and is especially important for nonlinear simulation problems. A representative example of such problems is fluid flow, which has been thoroughly studied to arrive at efficient simulations of complex flow phenomena. This paper presents a data-driven approach that learns to improve the accuracy of numerical solvers. The proposed method utilizes an advanced numerical scheme with a fine simulation resolution to acquire reference data. We, then, employ a neural network that infers a correction to move a coarse thus quickly obtainable result closer to the reference data. We provide insights into the targeted learning problem with different learning approaches: fully supervised learning methods with a naive and an optimized data acquisition as well as an unsupervised learning method with a differentiable Navier-Stokes solver. While our approach is very general and applicable to arbitrary partial differential equation models, we specifically highlight gains in accuracy for fluid flow simulations.

Numerical solvers for partial differential equations take a lot of time to get high resolution results since they have to explore high dimensional grid in function domain. Thus, it is important to interpolate between grids to get high resolution results. In this paper, the authors propose the model that assists PDE solver by correcting residuals in a data-driven way. Specifically, they try to approximate NN to correction function in supervised and unsupervised manners. They also propose a temporal regularization method that smooths behavior of fluid between times. As a result, proposed method can generate high resolution results in efficient way with smoke rising simulation dataset.
Significance
- The field that the paper target is too narrow so that it may not be significant to general machine learning community.
Novelty
- I think this approach is somewhat novel since it introduces new direction in the field where the model assists PDE solver to improve performance with learned correction function.
Clarity
- I think this paper is well written in most places, but the paper is not intended to make it easy for general machine learning researchers to understand and follow the problem thoroughly.
Pros
- The proposed method is orthogonal to other methods such as super-resolution and may be complemented to other existing approaches.
Cons
My main concern is that the evaluation of proposed method is quite limited in the sense that- The method is evaluated with only single dataset.- Only basic model is used to compare to proposed model. What about other baseline models use similar deep learning approaches?
- There should be various PDE problems other than Navier-Stokes to evaluate the general effectiveness of the proposed model.
- I think time comparison/analysis should be provided since the paper targets to get efficient approximation (using NN) in numerical analysis field.
Questions and comments
- I wonder the affect of hyper-parameter in temporal regularization.
- I wonder performance comparison to existing NN based approaches.


Space partitions of R d underlie a vast and important class of fast nearest neighbor search (NNS) algorithms. Inspired by recent theoretical work on NNS for general metric spaces (Andoni et al., 2018b;c), we develop a new framework for building space partitions reducing the problem to balanced graph partitioning followed by supervised classification. We instantiate this general approach with the KaHIP graph partitioner (Sanders & Schulz, 2013)  and neural networks, respectively, to obtain a new partitioning procedure called Neural Locality-Sensitive Hashing (Neural LSH). On several standard benchmarks for NNS (Aumüller et al., 2017), our experiments show that the partitions obtained by Neural LSH consistently outperform partitions found by quantization-based and tree-based methods as well as classic, data-oblivious LSH.
Recently, there has been a large body of work that studies how modern machine learning techniques (such as neural networks) can help tackle various classic algorithmic problems (a partial list includes Mousavi et al. (2015); Baldassarre et al. (2016); Bora et al. (2017); Dai et al. (2017); Metzler et al. (2017); Kraska et al. (2018); Balcan et al. (2018); Lykouris & Vassilvitskii (2018); Mitzenmacher (2018); Purohit et al. (2018)). Similar methods-under the name "learn to hash"-have been used to improve the sketching approach to NNS (Wang et al., 2016). However, when it comes to indexing, while some unsupervised techniques such as PCA or k-means have been successfully applied, the full power of modern tools like neural networks has not yet been harnessed. This state of affairs naturally leads to the following general question: Can we employ modern (supervised) machine learning techniques to find good space partitions for nearest neighbor search?

The paper proposes a scheme to learn space partitions for improved
nearest neighbor search by first converting the search problem into a
supervised classification problem by graph-partitioning the
k-nearest-neighbor graph, and then using some machine learning model
to learn space partitions corresponding to the supervised problem. The
scheme is theoretically motivated and the empirical results
demonstrates the utility of the proposed scheme against various
baselines.
While the paper presents a promising new direction for
nearest-neighbor search with space partitioning schemes, I am somewhat
on the border, leaning towards reject. This is mostly due to my lack
of understanding why Cayton & Dasgupta (2007) and Li et al. (2011) are
not valid baselines and are dismissed as "hyperplane
partitions". It would be very helpful to me as a reader to understand
why this literature does not warrant to be a valid
baseline. Especially given the following connection points:
- Firstly, the theoretical results presented by the authors also
  limit to hyperplane splits.
- Secondly, the authors present empirical evaluation of recursive
  hyperplane partitions where the proposed scheme is the only learning
  based scheme while the techniques presented in Cayton & Dasgupta
  (2007) and Li et al. (2011) would have been the learning based
  partitioning baseline to beat.
- Thirdly, LSH space partitioning is considered as a baseline in
  Section 4.2 with the Neural LSH and Neural Catalyzer + k-means being
  the only learning based baseline; Cayton & Dasgupta (2007)'s learned
  LSH scheme would have been a valid baseline to the best of my
  understanding.
- Finally, the authors mention the need for coupling the graph
  partitioning and the supervised learning phase. Li et al., 2011
  utilizes a loss function that essentially promotes a sparse graph
  cut while preserving near-neighborhoods, providing the closely
  coupled scheme. In the original paper, the authors utilize a
  recursive hyperplane partitioning scheme. However, a neural network
  with a softmax layer can be trained to minimize this closely coupled
  loss directly, obviating the need for the separate graph
  partitioning and supervised learning phases proposed in this
  paper. But it is possible that I have misinterpreted the connection
  and any clarification here would be very helpful.
Minor:
- It would be very helpful as a reader to understand how sensitive the
  neural network based partioning scheme is to the network
  architecture and hyperparameters. Some intuition behind the choices
  would be very useful.
- It is a little unclear how the plots in Figure 2 are generated --
  for each query, we would have access to the accuracy vs. number of
  candidates curve. In that case, it is unclear how the curves are
  aggregated across queries to generate average number of candidates
  (and 0.95 quantile) vs. accuracy curves. Given that not all queries
  in all the methods (including the proposed ones) will have the same
  sequences of accuracies, the quantile curve is especially unclear.
- The authors mention that neural LSH is only useful when the bin
  sizes are large, and otherwise k-means is more useful. Is there a
  straightforward way to decide when to switch between the two? Is it
  another hyperparameter we need to tune over while generating these
  space partitions?
- Theorem 3.1 considers only hyperplane partitions while neural
  networks usually generate nonlinear partitions. What are the
  conditions on the nonlinear splits that allows the guarantees to
  improve (or at least not degrade)? Is it obviously always improved?


There are time series that are amenable to recurrent neural network (RNN) solutions when treated as sequences, but some series, e.g. asynchronous time series, provide a richer variation of feature types than current RNN cells take into account. In order to address such situations, we introduce a unified RNN that handles five different feature types, each in a different manner. Our RNN framework separates sequential features into two groups dependent on their frequency, which we call sparse and dense features, and which affect cell updates differently. Further, we also incorporate time features at the sequential level that relate to the time between specified events in the sequence and are used to modify the cell's memory state. We also include two types of static (whole sequence level) features, one related to time and one not, which are combined with the encoder output. The experiments show that the proposed modeling framework does increase performance compared to standard cells.

This paper proposes Sparse Time LSTM, an extension of TLSTM to handle (the most importantly) the sparse features which is updated infrequently, and also the static features which does not change across the time. For sparse features, authors proposed an lazy update mechanism where the memory state is updated only at the time of a sparse feature update (i.e. when $m_{tk} = 1$), and then the hidden state / output gate are updated by aggregating over the hidden state proposals for all sparse features (using either average-/max-pooling or a MLP). For static features, the authors proposed inputing the feature embeddings only at the final time steps. The proposed method can also handle decay features by using the decay mechanism that already exists in TLSTM (although the authors proposed an extension to generalize the mechanism to vector input). The authors tested the proposed method on a synthetic dataset based on UCI electronic power consumption data, and a proprietary dataset for churn prediction.
I vote to reject the paper in its current form. I believe the paper is tackling an important task (extending neural network methods for sparse and missing time-series data), and I find the proposed methods intuitive and the experiment evaluations interesting. However, I find the technical description of the proposed method insufficient, in the sense that (1) there does not seem to be explanation about how the proposed mechanism is better than the previous method (e.g. intuition or theoretical justification of why handling sparse update using the TLSTM approach is inferior), or (2) no justification about the particular choice of the model (e.g. in Section 3.2, why choose this particular form of g)? Also, the experiment section does not seem to contain sufficient detail to be reproducible (e.g., how many simulation runs were conducted?), and the reported results do not contain standard error information which makes it difficult to decide the significance of the improvement. The organization of the paper can also be improved (see Major comments). In summary, the paper in its current form does not yet reach the ICLR standard due to the lack of clarity in its technical description, and lack of sufficient detail in experiment results to be convincing. This paper can benefit greatly with sufficient justification of its choice of model mechanisms, a more careful explanation of the advantage of its approach to sparse feature relative to previous approaches, and also a self-contained description of the experiment procedures and standard error information for the experiments.
Although the proposed method is of limited novelty (a straightforward extension to TLSTM) and the proposed methods for static/decay features feels somehow trivial, I feel this work is still interesting since it discusses an interesting topic. For future improvements, in addition to incorporating reviewers' comments from past submissions,  authors can consider shaping it into an in-depth study of how LSTM approaches should best handle sparse features.
Major Comments:
The organization of the method section can be improved. Earlier descriptions of the TLSTM should become a separate section called Related Work or Background, or at least being clearly marked as previous work. The proposed STLSTM method, which is the major innovation of this paper, should be introduced in the beginning of the Method section (rather than being delayed to Section 3.3).
A more detailed explanation of experiment protocol is needed. What is group sampling, and how many times were the experiments repeated? On a related note, in Table 1a-1b, please also report the standard error of F1 scores across multiple runs. This is needed to access whether the improvements brought by dense layer / additional features are significant.
Minor Comments:
Please add equation numbers to all equations.
Equation on page 5, please explain explicitly what is being aggregated over by the aggregation function L. Is the hidden states for all sparse features?
The titles and labels for the experiment result figure (Figure 5) are too small to be readable. Also there's no legend for the first two figures. What does each colored line mean? My guess is they are corresponding to the seven features, however author should explain this clearly in the figure caption or in the text description.


The Adam algorithm has become extremely popular for large-scale machine learning. Under convexity condition, it has been proved to enjoy a data-dependent O( √ T ) regret bound where T is the time horizon. However, whether strong convexity can be utilized to further improve the performance remains an open problem. In this paper, we give an affirmative answer by developing a variant of Adam (referred to as SAdam) which achieves a data-dependent O(log T ) regret bound for strongly convex functions. The essential idea is to maintain a faster decaying yet under controlled step size for exploiting strong convexity. In addition, under a special configuration of hyperparameters, our SAdam reduces to SC-RMSprop, a recently proposed variant of RMSprop for strongly convex functions, for which we provide the first data-dependent logarithmic regret bound. Empirical results on optimizing strongly convex functions and training deep networks demonstrate the effectiveness of our method.

This paper studies Adam and proves that under strong convexity assumption, it obtains the improved regret bound $O(log(T))$. The regret bound is data-dependent, thus as a side-effect it also improves previous known result for strongly convex RMSProp (SC-RMSProp).
The paper is clear and well-written and I also think that theoretical results are correct and new. However, I have some concerns on the possible impacts of the results especially in the context of ICLR:
- First of all, the assumption to show improved regret is strong convexity of all functions $f_t$. However, this is very restrictive and much stronger than the assumption that the sum of functions is strongly convex. In addition, from what I see, in the proof of Theorem 1, the authors use strong convexity with the $x^\star$, so they can maybe replace global strong convexity to strong convexity restricted to the path towards the solution. A reference where these restricted strong convexity type assumptions are studied:
Necoara, Nesterov, Glineur, “Linear convergence of first order methods for non-strongly convex optimization”, Math. Prog. 2019.
- To show improved regret, consistent with previous work SC-Adagrad and SC-RMSProp, the authors modify the algorithm to use $V_t^{-1}$ in page 3 in the step size, instead of $V_t^{-1/2}$ of regular Adam. It is easy to see that this is to make sure step size has a faster decrease, which is needed also to show standard SGD gets $1/k$ rate for strongly convex problems. However, given that one might not now if the problem has strong convexity (there might exist cases where this property only exists locally), it is not clear if one should apply Adam or SAdam.
- Another remark related to the previous one is the following. Standard SGD uses step size $\alpha_0/\sqrt{k}$ for convex optimization without strong convexity and $\alpha_0/k$ for strongly convex optimization. If one uses $alpha_0/k $for convex optimization without strong convexity, one gets a very bad rate $1/log(k)$ and very bad practical performance. So, given SAdam gives step sizes suited for strongly convex optimization (similar to SGD for strongly convex optimization), I would expect SAdam's step sizes to be not very suitable when there is no strong convexity.
- An additional point is that the step size of SAdam depends on the global strong convexity parameter $\lambda$ which further restricts the applicability of the method. For the theoretical results to hold, the step size should be set according to $\lambda$, and when the step size is not selected that way, one loses the fast convergence rate.
- In the experiments, the authors show the performance of SAdam for neural network training and related to my previous remarks, I have the following concerns. First of all, how do the authors pick step sizes now since it depends on strong convexity constant as in eq. (7). In addition, given that neural networks are certainly non-strongly convex, I would expect that the fast decreasing step size caused by using $V_t^{-1}$ might also hurt the performance considerably, which happens as I discussed above even for convex but non-strongly convex losses. I would suspect that much worse effects can be seen for non-convex optimization. Of course, the authors can argue that if the loss landscape of neural network has some local strong convexity parameters, SAdam would adapt and get faster convergence. But unfortunately, I would not agree with such a statement, because the analysis is not made to adapt to local strong convexity and a dependence to strong convexity constant is present due to eq. (7), so if one does not know the constant, the theoretical guarantees would not apply. In addition, the provided experiments for neural network training is not extensive enough to convince practitioners to use SAdam instead of Adam which has been used for years.
Overall, I think that it is interesting to see that a variant of Adam can be shown to obtain improved regret under strong convexity, I find the assumptions strong and the impact for neural network training, therefore for ICLR, quite questionable.


Learning rate decay (lrDecay) is a de facto technique for training modern neural networks. It starts with a large learning rate and then decays it multiple times. It is empirically observed to help both optimization and generalization. Common beliefs in how lrDecay works come from the optimization analysis of (Stochastic) Gradient Descent: 1) an initially large learning rate accelerates training or helps the network escape spurious local minima; 2) decaying the learning rate helps the network converge to a local minimum and avoid oscillation. Despite the popularity of these common beliefs, experiments suggest that they are insufficient in explaining the general effectiveness of lrDecay in training modern neural networks that are deep, wide, and nonconvex. We provide another novel explanation: an initially large learning rate suppresses the network from memorizing noisy data while decaying the learning rate improves the learning of complex patterns. The proposed explanation is validated on a carefully-constructed dataset with tractable pattern complexity. And its implication, that additional patterns learned in later stages of lrDecay are more complex and thus less transferable, is justified in realworld datasets. We believe that this alternative explanation will shed light into the design of better training strategies for modern neural networks.

The paper investigates the role of learning rate decay in neural network training. While there are prevalent ideas of how/why learning rate decay help both optimization and generalization of neural networks, this work proposes interpretation based on pattern complexity. The mechanism the paper proposes is that initial learning rate helps ignore noise in the beginning and decayed learning rate help to learn complex patterns.
The question the paper tackles is a very important question in understanding deep learning that requires careful study. As the learning rate schedule benefits neural network models beyond specific domain, this question has high significance and potential impact.
The authors propose a view from a pattern complexity. I think this view point is well motivated in the light of Li et al. (2019). The artificial task constructed by authors reveals that with different phases of learning rates, different types of complexity are being learnt. Also transfer learning tasks also reveals that one might want to use models before dropping the learning rate for purpose of transfer learning.
While the question studied is of high importance, I am not confident that the methods presented justifies the claims in the paper. Especially authors analysis on how previous understanding of learning rate decay is flawed doesn’t seem fully supported. With current submission I slightly lean towards rejecting.
One reason is that overall the details of experiments are not specific enough that I personally wouldn’t feel comfortable reproducing the results. For example, in Section 4, was data augmentation or weight decay used? What was the mini-batch size used for SGD experiments? Which learning rate is used for  large learning rate and to which learning rate was it decayed to? I think even one is using full batch GD, the performance shown in Figure 4 which is just above 60% test accuracy is quite low for models like WideResNet.
Also I do not agree with the claim that SGD explanation leads to training curves as in Figure 6. The plot indicates large learning rate / lower learning rate have about the same probability of reaching the 'global minima’. However as described in section 3.2, the role of lowering the learning rate should be increasing the probability of global minima. To me experiments shown in Figure 7, supports both explanations from Lecun et al (1991) Kleinberg et al. (2018) described in Table1.
Few extra comments:
One question I have regarding transferability analysis. Kornblith et al. (2019) showed that better Imagenet models transfers better. However section 6 shows that  transferability is higher for stage 2 which has higher error. How do you reconcile this discrepancy?
I don't quite comprehend the motivation for putting two complexity patterns in disjoint channels. Especially the comment “This mimics the intuition of patterns as the eye pattern and the nose pattern have different locations in an image of human face.” is unclear. Could you elaborate?
Figure 15 seems incomplete. Should there be arrows for decay points? Please also include details of those experiments too.


In this paper we present a method for algorithmic melody generation using a generative adversarial network without recurrent components. Music generation has been successfully done using recurrent neural networks, where the model learns sequence information that can help create authentic sounding melodies. Here, we use DCGAN architecture with dilated convolutions and towers to capture sequential information as spatial image information, and learn long-range dependencies in fixed-length melody forms such as Irish traditional reel.

This paper describes a (DC)GAN architecture for modeling folk song melodies (Irish reels).
The main idea of this work is to exploit the rigid form of this type of music --- bar structure and repetition --- to enable 2-dimensional convolutional modeling rather than the purely sequential modeling that is commonly used in recent (melodic) music generation architectures.
The ideas in this paper seem sound, though they primarily consist of recombining known techniques for a specific application.
The main weaknesses of this paper are in the evaluation (see below), and while I understand that evaluating generate models for creative applications is difficult and fraught territory, I don't think the efforts taken here are sufficiently convincing.
Strengths:
- The paper is clearly written, and the authors have taken great care to describe the unique structures of the data they are modeling.
- The proposed architecture seems well motivated, and matches to the structure of the data.
Weaknesses:
There are three components to the evaluation, and each of them are problematic:
- The first evaluation (Fig 2) compares the average Frechet distance between phrases generated by different models, and within the original dataset.  Some brief argument is given for why Frechet is a good choice here, but it still seems quite tenuous: what does this distance intuitively mean in terms of the data?  How should the scale of these distances be interpreted / what's a meaningfully large difference?  How concentrated are these average distances (ie, please show error bars, variance estimates, or some notion of spread)?
- The second evaluation (Fig 3) uses t-SNE to embed the generated melodies into a 2D space to allow visual inspection of the differences between distributions produced by each model.  While this might be a reasonable qualitative gut-check, t-SNE is by no means an appropriate tool for quantitative evaluation.  The authors at least did multiple runs of t-SNE, but this hardly amounts to compelling evidence.  Moreover, combining all data sources into one sample prior to running t-SNE induces dependencies between the point-wise neighbor selection distributions, which seems undesirable if the eventual goal is to determine how similar each model's distribution is to the source data.  A better approach might be to create independent plots for each model's output (with the original data), but I'd generally advise against using t-SNE for this kind of analysis altogether.
- The third evaluation (Fig 4) measures the amount of divergence from the key (D) in terms of note unigrams.  This evaluation is done qualitatively, and the histogram is difficult to read --- it may be easier to read if the octave content was collapsed out to produce pitch classes rather than pitches.  If, however, the goal is to actually measure distance from the target key, one could do this quantitatively by comparing histograms to a probe tone profile (or otherwise constructed unigram note model) to more clearly characterise the behaviors of the various models in question.
At a higher level, there is no error analysis provided for the model, nor any ablation study to measure the impact of the various design choices taken here (eg dilation patterns in Figure 1).
The authors seem to argue that these choices are the main contribution of this work, so they should be explicitly evaluated in a controlled setting.


In this paper, we present a deep convolutional neural network (CNN) that performs arbitrary resize operation on intermediate feature map resolution at stage-level. Motivated by weight sharing mechanism in neural architecture search, where a super-network is trained, and sub-networks inherit the weights from the supernetwork, we introduce a novel CNN approach. We present Resizable Neural Networks (ResizableNet), which resize feature map with arbitrary scaling factor at stage level. The inference cost of ResizableNet is equivalent to a network with a single scale because the weights of a convolutional layer are shared across diversified scales. Additionally, we provide essential techniques to build and train our propose ResizableNet. We demonstrate the effectiveness of ResizableNet on large-scale ImageNet classification with various modern network architectures such as MobileNet, ShuffleNet, and ResNet. To go even further, we present three applications of resizable networks: 1) Resizable Augmentation (Resizable-Aug). We present a data augmentation technique based on ResizableNet. It obtains superior performance on ImageNet classification, outperform AutoAugment by 1.2% with ResNet-50. 2) Resizable Network for Neural Architecture Search (Resizable-NAS). We introduce ResizableNet as a single network that can deploy on many different hardware to meet various efficiency constraints. 3) Adaptive Resizable Neural Network (Resizable-Adapt). We introduce the adaptive resizable networks as dynamic networks, which further improve the performance with less testing cost via data-dependent inference.

In a standard multi-stage neural network such as a ResNet, the resize operation between stages typically reduces the spatial resolution by 0.5.  (input ---> stage 1---> 0.5 reduction ---> stage 2 --> 0.5 reduction ---> ....). In this paper the authors apply a variety of reduction factors in between these stages for different training examples (input ---> stage 1---> variable reduction ---> stage 2 --> variable reduction ---> ....). They demonstrate that simply training in this way is a powerful form of augmentation. It also produces a network which may be scaled depending on a FLOP budget.
I think this is a really neat idea, and as far as I'm aware it is novel.  It is similar in spirit to EfficientNet, although more flexible. The experimental results are good. However, the paper is let down by poor writing and a lack of detail.
The paper needs a rewrite as there are many grammatical errors, which cause a bad impression:
- "performs arbitrary resize operation" ->  "performs arbitrary resize operations"
- "Scale is the fundamental component of the physical world" ---> *a* fundamental component!
- "i.e the acuracy of NISP" --> e.g.
- "compare with baseline" -> "compared to the baseline"
- There are several instances of a space missed out between a letter and an open bracket
- "The Figure 2(a)" --> "Figure 2(a)"
The comparison to weight-sharing NAS methods is unnecessary. Those entail searching for architectures and sharing weights through the search process, whereas in this work it is having a network that can take different sized inputs at different stages. On that note, it doesn't feel right to me to refer to there being many "subnetworks". What there really is, is just a single network that is robust to multi-scaled inputs (which is a good thing!)
Figure 1 is nice!
EfficientNet-B0 is a base-network that can be scaled up with a compound scaling approach found through grid search. What happens if you scale up your Resizeable net to the same FLOPs as e.g. EfficientNet-B7?
I like the old-school vision citations, although referring to object detection makes me wonder why there are no experiments on it. For distillation, I recommend you cite https://arxiv.org/abs/1312.6184, as the Hinton paper is really just an extension of this.
The fair sampling seems important to the method. Could a detailed explanation be included in the appendix? Do you given the performance as a result of naive sampling? What do you mean by "Some convolutional layers can go through more training issues than others"?
A big problem in this paper is that (as far as I can tell) the scaling factors considered aren't given (but we are told that they lie between 0 and 1). It isn't possible to sample these arbitrarily as you indicate that a batch-norm layer is needed for each one, so these must be discrete (because of this, it isn't correct to say that you have infinite networks contained within).  It therefore isn't clear e.g. in Table 1 which permutation of scaling factors were used in upsampling the networks. Are the results given representative of the best-case selection of these scaling? I hope this isn't the case. I am assuming it is uniform scaling in the case of the "individually trained counterparts". It would be interesting to know more generally which combinations worked well.
In Section 3.5 I'm not sure what is meant by "It is inefficient to process all images to one-subnetwork, as the algorithm spends equal energy at each sample".  I assume energy use is proportional to the number of FLOPS, which in turn depends on spatial resolution.
The legend in Figure 2 is close-to unreadable and needs changing.
The results are impressive, but error bars would be appreciated if possible. As ImageNet is the only dataset considered, this would give some needed clout.
Pros
-------
- Nice, novel method
- Good experimental results
Cons
--------
- Paper is poorly written
- Very few details of the scaling factor variations
- Only one dataset considered
Although the paper is written badly and the narrative is muddled, the underlying idea is a nice one, which is executed well experimentally. Because of this I would like to recommend a Weak Accept, subject to the authors (i) doing a rewrite and (ii) including more information regarding the scaling permutations.


Variational inference (VI) plays an essential role in approximate Bayesian inference due to its computational efficiency and general applicability. Crucial to the performance of VI is the selection of the divergence measure in the optimization objective, as it affects the properties of the approximate posterior significantly. In this paper, we propose a meta-learning algorithm to learn (i) the divergence measure suited for the task of interest to automate the design of the VI method; and (ii) initialization of the variational parameters, which reduces the number of VI optimization steps drastically. We demonstrate the learned divergence outperforms the hand-designed divergence on Gaussian mixture distribution approximation, Bayesian neural network regression, and partial variational autoencoder based recommender systems.

Summary of the paper:
This paper proposes to meta-learn a parametric divergence measure for variational inference (VI). Specifically, the paper focuses on alpha-divergence and f-divergence, aims at choosing a good alpha or f for a particular task leveraging the experience learned from previous tasks. The meta-learned divergence is applied to approximate a mixture of two Gaussians, regress sinusoid function with Bayesian neural networks, and learn recommender systems with partial VAE.
Summary of my opinion:
I am leaning towards reject this paper because
1) I'm not convinced by their motivation -- "this line of work has also shown that the optimal divergence can vary depending on tasks". There are many key factors to achieve good performance in VI, such as a good likelihood model, a good variational posterior, a good optimizer and so on. The divergence itself, in my opinion, is less significant. If you really care about the flexibility, the Wasserstein distance may be a better choice.
2) Choosing the alpha or f is a hyper-parameter search problem to me. What you need to do is to prepare a validation set and select the best hyper-parameters based on the performance there. It is impractical to collect M tasks and search for hyper-parameters according to some meta-losses.
3) The experimental part is weak -- two toy problems plus an unusual recommendation system problem is not the typical way that people choose to evaluate VI methods. I suggest to evaluate on standard VAE/BNN benchmarks.
Major comments:
1. From eq.(4) and eq.(8), it seems one has to compute the value of the density function p(theta, D) in order to compute the gradients, which is obviously not possible for BNNs and VAEs. More details on how to perform VI with alpha-divergence and f-divergence should be covered.
2. Can you let the meta-loss to be equal to D_eta? Meta-learning algorithms often optimize the same loss function in the inner and outer level. I couldn't see the point why you want to learn a f-divergence while setting the meta-loss to be D_0.5.
3. Do you have a validation set for the Bayesian optimization (BO) baseline or do you use cross-validation? The details of this baseline should be elaborated. In fact, it is possible to choose the best alpha for each task using BO. Should this be compared?
4. What does VB mean in Section 4.2? If VB uses KL rather than alpha-divergence or f-divergence, the outperformance may only suggest that KL is insufficient there.
5. Have you considered input convex neural networks (Amos et al. 2016) for implementing f?
Minor comments:
1. The proofs of Prop 1 & 2 are missing.
2. "However, using this KL divergence for VI has been criticized for under-estimating the uncertainty..." Any reference to this?
3. Figure 1: alpha is used before it is defined.
4. "Other existing definitions of alpha-divergences have dis-continuous gradients at the alpha values corresponding to KL divergences" -- missing reference.
5. "When D_0.5 is in use as the meta-loss, the corresponding log f*'' is analytical" -- could you elaborate on this?
------------------------------
After rebuttal:
I would like to thank the authors for their extremely detailed responses! The paper is now greatly improved.
I would like to increase my score from 3 to 5, however, there is no such option this year.
The reason why I am still hesitated to accept this paper is mainly because I am not sure how useful learning the divergence to VI is. For unsupervised learning, people use GAN or VQ-VAE to obtain realistic samples, but which is not attributed to better divergence measures. Plus, what we refer to "better" depends on the supplied meta-loss, which may not be better at all. Another unsatisfactory point of this paper in my opinion is that it only demonstrated results on toy datasets and MNIST. Although the proposed method is valid for alpha-divergence and f-divergence, I suggest the authors to find a real application which demonstrates the merit of the idea in practice.


We introduce a unified probabilistic approach for deep continual learning with open set recognition, based on variational Bayesian inference. Our model combines a joint probabilistic encoder with a generative model and a linear classifier that get shared across tasks. The proposed open set recognition bounds the approximate posterior by fitting regions of high density on the basis of correctly classified data points and balances open set detection with recognition errors. Catastrophic forgetting is significantly alleviated through generative replay, where the open set recognition is used to reject statistical outliers from low density areas of the class specific posterior. Our approach naturally allows for forward and backward transfer while maintaining past knowledge without the necessity of storing old data, regularization or inferring task labels. We demonstrate compelling results in the challenging continual learning scenario of incrementally expanding the singlehead classifier for both class incremental visual and audio classification tasks, as well as incremental learning of datasets across modalities. We further quantitatively demonstrate our models ability to successfully distinguish unseen unknown data from trained known tasks and subsequently prevent misclassification.

Summary: This paper proposes a unified model for continual learning and aims to address the following problems:
Out-of-train-domain dataset recognition
Catastrophic forgetting
The out-of-domain or open set recognition model is not only used to detect outliers but also for sampling “representative data” of previous tasks for forward (and backward) transfer.
COMMENT(S)/QUERIES:
------------------------------

While the experiments do justice to the contributions mentioned in the paper, I believe that certain sections need clarifications and expansion (while some can be dismissed).
1. Given my limited knowledge in the literature on this topic, I would have appreciated a proper related work section.
2. I believe equation (1) could have been moved to supplementary for reference since equation (2) is the only important equation in the paper.
I don’t believe Equation (1) is really used for testing purposes. It’s just that the data is sent through the probabilistic encoder and then classified. There is no need for reconstruction of the data point.
3. It’s difficult to follow the flow of the method. Shouldn’t generative replay (algorithm 3) be placed before open set recognition of unknown and uncertain inputs (algorithm 2), since the latter is probably just used during the test, while the former affects the training procedure directly (implicit data augmentation)?
4. Major concern: It’s somewhat strange to observe that the VAE model is able to generate multiclass data so fluidly with a simple gaussian prior. This kind of challenges the belief that current generative models are unable to capture all modes perfectly. A small note about why multimodal prior was not used (which intuitively and mathematically makes more sense) and also a statement about the average time to generate multiclass multi-modal data using algorithm 3 would have been nice.
5. In section 3.2, “For our single-head expanding classifier this ensures..”. While having a single-head expanding classifier is listed as one of the important contributions, it hasn’t been given enough justice w.r.t to the implementation details. Is it like adding a completely new classifier during the training of the new task?
The entire section on hyper-parameters could’ve been moved to the supplementary if space was a major constraint but compromising on details about an important contribution only weakens the paper.
I especially like figure 2 in the experiments, where the importance of Weibull CDF outlier rejection prior is highlighted.
Typo:
incorrect opening inverted comma for the word background in the introduction section (page 1)
OVERALL COMMENT
-----------------------------
The paper combines the generative, and discriminative models into one framework for multiple important tasks. While the contributions are clear in the introduction, the presentation of the paper is somewhat too complicated at a couple of places. It does not do full justice to explaining its more vital components and some parts of the method section feel like a jigsaw puzzle, where readers are heavily expected to “figure out on their own”.
With proper presentation though, I can envision this paper contributing positively to unified frameworks in general.
Due to the above reasons. I am giving it a score of 6.


Model-based imitation learning methods require full knowledge of the transition kernel for policy evaluation. In this work, we introduce the Expert Induced Markov Decision Process (eMDP) model as a formulation of solving imitation problems using Reinforcement Learning (RL), when only partial knowledge about the transition kernel is available. The idea of eMDP is to replace the unknown transition kernel with a synthetic kernel that: a) simulate the transition of state components for which the transition kernel is known (s r ), and b) extract from demonstrations the state components for which the kernel is unknown (s u ). The next state is then stitched from the two components: s = {s r , s u }. We describe in detail the recipe for building an eMDP and analyze the errors caused by its synthetic kernel. Our experiments include imitation tasks in multiplayer games, where the agent has to imitate one expert in the presence of other experts for whom we cannot provide a transition model. We show that combining a policy gradient algorithm with our model achieves superior performance compared to the simulation-free alternative.

* Paper summary.
The paper considers an IL problem where partial knowledge about the transition probability of the MDP is available. To use of this knowledge, the paper proposes an expert induced MDP (eMDP) model where the unknown part of transition probability is modeled as-is from demonstrations. Based on eMDP, the paper proposes a reward function based on an integral probability metric between a state distribution of expert demonstrations under the target MDP and a state distribution of the agent under the eMDP. Using this reward function, the paper proposes an IL method that maximizes this reward function by RL. The main theoretical result of the paper is that the error between value function of the target MDP and that of the eMDP can be upper-bounded. Empirical comparisons against behavior cloning on discrete control tasks show that the proposed method performs better.
* Rating.
The main contribution of the paper is the eMDP model, which enables utilizing prior knowledge about the target MDP for IL. While this idea is interesting, eMDP is too restrictive and its practical usefulness is unclear. Moreover, there are other issues that should be addressed such as clarity and experiments. I vote for weak rejection.
* Major comments:
- Limited practicality due to an assumption on the unresponsive transition kernel.
In eMDP, the unresponsive transition kernel is modeled from demonstrations by directive using the observed next states (in demonstrations) as a next state of the agent. This modeling implicitly assumes that the agent cannot influence the unresponsive part of the state space. This is too restrictive, since actions of the agent usually influence other parts of the state space such as opponents and objects. For instance, in pong, actions of the agent indeed influence trajectories of the ball. Due to this restrictive assumption, I think the practicality of eMDP is too limited.
Also, it is unclear what happens when the transition of unresponsive state space is stochastic. In this case, the unresponsive transitions of eMDP would be incorrect since eMDP assumes deterministic transitions as-is from demonstrations. Though I am not sure about this.
- The equality in Eq. (3) should be an upper-bound. The IPM is defined by the absolute difference between summations (expectations), but the right-hand side of Eq. (3) is summations (expectations) of the absolute difference. These two are not equal, and the right-hand side should be an upper-bound.
- The paper is difficult to follow. There are many skipping contents in the paper. Especially, in the description of eMDP and its solution, where the paper describes the optimal solution of eMDP (Section 2) before describing the eMDP model (Section 4). Also, it is unclear from my first reading pass what is the actual IL procedure. Including a pseudo-code would help.
- The state-value function’s error bound ignores a policy. The proof utilizes the state distribution P(s’|s). However, this distribution should depend on a policy. It is unclear to me what is the policy function used in this error bound. Does this error-bound hold for any policy or only for the optimal policy? In the case that it only holds for the optimal policy, this result still does not provide useful guarantees when the optimal policy is not learned, similarly to the result in Proposition 2.1.
- Empirical evaluation lacks strong baseline methods. The paper compares the proposed method against behavior cloning, which is known to perform poorly under limited data. The paper requires stronger baseline methods such as those mentioned in Section 3. Also, it is strange that the vertical axis in Figure 2 represents the reward function of eMDP, which is an artificial quantity in Eq. (5). The results should present the reward function of the target MDP, which is the ground-truth reward of the task.
* Besides the major comments, I have few minor comments and questions:
- Does the reward function in Eq. (5) correspond to the Wasserstein distance for any metric d_s? If it is not, then the value function’s error-bound for the Wasserstein distance is not applicable to this reward function.
- What is the reward function r(s) in the value function’s error-bound that we can control the Lipschitz constant? We do not know the ground-truth reward r(s,a) so it cannot be controlled. The reward r(s,s’) of eMDP is given by the metric d_s of the state space so again we cannot control it.
- What is the metric d_s used in the experiments?
- How do you perform BC without expert actions in the experiments?
- Confusing notations. E.g., the function r is used for r(s,a) (MDP’s reward), r(s, s’) (eMDP’s reward), and r(s) (reward in the error bound’s proof); these r’s have different meaning and should be denoted differently.
- Typos. E.g., “Lemma 2.1” in Section 5 should be “Proposition 2.1”, “Lemma A.1” should be “Proposition 2.1”. "tildeF". etc.
*** After authors' response.
I read the other reviews and the response. My major concern was the impracticality of eMDP. While I can see that the eMDP model provides advantages over BC, I still think that it is impractical since it assumes the agent cannot influence the unresponsive part. Moreover, extracting next-unresponsive states as-is from demonstrations is problematic with stochastic transitions, since actual next-unresponsive states can be different from next-unresponsive states in demonstrations even when the sames state and actions are provided. Given that the major concern remains, I still vote for weak rejection.


In probabilistic classification, a discriminative model based on Gaussian mixture exhibits flexible fitting capability. Nevertheless, it is difficult to determine the number of components. We propose a sparse classifier based on a discriminative Gaussian mixture model (GMM), which is named sparse discriminative Gaussian mixture (SDGM). In the SDGM, a GMM-based discriminative model is trained by sparse Bayesian learning. This learning algorithm improves the generalization capability by obtaining a sparse solution and automatically determines the number of components by removing redundant components. The SDGM can be embedded into neural networks (NNs) such as convolutional NNs and can be trained in an end-to-end manner. Experimental results indicated that the proposed method prevented overfitting by obtaining sparsity. Furthermore, we demonstrated that the proposed method outperformed a fully connected layer with the softmax function in certain cases when it was used as the last layer of a deep NN.

This paper proposes a classifier, called SDGM, based on discriminative Gaussian mixture and its sparse parameter estimation.
The method aims to have a flexible decision boundary due to the mixture model as well as to generalize well owing to the sparse learning strategy.
While the method incorporates distinct ideas in the literature such as increased model complexity and use of a sparse prior, the paper needs more clearer explanation of the method design and careful empirical investigation.
My major concerns are as follows.
1) Explanation of the learning algorithm of Section 2.2 is unsatisfactory.
Specifically, the objective function and the principle of algorithm are unclear.
My guess is:
* c, m and T, z are equivalent. (T, z are one-hot representation of c, m.)
* The objective is to maximize the conditional probability of equation (1) given labeled data (x, t).
* This maximization is carried out in a similar way to the EM algorithm where m (or z) is regarded as a latent variable. This gives equation (9).
* A MAP estimate of parameter w is calculated using the prior (8).
Assuming the points above, I am still unsure about the following points:
1-a) How is the sparsity induced?
The prior of w seems to have the l2 regularizer, but how are \pi and r pulled toward zero?
1-b) How is the update of \alpha (17) derived?
Does this strengthen the sparsity by increasing \alpha given small \hat{w}?
What is the "orthogonal component" of \Lambda?
1-c) Is structure in w ignored?
Equation (7) indicates that parameter w has a certain structure such as nonnegative s_{cmii} or the determinant |\Sigma_{cm}| interacting with s_{cmij}.
In other words, the degree of freedom in w assuming the Gaussian likelihood of x is smaller than H.
These structure would be violated if the gradient descent or Newton's method is applied.
Do you mean by *discriminative* that we can freely set the parameter w?
Then, this point should be emphasized.
2) Which parameter is learned when combined into NN?
Parameters w and \pi?
How are these parameters made sparse in the end-to-end learning?
3) Ablation test to investigate which aspect impacts the performance.
Section 4 describes SDGM incorporates disciminative model, mixture model, and sparse Bayesian parameter estimation.
It is more informative to provide empirical results to see the impact of each property by comparing SDGM with, for example, RVM, discriminative GMM and sparse GMM.
Some other commets follow.
* \sqrt{\alpha_{cmh}} may be missing from the numerator of (8).
* For what distribution the expectation with regard to z is taken in e.g. (9, 11, 12, 13, 16)?
* What is D for CIFAR-10 experiment? DenseNet seems to use D=1000 units for the fully connected layer. Did the authors adopt this value?


Identifying causes from observations can be particularly challenging when i) potential factors are difficult to manipulate individually and ii) observations are complex and multi-dimensional. To address this issue, we introduce "Back-to-Back" regression (B2B), a method designed to efficiently measure, from a set of co-varying factors, the causal influences that most plausibly account for multidimensional observations. After proving the consistency of B2B and its links to other linear approaches, we show that our method outperforms least-squares regression and cross-decomposition techniques (e.g. canonical correlation analysis and partial least squares) on causal identification. Finally, we apply B2B to neuroimaging recordings of 102 subjects reading word sequences. The results show that the early and late brain representations, caused by low-and high-level word features respectively, are more reliably detected with B2B than with other standard techniques.

The authors introduce a method (B2B) for disentangling effects of correlated predictors in the context of high dimensional outcomes. Their model assumes the outcomes are constructed by a linear transformation of a set of true causes plus measurement noise. Specifically, they fit the model Y=(XE+N)F, where X are the predictors, E is a binary matrix indicating the true causes, N is a noise term, and F is a mixing term. They provide a closed form solution the model fit based on a pair of l2-regularized regressions. They simulate from the given model and provide comparisons against least-squares regression, canonical correlation analysis, and partial least squares. They also apply the method to brain imaging data containing 102 individuals reading 120 sentences plus scrambled sentences, with the goal of inferring which features of the words have an effect on imaging results.
This paper appears to be technically sound, but it should be rejected based on 1) the relatively limited applicability of the model and 2) a lack of thorough experimentation indicating that this is an appropriate method under more general circumstances. It is odd that the model assumes the outcomes are measured without error. Instead, it is assumed that the causes are measured with error, and mixed via F. A more appropriate model may be: Y=(XE+N)F+M, where an additional noise term M allows for Y to be measured imprecisely. Viewed in this light, the model starts to look a lot like canonical correlation analysis. Consider a model Y=ZF+M, X=ZG+N, if dim(Z) = dim(X) and G is invertible, this can be re-written as Y=(X inv(G)-N)A+M, and we arrive at a similar model with specific restrictions about the structure of inv(G) (E will in general not be invertible, so they are not the same). It is particularly odd, then, that the authors do no comparisons against any regularized form of CCA, which would seem to be the most natural method to use in this circumstance. Moreover, in the simulations where they show B2B outperforms CCA, they use 1000 training samples with 10-100 possible causes. In their experiments on real data, where it seems the CCA results are much closer to the results that B2B gives, they have 2700 samples and 4 possible causes. This seems to imply the real data case might be better conditioned than the simulated case, so that regularization would have less of an impact.
In conclusion, the authors present a sound method for disentangling correlated possible causes when the outcome is high-dimensional. However, the authors do not provide enough evidence that this method is generally useful and better than established methods to merit acceptance to ICLR. A comparison to regularized CCA, application to more datasets and simulations under violations of the model would greatly improve the paper. I also have two minor points. 1) the word “causal” can mean many things, and here it refers specifically to disentangling correlated predictors, rather than confounding in observations or direction of effect. It would improve the paper to add some discussion of this point. 2) The comparisons in the experiments are done between E estimated from B2B and E=sum_j H_j^2 for other methods that do not directly estimate E. However, a more natural comparison might be against EF as this also includes estimates of the strength of influence of each observation, which is implicit in the sum above.


Neural architecture search (NAS), or automated design of neural network models, remains a very challenging meta-learning problem. Several recent works (called "one-shot" approaches) have focused on dramatically reducing NAS running time by leveraging proxy models that still provide architectures with competitive performance. In our work, we propose a new meta-learning algorithm that we call CoNAS, or Compressive sensing-based Neural Architecture Search. Our approach merges ideas from one-shot NAS approaches with iterative techniques for learning low-degree sparse Boolean polynomial functions. We validate our approach on several standard test datasets, discover novel architectures hitherto unreported, and achieve competitive (or better) results in both performance and search time compared to existing NAS approaches. Further, we provide theoretical analysis via upper bounds on the number of validation error measurements needed to perform reliable meta-learning; to our knowledge, these analysis tools are novel to the NAS literature and may be of independent interest.

This paper proposes a new algorithm for one-shot neural architecture search (NAS) via compressive sensing. The authors propose a new search strategy, as well as a slightly different search space compared to DARTS [1], ProxylessNAS [2], etc. They use architecture samples from the one-shot model evaluated with the search parameters as a surrogate of the true objective in order to speed-up the search. Afterwards, these surrogate function evaluations are used to compute Fourier coefficients which are eventually used to optimize the vector of binary parameters encoding the architecture.
Overall, I think the proposed algorithm is interesting and of practical usefulness. However, in terms of novelty, this work seems more to be an application of Harmonica [6] to the NAS problem (with small modifications in order to make it applicable). In page 10 you state some of the differences of your method with Harmonica. I agree that the number of function evaluations you use (coming from the one-shot model) is larger and computationally less expensive to obtain, however this does not guarantee that these are a good surrogate of the true objective that NAS aims to minimize, i.e. the validation/test accuracy of final (stand-alone) architectures . The empirical evaluations of their algorithm seem to outperform/be competitive compared to other NAS methods on all benchmarks used in the paper, however only DARTS is evaluated on their search space and the other results are taken from the corresponding papers. The paper is well-written and -structured with the caveat of being more than the recommended 8 pages.
I will adjust my score depending on the authors responses concerning the following questions/issues:
1. The correlation between the architectures evaluated using the one-shot weights and retrained from scratch, seems to be of crucial importance in your method, since you directly use the one-shot weights to collect the measurements, similarly to Random Search with weight sharing [3], ENAS [4] or Bender et al. [5]. What is the correlation of these measurements with the stand-alone architectures trained from scratch using the final evaluation settings? How did you tune the p in the Bernoulli distribution during the one-shot weight updates. According to Bender et al. [5] the ScheduledDropPath probability is an important hyperparameter affecting the aforementioned correlation.
2. What is the main motivation for using 5 operations in the operation set and not 8 as in DARTS [1] for example? Does the main contribution in the competitive results come from the different search space or the search method?
3. Is there any reference or proof for the correctness of Theorem 3.2?
4. I think there are some parts that can be moved in the Supplementary, such as the pseudocode for the proposed algorithm or Figure 3, and some other parts that can be compressed, such as the Related Work section.

References
[1] Hanxiao Liu, Karen Simonyan, and Yiming Yang.  DARTS: Differentiable architecture search.  In ICLR, 2019.
[2] Han Cai, Ligeng Zhu, Song Han. ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In ICLR, 2019.
[3] LIAM LI, AMEET TALWALKAR. Random Search and Reproducibility for Neural Architecture Search.
[4] Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, Jeff Dean. Efficient Neural Architecture Search via Parameter Sharing. In ICML, 2018
[5] Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, Quoc Le. Understanding and Simplifying One-Shot Architecture Search. In ICML, 2018
[6] Elad Hazan, Adam Klivans, Yang Yuan. Hyperparameter Optimization: A Spectral Approach


We study how the behavior of deep policy gradient algorithms reflects the conceptual framework motivating their development. To this end, we propose a finegrained analysis of state-of-the-art methods based on key elements of this framework: gradient estimation, value prediction, and optimization landscapes. Our results show that the behavior of deep policy gradient algorithms often deviates from what their motivating framework would predict: the surrogate objective does not match the true reward landscape, learned value estimators fail to fit the true value function, and gradient estimates poorly correlate with the "true" gradient. The mismatch between predicted and empirical behavior we uncover highlights our poor understanding of current methods, and indicates the need to move beyond current benchmark-centric evaluation methods.

[Summary]
This paper empirically studies the behavior of deep policy gradient algorithms during the optimization. The conclusion is that, while these methods generally improve the policy, their behavior does not comply with the underlying theoretical framework. First, sample gradients obtained with a reasonable batch size have little correlation with each other and with the true gradient. Second, a larger batch size requires a smaller step-size. Third, the value baseline is far from true values and only marginally reduces variance, yet it considerably helps with optimization. Finally, the optimization landscape highly varies with the choice of objective function and the number of samples used to estimate it.
[Decision]
I vote for acceptance. To the best of my knowledge, the findings of this paper are new and not predictable by the current theory. These negative results have some merit as they call for theory that explains the behavior of these algorithms, or an algorithm whose behavior is predictable by the current theory. The paper is well-written, with a few small issues in presentation that should to be addressed in the final revision.
[Comments]
In Fig. 4 (b) it does not look like that the value error is high. It is said that "the learned value function is off by about 50% w.r.t. the underlying true value function." This sentence should be clarified or visualized.
What is \pi in Eq (13) in A1? If it is the agent's current policy, how is it different than \pi_\theta? If \pi corresponds to the distribution of state-action pairs in the replay buffer, how can one obtain a policy \pi that has led to this distribution of states in order to construct the importance sampling ratio?
In 2.2, the claim that a learned value baseline results in significant improvement in performance should be supported by results or reference to previous work.
Figs. 6 and 7 compare the loss surface with different objectives and sample regimes. Do these factors (objective and sample size) affect the part of the parameter space that is visualized (by changing the origin and the update direction), or are they only used to evaluate the values on the z-axis for the same area in the parameter space? Observing a different landscape in a different part of the parameter space is not surprising.
[Minor comments]
- Is V_\theta_{t-1} in Eq (4) a function of state? If so, a (s_t) is missing before the plus sign.


While supervised object detection and segmentation methods achieve impressive accuracy, they generalize poorly to images whose appearance significantly differs from the data they have been trained on. To address this in scenarios where annotating data is prohibitively expensive, we introduce a self-supervised approach to detection and segmentation, able to work with monocular images captured with a moving camera. At the heart of our approach lies the observations that object segmentation and background reconstruction are linked tasks, and that, for structured scenes, background regions can be re-synthesized from their surroundings, whereas regions depicting the object cannot. We encode this intuition as a self-supervised loss function that we exploit to train a proposal-based segmentation network. To account for the discrete nature of the proposals, we develop a Monte Carlo-based training strategy that allows the algorithm to explore the large space of object proposals. We apply our method to human detection and segmentation in images that visually depart from those of standard benchmarks, achieving competitive results compared to the few existing self-supervised methods and approaching the accuracy of supervised ones that exploit large annotated datasets.

This submission proposes a self-supervised segmentation method, that learns from single-object videos by finding the region where it can segment an object, remove the entire bounding box around it, inpaint it, then finally put the object back. The loss is a balance between a reconstruction error and a negative inpainting error (high error means object is probably present, due to the weak correlation with the background).
My decision is Weak Accept. I like the method very much and think it’s a clever and well-executed algorithm. The reason for being Weak is because the experimental evidence could be stronger, especially comparing with Croitoru et al. and Rhodin et al. The paper leaves some open problems, inviting future work to be built on top of it (i.e. leveraging time more and handling multiple objects better). I think it should be accepted to promote such future work.
Method:
The method is interesting and clever. Similar efforts have been made, such as Bielski & Favaro and Crawford & Pineau. However, key contributions relax some of the contrived requirements of these past methods (e.g. simple foreground translation over background; requirement of plain background). Thanks to the inpainter, the importance sampler, and avoidance of collapsing into trivial solutions, this paper is able to put together a method that works on moving cameras and fairly complex scene semantics (still limited to one object though). Actually getting this to converge with only two loss terms balanced against each other is impressive. Having certain heads of the network trained only on one of the two terms seems to be a key contribution.

Experimental results:
The comparison with Rhodin et al. on H36M is characterized as “slightly” lower, but I would call 71% vs. 58% a significant difference. Of course, I understand that Rhodin et al. relies on the static background, so this is not a fair comparison.
Ski-PTZ-Dataset should then offer a better comparison, but here the method struggles to compete with Croitoru et al. 2019, which has a 11-point higher F-measure. On Handheld190k, the dataset proposed in this paper, is where the method finally shines, but still only offers a 1-point F-measure improvement over Croitoru et al. That being said, considering how different the methods are, and how Croitoru et al. requires two-stage training, there are many benefits to this method. Croitoru et al. also relies on video to extract the object features, and this requirement is not as explicit in this work. Actually, that brings me to one question I had. The paper states that “as long as videos or picture collections of a single object in front of the same scene are available.” I didn’t quite understand why this must be trained on multiple images of the same scene. If the inpainter is general to any background scenery, couldn’t it work on single-images as well? The conclusion even says you do not use temporal cues.
Other:
I don’t think it’s that meaningful to include precision/recall in the tables. It is also not that meaningful to point out that your method’s precision is higher than that of Croitoru et al., when the F-measure is shy 11 points. The reason is because many points on the precision/recall could be constructed simply by applying for instance a gamma curve on the segmentation predictions. The high precision is clearly at the cost of a low recall, and another point on this tradeoff curve could be presented. This is why F-measure and average precision are much better.


Feature maps contain rich information about image intensity and spatial correlation. However, previous online knowledge distillation methods only utilize the class probabilities. Thus in this paper, we propose an online knowledge distillation method that transfers not only the knowledge of the class probabilities but also that of the feature map using the adversarial training framework. We train multiple networks simultaneously by employing discriminators to distinguish the feature map distributions of different networks. Each network has its corresponding discriminator which discriminates the feature map from its own as fake while classifying that of the other network as real. By training a network to fool the corresponding discriminator, it can learn the other network's feature map distribution. Discriminators and networks are trained concurrently in a minimax twoplayer game. Also, we propose a novel cyclic learning scheme for training more than two networks together. We have applied our method to various network architectures on the classification task and discovered a significant improvement of performance especially in the case of training a pair of a small network and a large one.

= Summary
This paper presents a new deep mutual learning (i.e., online peer-teaching) method based on Knowledge Distillation (KD) in a feature map level. The target task is similar with the original KD in the sense that the a network is taught by another network as well as groundtruth labels, but different with the KD in the sense that the networks are not a (frozen) teacher and a student but teaching each other in an online manner. Most approaches in this relatively new line of research rely on logit-based KD for transferring knowledges between networks, and the paper demonstrates that by an additional feature map level KD the performance can be further improved.
= Decision
The current decision is borderline in my mind, but officially weak accept. Although the proposed method is simple and consists of known ideas, it is designed convincingly and enhances performance practically. Also, I believe the target task itself is worth to be introduced as a next direction of KD. However, the submission is weak in terms of novelty and the manuscript should be polished carefully.
= Comments
1) Weak clarity
- The main motivation and advantages of the deep mutual learning is not well introduced in Section 1. Although the two papers (i.e., ONE and DML) are cited here, it would be much better to explicitly describe the main idea and motivation of peer-teaching, what is the difference between the task and the original KD, and the achievements in the previous work. Without these, readers, including me, may get confused why the online KD is required and why there is no clear teacher-student relationship between networks.
- In a similar context, the motivation of introducing more than two student networks should be given.
- The architecture of the discriminator seems not described even in the appendix.
- The meaning of various arrow types in Figure 1 is not clearly described.
2) Insufficient experiments
It would be better to report the performance of vanilla and (offline) KD in Table 1 to show more clearly that the feature map alignment is useful and that online KD is better than its offline counterpart.
3) Limited novelty and performance improvement
- The main idea is already introduced in previous work on the task and the feature map level KD has been studied widely for various applications, their combination is somewhat new though.
- The performance improvement by the proposed feature map level KD seems marginal as shown in Table 2.
- The performance gap between DML and the proposed model seems also marginal.