When constructing random forests, it is of prime importance to ensure high accuracy and low correlation of individual tree classifiers for good performance. Nevertheless, it is typically difficult for existing random forest methods to strike a good balance between these conflicting factors. In this work, we propose a generalized convolutional forest networks to learn a feature space to maximize the strength of individual tree classifiers while minimizing the respective correlation. The feature space is iteratively constructed by a probabilistic triplet sampling method based on the distribution obtained from the splits of the random forest. The sampling process is designed to pull the data of the same label together for higher strength and push away the data frequently falling to the same leaf nodes. We perform extensive experiments on five image classification and two domain generalization datasets with ResNet-18, ResNet-50 and DenseNet-161 backbone networks. Experimental results show that the proposed algorithm performs favorably against state-of-the-art methods.

The paper proposes a scheme to learn representations and a random
forest model in a closely coupled manner, where the representations
are learned to simultaneously improve the predictive performance of
the individual trees in the random forest and reduce the correlation
between these trees. This is achieved via a novel triplet sampling
scheme where the positive samples are used to improve the predictive
performance of the individual trees, while the negative samples are
used to reduce the correlation between the trees. The empirical
results indicate the improvement of the proposed scheme over random
forests learned over the original representations across various
datasets.
The manuscript is well presented and I am leaning towards accept, one
major issue with this manuscript is the number of pages which crosses
the 8 page recommended limit in the ICLR CfP. Given that higher
standard, I feel that there are multiple minor issues which should be
addressed.
- It is not clear why the proposed scheme is especially well suited
  for "domain generalization" and "visual recognition" and not for
  general supervised classification tasks (like those on CIFAR, SVHN,
  etc). This needs to be clarified.
- The theoretical results for random forest rely on the independence
  of the individual trees. Given that the trees in the random forest
  are no longer independent in the proposed scheme, the upper bound in
  Equation (1) may no longer be valid. While this does not affect the
  empirical performance, it might be good to discuss the theoretical
  implications of the proposed scheme.
- It is not clear why the curves for the canonical features in Figure
  3 not improving with the number of iterations (which corresponds to
  number of trees to the best of my understanding). The results in
  Table 1 do indicate improvement in the performance with canonical
  features, putting Figure 3 and Table 1 at odds.
Minor:
- The individual trees in a random forest are supposed to have low
  bias but high variance, and the heterogeneity between the trees is
  supposed to squash the variance with the bagging. The introduction
  mentions that the individual trees in the forest are "less
  biased". Maybe this is because of our different definitions of bias
  but any clarification here would be helpful.


The importance weighted autoencoder (IWAE) (Burda et al., 2016) is a popular variational-inference method which achieves a tighter evidence bound (and hence a lower bias) than standard variational autoencoders by optimising a multi-sample objective, i.e. an objective that is expressible as an integral over K >

UPDATE: bumping up my score after the revisions
---
Nice connections but novelty and practical takeaways unclear
SUMMARY OF THE PAPER:
This paper views recent IWAE-based [1] methods (IWAE-STL [2], IWAE-DREG [3], RWS [4, 5]) for training generative models p and inference networks q under a common framework, AISLE.
This heavily relies on the "double-reparameterization" property by [2] and is restated in Lemma 1.
This framework makes it explicit that we're interested in
1) maximizing the (log) marginal likelihood wrt p parameters, and
2) minimizing some divergence between the posterior in the learned model to q.
In AISLE, IWAE-STL's q-gradient is viewed as a doubly-reparameterized self-normalized importance sampling (SNIS) estimate of KL(p || q).
This is in contrast to viewing it as a biased estimator of the IWAE's q-gradient.
This can potentially explain why it performs well when number of SNIS samples are increased.
It is also some evidence against the fact that having no unified objective is bad (because there isn't evidence of IWAE-STL diverging despite there being no unified objective).
IWAE-DREG's q-gradient is viewed as a doubly-reparameterized SNIS estimate of X-divergence(p || q) (up to multiplicative constant of the number of SNIS samples).
This is in contrast to viewing it as an unbiased estimator of the IWAE's q-gradient.
The view on RWS is unchanged: the q-gradient is a SNIS estimate of KL(p || q).
For me, the main contribution is viewing IWAE-STL and IWAE-DREG q-gradient estimators as biased gradients of an explicit divergence rather than of the IWAE objective.
I also found the observation that the signal-to-noise (SNR) decrease in IWAE's q-gradient can be proved by noting that it is a SNIS estimator of a zero vector nice.
STRUCTURE:
The article is well-written and easy to understand.
NOVELTY:
A different view on IWAE-STL and IWAE-DREG is interesting and novel (as mentioned above).
This means that IWAE-STL and IWAE-DREG are good not only because they reduce gradient variance (as previously understood) but also potentially because they directly target a divergence.
Viewing generalization of RWS as a main contribution (first bulletpoint of Section 1.2: "...we show that AISLE admits RWS as a special case.") is a bit of a stretch since this generalization is very straightforward from the way RWS is formulated.
The recommendation of using RWS-style algorithms over IWAE as given in the abstract ("we argue that directly optimising the proposal distribution in importance sampling as in the RWS algorithm is preferable to optimising IWAE-type multi-sample objectives) is also not novel since this is also advocated by [5] (section 3.2: "This makes RWS a preferable option to IWAE for learning inference networks because the phi updates in RWS directly target minimization of the expected KL divergences from the true to approximate posterior").
The recommendation as a method for non-reparameterisable latent variables at the end of section 1.2 ("as well as further algorithms which do not require reparameterisations") is also given in [5].
Are there different adaptive importance sampling algorithms that could be used within AISLE that would improve on IWAE-STL/IWAE-DREG/RWS?
EXPERIMENTS:
There are no experiments in the main paper.
However, experiments that would support/falsify the following points could be good:
- RWS and IWAE-STL don't suffer from non-unified objectives because IWAE-STL has non-unified objectives but doesn't diverge,
- [targeting direct divergence] is more useful than (or as useful as) [lower variance gradient estimators].
CONCLUSION:
While I really like the presentation and connections made in the paper, I'm not sure what the practical takeaways are (other than use IWAE-STL, IWAE-DREG, RWS over IWAE which is advocated by [2], [3], [5]).
I'm giving this a weak accept due to the former.
I'm willing to bump up my score if
- the paper is modified to more accurately reflect the contributions or
- there are experiments that provide additional support for the [targeting direct divergence] view in addition to [2, 3, 4, 5], or
- there is a new practical algorithm that the AISLE generalization would suggest that is better than IWAE-STL, IWAE-DREG, RWS in some respects.
[1] Importance Weighted Autoencoders. https://arxiv.org/abs/1509.00519
[2] Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference. https://arxiv.org/abs/1703.09194
[3] Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives. https://arxiv.org/abs/1810.04152
[4] Reweighted Wake-Sleep. https://arxiv.org/abs/1406.2751
[5] Revisiting Reweighted Wake-Sleep for Models with Stochastic Control Flow. https://arxiv.org/abs/1805.10469
[6] Variational Inference via χ-Upper Bound Minimization. https://arxiv.org/abs/1611.00328
[7] Tighter Variational Bounds are Not Necessarily Better. https://arxiv.org/abs/1802.04537


Overfitting is a common issue in machine learning, which can arise when the model learns to predict class membership using convenient but spuriouslycorrelated image features instead of the true image features that denote a class. These are typically visualized using saliency maps. In some object classification tasks such as for medical images, one may have some images with masks, indicating a region of interest, i.e., which part of the image contains the most relevant information for the classification. We describe a simple method for taking advantage of such auxiliary labels, by training networks to ignore the distracting features which may be extracted outside of the region of interest, on the training images for which such masks are available. This mask information is only used during training and has an impact on generalization accuracy in a dataset-dependent way. We observe an underwhelming relationship between controlling saliency maps and improving generalization performance.

Summary
---
(motivation)
CNN image classifiers tend to overfit to distractor patterns.
Perhaps these patterns are spatially local, such that in most images signal is at one
location while the noise models tend to overfit to is somewhere else.
If so, then generalization should improve if models are given additional
supervision (i.e., a mask identifying salient regions) that specifies where the signal is and is not.
This paper designs the Actdiff loss to realize this intuition.
(approach)
Actdiff:
1) The Actdiff loss requires a mask that highlights areas of the input
image which have signal and not distractor regions. It extracts features from
the original input image and its masked version then encourages the two features
to be similar at every layer of the CNN using an L2 loss.
Actdiff is compared to 5 other methods including a reconstruction loss and Gradmask (previous work).
(evaluation - synthetic dataset)
A synthetic dataset is constructed for a simple binary classification task based on the presence of simple shapes.
Two patterns can predict the correct class at train time, but one of the patterns is removed at test time so additional information (masks in this case)
is required to specify which pattern the classifier should use.
All the losses except Actdiff achieve at best 50% accuracy on the val set, but Actdiff gets 80% or more accuracy on 3 of 4 tested model variations.
This shows that Actdiff can effectively introduce the relevant masking information.
(evaluation - Medical Segmentation Decathalon)
This dataset provides 3 segmentation tasks (Liver, Cardiac, Pancreas), including ground truth masks for those images.
All 6 methods outperform all the others at least some of the time.
The conclusion is that adding mask information using Actdiff doesn't improve segmentation performance.
(evaluation - Multi-Site dataset)
A final task tries to construct another synthetic dataset out of real X-ray images
collected at two different places.
The train set is largely from one place and the test set is mostly from the other place, and masks are constructed so Actdiff can try to eliminate this bias.
Actdiff causes a negligible increase in performance.
The paper concludes that signals CNNs tend to fit to in this type of data are not very spatially distinct.
Strengths
---
There is some novelty in the approach. The actdiff loss makes sense and masking + activation mapping have not been tried together before to my knowledge.
Experiments follow a logical progression, starting by verifying the idea on a synthetic dataset, then moving to real data, and then evaluating on a half-synthetic dataset designed to debug the approach.
Experiments average over many random initializations.
The paper embraces its negative result.
Weaknesses
---
The approach is not very compelling to me:
* Implicit in this paper is that any information outside the mask is a distractor and any information inside is not a distractor. Why should the particular masks chosen for the experiments have this property? How can an expert know which features a model will find useful?
The paper's novelty is somewhat limited. The idea of regularizing using saliency maps has been explored and even applied to medical data like the MSD used here in Gradmask (one of the strong baselines this paper compares to). Activation matching is also common (e.g. [1]), though it has not been combined with masking before.
While the weights applied to the various losses are provided, it's not clear how they were tuned. In this case there may be lots of competing losses, so it's important to tune the weights somehow to ensure the tradeoff between losses is optimal.
The results (and the conclusions) suggest Actdiff is not very effective at increasing generalization. Table 2 reports test results on all datasets. In that table, each loss outperforms all the other losses in at least two cases (a case is a model-dataset pair).
[1]: Gatys, Leon A. et al. “A Neural Algorithm of Artistic Style.” ArXiv abs/1508.06576 (2015): n. pag.
Missing experiments:
* In the Multi-Site experiment, compare to what happens when the circular mask is applied to all images and not just those from one site. This is a necessary control to be sure that any benefits from masking are due to domain transfer and not other regularization effects. Either conclusion could be useful, but it would be nice to know.
Presentation weaknesses:
* In the synthetic dataset the model cannot tell the difference between correct and incorrect signals at train time. Therefore, I think there's no way for some of the baselines (plain classifier, autoencoder) to generalize correctly. Is that right? If so, it should be clear that comparisons to these baselines are not fair when discussing the synthetic evaluation in section 4.
Missing details / points of clarification:
* What is the Conv AE? I assume it is a CNN based autoencoder of some sort. A detailed description of the non-standard architectures would be useful for reproducibility, though probably only in the appendix.
* What are the lambda hyperparameters? I assume these are weights on the corresponding loss terms, but this is never made explicit.
* How does f(.) relate to the function o_l(.)? Is o_l(.) an intermediate step in f(.)?
* The MSD dataset is not clearly described. Is this a classification dataset where classes are different diseases? What do the ground truth masks capture?
* I think only Conv AE and UNet contain reconstruction losses. This presentation is a bit confusing since reconstruction loss was presented as another loss and it shows up in the tables implicitly based on the architecture being compared.
Suggestions
---
* This paper would be a bit more convincing if it started with a concrete example of the problem illustrated on some dataset (e.g., maybe an example from Gradmask). That may also help drive intuitions later on in the paper.
Preliminary Evaluation
---
Clarity: The paper is fairly clear.
Quality: Quality is mixed. Lots of relevant experiments are reported but they don't support clear conclusions and I'm not sure how well the models were tuned.
Originality: There is some novelty, but it is limited as discussed above.
Significance: I see limited significance.
For me this paper requires special scruitiny because it presents a negative result. Here are some factors that come to mind when thinking about whether to publish a negative result:
* Is the approach compelling? - This approach is not very compelling (e.g., comments about limited novelty and lack of concrete examples to boost intuition).
* Are the experiments thorough? - The experiments could be significantly more thorough (e.g., comments about tuning lambda).
* Will readers learn something useful? - This paper may help researchers trying to leverage similar intuitions, but it won't be very useful outside this audience.
* Does the paper present experiments that promote deeper understanding of why the approach failed? - This paper makes significant reasonable steps in that direction with sections 4 and 6, but I was still a bit dissapointed with the conclusions of these sections.
* Does the paper discuss alternative approaches that were investigated? - Many alterantive approaches were considered and their performance reported.
Overall I think this paper is close but fails to meet the bar because it does a bit worse than expected on most criteria above.


Video prediction is a challenging task: models have to account for the inherent uncertainty of the future. Most works in the literature are based on stochastic imageautoregressive recurrent networks, raising several performance and applicability issues. An alternative is to use fully latent temporal models which untie frame synthesis and dynamics. However, no such model for video prediction has been proposed in the literature yet, due to design and training difficulties. In this paper, we overcome these difficulties by introducing a novel stochastic temporal model. It is based on residual updates of a latent state, motivated by discretization schemes of differential equations. This first-order principle naturally models video dynamics as it allows our simpler, lightweight, interpretable, latent model to outperform prior state-of-the-art methods on challenging datasets.

Summary:
The paper proposes a video prediction method based on State-Space Models. The paper describes two main contributions:
1. By learning dynamics in the latent state space, the method avoids the high computational cost and accumulating image reconstruction errors of autoregressive models that condition on generated frames.
2. To model dynamics, the paper proposes a residual update rule inspired by Euler’s method to solve ODEs. According to this rule, the update to the latent state y_t is modeled as an additive residual f(y_t, z_{t+1}). This has the advantage that the step size of the discretization can be adjusted freely, e.g. between training and inference.
The paper provides extensive experimental comparison of their model to the SVG and SAVP models on several standard datasets. The paper further contains experiments illustrating features of the model such as disentangling dynamics and content, and interpolation of dynamics in the latent space.
Decision:
The paper is written clearly and the mathematical treatment and experiments appear rigorous. The idea of predicting video using state-space models is interesting and promising. However, as described below, the paper overstates its novelty and falls short of showing the advantages of the method beyond incremental improvements on frame-wise image quality metrics. I therefore suggest rejection in its current version.
Supporting arguments and suggestions:
1. The idea to use fully latent models for video prediction, to untie frame synthesis and dynamics, is not new and the paper does not fully cite this literature. For example, [1] and [2] perform unsupervised, non-autoregressive video prediction. The differences to these models should be discussed.
2. The advantages of the residual update rule are not made clear enough. The parallels to the ODE literature seem tenuous. The main advantage described in the paper is the ability to synthesize videos at different frame rates, but interpolation over such short time horizons is not a hard problem. At least, the paper should compare to existing methods for frame interpolation. Apart from interpolation (variable step size), it appears that the update rule could be changed from y_{t+1} = y_t + f(y_t, z_{t+1}) to y_{t+1} = f(y_t, z_{t+1}) without impact to the model. How is it different from the standard VRNN formulation [3]? More experiments to show the advantage of the proposed update rule would be helpful.
3. Some of the experiments seem like interesting starting points but do not support general claims. For example, Fig 2 (b) shows that the proposed dynamics model is better than an MLP or GRU on deterministic Moving MNIST, but is this also true on real datasets, which have much more complex dynamics? Similarly, the interpolation in Figure 9 is intriguing, but it would be helpful to describe and test how this ability is useful for applications of the predictive model.
4. The comparisons use frame-wise metrics of image quality (PSNR, SSIM, LPIPS). Even though they are common in the literature, these metrics are unsuitable for comparing long video sequences due to their stochasticity. The metrics are probably dominated by relatively uninteresting features such as the quality of the static background. Metrics for comparing entire videos exist (e.g. FVD [4]) and should be used. Even better, the paper should demonstrate the usefulness of the model for downstream tasks such as reinforcement learning, although I understand that this may be out of scope.
Minor comments:
- As far as I know, the correct term for error terms is residual, not residue.
- What do the error bars in the figures show? Please add this information to the figure legends.
[1] Wichers et al., 2018, https://arxiv.org/pdf/1806.04768.pdf
[2] Minderer et al., 2019, https://arxiv.org/abs/1906.07889
[3] Chung et al, 2015, https://arxiv.org/abs/1506.02216
[4] Unterthiner et al., 2018, https://arxiv.org/abs/1812.01717


We study the interplay between memorization and generalization of overparameterized networks in the extreme case of a single training example and an identitymapping task. We examine fully-connected and convolutional networks (FCN and CNN), both linear and nonlinear, initialized randomly and then trained to minimize the reconstruction error. The trained networks stereotypically take one of two forms: the constant function (memorization) and the identity function (generalization). We formally characterize generalization in single-layer FCNs and CNNs. We show empirically that different architectures exhibit strikingly different inductive biases. For example, CNNs of up to 10 layers are able to generalize from a single example, whereas FCNs cannot learn the identity function reliably from 60k examples. Deeper CNNs often fail, but nonetheless do astonishing work to memorize the training output: because CNN biases are location invariant, the model must progressively grow an output pattern from the image boundaries via the coordination of many layers. Our work helps to quantify and visualize the sensitivity of inductive biases to architectural choices such as depth, kernel width, and number of channels.

This paper studies the inductive bias of neural nets by considering the toy example of learning an identity map through a single data point (and hence the NNs are always overparametrized). The authors compare CNNs versus FCNs, and find that CNNs tend to “generalize” in terms of actually learning the concept of an identity, whereas FCNs are prone to memorization. The authors also present results under various different settings such as changing the filter size or the number of hidden channels of CNNs. The conclusion is that the simpler the network architecture is, the better it generalizes. Another observation is that deep CNNs exhibit extreme memorization.
Overall, this is a well-written paper with an interesting set of experiments. However, I do have several concerns regarding the generality of the observed phenomenon in this paper. The first one is that the authors have chosen the comparison between CNNs and FCNs. In this case, I think a more fair comparison is to restrict the number of links (number of nonzero entries in the weight matrices) for both to be the same. In the current setup, however, the authors seem to consider the same number of hidden neurons, which naturally grants advantages to CNNs as they are of lower complexity.
Second, the training data is always a simple image from MNIST, and I am unsure how much this setting can generalize to other tasks, such as different data (say, music or text) or more complicated images (how do these experiments compare when we use a single data from ImageNet or CIFAR to train?). For instance, since CNNs are designed to capture invariances in natural images, it is unsurprising that they can generalize better on image data, but it would be quite astonishing if the same still holds true for acoustic data. In that case, the conclusion of this paper can be strengthened from “CNNs generalize better on image data” to “CNN generalize better”. Given the scope of the current paper, however, the best we can conclude is that “CNNs generalize better on MNIST”.
Last, again regarding the fair comparison, when comparing deep CNNs versus shallow ones, it is also of interest to see that, when restricted to the same number of parameters, if the deep CNNs still exhibit worse generalization. Otherwise, if some network complexities keep growing, we cannot really tell whether it is the network architecture that induces the inductive bias or it is simply the effect of complexities.
I also would like to suggest a future direction based on the idea in this paper: Comparing the inductive bias of GD versus SGD, a subject of intense study in the current literature. Since the authors considered a single training data, the results in this paper are always for GD. Now, say let us use 5 data, and compare the training of CNNs with different batch-size. Do the results differ? I think such a thought experiment would shed some light on the mysterious behaviors of the first-order algorithms that are widely used in practice.
Finally, a question: another submission to ICLR2020 [1] seems to suggest that optimization methods do not play a role in generalization, which is the opposite observation of this paper. Do the authors have any insight towards this contradiction?
[1] Fantastic Generalization Measures and Where to Find Them https://openreview.net/forum?id=SJgIPJBFvH


The relational model is a ubiquitous representation of big-data, in part due to its extensive use in databases. However, recent progress in deep learning with relational data has been focused on (knowledge) graphs. In this paper we propose Equivariant Entity-Relationship Networks, a general class of parameter-sharing neural networks derived from the entity-relationship model. We prove that our proposed feed-forward layer is the most expressive linear layer under the given equivariance constraints, and subsumes recently introduced equivariant models for sets, exchangeable tensors, and graphs. The proposed feed-forward layer has linear complexity in the data and can be used for both inductive and transductive reasoning about relational databases, including database embedding, and the prediction of missing records. This provides a principled theoretical foundation for the application of deep learning to one of the most abundant forms of data.

The authors extend recent work in equivariant set encoding to the setting of entity-relation data.  Similar to the cited previous work, they encode sets of objects (in this case the tuples of a relational database) with a permutation invariant function. They use a parameter tieing scheme to enforce this invariance.
I don’t find the paper to be particularly well motivated. Relational DBs are not necessarily a setting where you would always want equivariance. While the ordering of tuples do not matter, relations are often asymmetrical. Further, the idea of concatenating all tuples of a relational DB to be passed through a particular feed forward layer is infeasible for all but the smallest datasets. Real world databases such as knowledge bases contains millions to billions of entries. Scaling issues aside, the experiments do not actually show that this method outperforms any reasonable baselines such as a simple tensor factorization.
I also found it particularly hard to follow the descriptions of the methods. A few specific points:
- The text and notation in the beginning of section 2 could be a lot clearer. I had to read these paragraphs multiple times to pick out precisely what you were trying to say. Maybe have the set of relations be [R] like your other sets rather than a different script R.
- In the next paragraph you say “A particular relation R is a multiset if it contains multiple copies of the same entity” but you previously defined R to be a set of instances and not entities. I think you need to be more consistent with your terminology since the differences between type level entities and instances is important for your definitions and as you noted in your first footnote, these terms are often used very differently.
- This explanation could be helped a lot by improving the figure. The caption and images are both very dense but still requires a lot of coreference. For example, labeling the entities and relations with their ids in figure 1a directly would make it much easier to mentally map to what you are explaining in the caption. Also figure 1c is not at all clear.
Lastly, the methodology seems quite incremental over the previous work. A lot of the context and background that was sent to the appendix should be included in the main paper, particularly the relation to related work
edits:
double ‘the’ in abstract “linear complexity in the the data “


For numerous domains, including for instance earth observation, medical imaging, astrophysics,..., available image and signal datasets often irregular spacetime sampling patterns and large missing data rates. These sampling properties is a critical issue to apply state-of-the-art learning-based (e.g., auto-encoders, CNNs,...) to fully benefit from the available large-scale observations and reach breakthroughs in the reconstruction and identification of processes of interest. In this paper, we address the end-to-end learning of representations of signals, images and image sequences from irregularly-sampled data, i.e. when the training data involved missing data. From an analogy to Bayesian formulation, we consider energy-based representations. Two energy forms are investigated: one derived from auto-encoders and one relating to Gibbs energies. The learning stage of these energy-based representations (or priors) involve a joint interpolation issue, which resorts to solving an energy minimization problem under observation constraints. Using a neural-network-based implementation of the considered energy forms, we can state an end-to-end learning scheme from irregularly-sampled data. We demonstrate the relevance of the proposed representations for different case-studies: namely, multivariate time series, 2 images and image sequences.

The paper proposes a framework to learn representations of signals when they the signals are irregularly sampled.  They propose to do this by using some modified iteration steps from DINEOF algorithm. In addition to this, they propose a new energy function which is inspired by the idea of Gibbs distribution. The parametrize the energy function by some convolutional filters with a constraint. As claimed by the paper in the introduction, they only use the under sampled signals to learn as opposed to using the fully sampled ground truth signals in the previous deep learning basic approaches.
I choose to reject this paper. 1) The paper doesn't justify what these representations are or where these representations will be used 2) There is no theoretical or extensive empirical motivation that the representation that is coming out of this network is actually useful for downstream tasks of interest. 3) Ill-defined evaluation metrics 4) Missing comparisons with basic models - why not compare with in-painting models for images and time series in experiments? 5) The paper is hard to read, not well organized and missing a lot of details.
Main Argument:
It is not clear to me the motivation for learning these representations or how the usefulness of this representation will be evaluated. The only place where I see a use for representation other than reconstruction of the signal is in Table 1. They use a 3 layered MLP on top of the representation from auto encoder. Why is this interesting? If you're trying to claim that the representation is powerful enough, shouldn't you be able to do the classification with just a linear classifier? Can you justify the usefulness of representation for some downstream tasks in your other experiments?
In any case, I don't understand why the authors don't compare with a simple auto-encoder. The authors assumes that they know where the signal is sampled and where is is not. Why not train an auto-encoder with MSE loss, but calculating MSE over only those pixels which the authors know are sampled. Comparison with such a model will be helpful in understanding if the framework is adding anything of value.
Authors useful I-Score, R-Score, AE-Score and C-Score to compare across models. These metrics are not defined in the main text (or any reference) but there is a loose (english) definition in the caption of Table 1. For example, R-Score, is the "reconstruction performance of known image areas". Why are these numbers in percentages? Shouldn't you be using MSE, PSNR or SSIM? Why are some of scores negative?
If you are comparing how well you're interpolating for natural images or time series, you should compare with standard interpolation techniques in image processing and time series analysis.
The paper have a lot of missing details, including simple things like not linking to a Figure or Section the authors are referring to. Some examples are:
1. Authors motivate using Gibbs energy by saying that in auto encoders you have to project to a lower dimensional representation and gibbs energy based solution has no such dimensionality reduction constraint. There are no such constraint in AE as well - you can vey well have your hidden representation to be over complete - so this is not a good motivation for Gibbs energy
2. Main section of 4.1 says that they use MNIST but the caption of Table 1 says they use Fashion MNIST
3. The naming convention of the model is hard to follow and the authors notation itself is inconsistent.
Some other problems:
1. In Abstract, they say 2. images. Why 2?
2. Introduction, the citation for Gaussian priors is empty
3. Last word before beginning of section to, broken link
4. In the second para of 3.2 did you mean to say solve for (2) instead of (3)?
5.  What is "XXX" matrix in second para of 3.2?


The contributions of this paper are two-fold. We define a new filtration called the cover filtration built from a single cover based on a generalized Jaccard distance. We provide stability results for the cover filtration, and show its equivalence to thě Cech filtration under certain settings. We then develop a language and theory for stable paths within this filtration, inspired by ideas of persistent homology. This framework can be used to develop new learning representations in applications where an obvious metric may not be defined but a cover is readily available. We demonstrate the utility of our framework as applied to recommendation systems and explainable machine learning. We demonstrate a new perspective for modeling recommendation system data sets that does not require defining a bespoke metric. As an application, we find that stable paths identified by our framework in a movies data set represent sequences of movies constituting a gentle transition and ordering from one genre to another. For explainable machine learning, we present stable paths between subpopulations in the mapper of a model as explanations. Our framework provides an alternative way to build a filtration from a single mapper, and explore stable paths in it. For illustration, we build a mapper from a supervised machine learning model trained on the FashionMNIST data set. Stable paths in the cover filtration provide improved explanations of relationships between subpopulations of images.

1. Paper summary
This paper develops a novel filtration for the analysis of based on the
idea of *covers* of data sets. The filtration employs the notion of
a generalised Jaccard distance to define stable paths within the nerve
of the cover. These stable paths are then shown to be useful for the
creation of 'gentler' transitions for recommendation systems, as well
as the development of explainable supervised machine learning models.
2. Review summary
I found this paper to be very interesting in terms of its new
perspective on filtrations and its idea of the introducing a
'stability' concept inspired by topological persistence.
However, I recommend rejecting the paper in its current form due to the
following issues:
A. Missing clarity: the paper requires readers to be familiar with the
   Mapper algorithm and glosses over important details. Several results
   that are mentioned as core contributions in the main texts are only
   stated in the supplementary materials.
B. Missing experimental depth: the experiments shown in the paper are
   interesting, but only scratch the surface. To have a convincing case
   study, a more quantitative analysis is required. At present, the
   selection and depiction of results seems ad-hoc. I realise that,
   when the foremost goal is interpretability, providing a quantitative
   analysis is not always easily possible. Nevertheless, each data set
   should be used for more than one case study; ideally, multiple
   results that are 'surprising' can be reported. For example, are
   there paths that do not match our intuition? What do they tell us
   about the data set?
C. Doubts about the technical correctness: the supervised learning
   example requires an elaborate setup (dimensionality reduction,
   UMAP for further reductions, setting up Mapper with additional
   parameters). I am wondering how trustworthy any result obtained
   from such an analysis can be. I would suggest discussing the
   parameter selection process in more detail and ideally providing more
   information about the impact of these choices.
In the following, I will comment on these issues in more detail.
3. Clarity
- Concepts such as 'cover' should be briefly introduced; already the
  abstract presumes that readers are familiar with several TDA concepts
- I would shorten the abstract to improve its flow; the paragraphs 'We
  demonstrate...' and 'For explainable...' could also be added to the
  introduction to improve the exposition of the paper.
- The introduction already starts with TDA concepts; a brief motivation
  would make the paper more accessible
- I disagree with the statement that the existence of a metric on the
  data is an implicit assumption of TDA. My perspective is quite the
  opposite: TDA is rather flexible *because* of its support for
  different metrics; Vietoris--Rips complex calculation, for example,
  only requires a matrix of pairwise distances. Hence, I would rewrite this
  sentence.
- To add to the previous point: the paper itself introduces a new
  metric based on a generalised Jaccard distance. I feel that the
  discussion of 'ill-fit or incomplete' metrics detracts from the
  main message of the paper.
- The first explanation/definition of Mapper in the introduction is
  rather complicated ('nerve of a refined pullback cover'); I would
  suggest rephrasing this.
- The point about the $1$-skeleton warrants more explanation: it is my
  understanding that the paper uses paths that are defined using this
  skeleton as well. My first pass of the paper slightly confused me
  because I figured that the described paths would be generalised over
  high-dimensional simplices as well. I would suggest making this
  'restriction' (it is not a proper restriction because paths can be
  defined for arbitrary simplices, but it is a restriction in terms of
  the dimensionality) or property clear from the beginning.
- The discussion of hypercube covers is repeated multiple times in the
  introduction and the related work section; I would suggest mentioning
  this only once as it does not have to a large bearing on the methods
  described in the paper anyway.
- Discussing stability of the paths/cover in the introduction is
  misleading because these aspects are only discussed in the appendix.
  Same goes for some of the contributions in Section 1.1; the
  conjectures or connection to other filtrations are only mentioned in
  the appendix.
  This needs to be rewritten; I would pick a few contributions that the
  main text can focus on and mention the rest in passing.
  The paper lacks this structure at present and for a conference
  submission, all main contributions should also be a part of the main
  text.
- Stability is meant to reflect a property of a cover; this could be
  clarified by means of extending Figure 1, for example. During my
  first pass of the paper, some of the subsequent definitions were
  lacking a clear motivation. I would suggest to _clearly_ state that
  the goal is to circumvent issues with paths in a cover that only
  depend on a few data points.
- Moreover, it would be interesting to point out to what extent the
  method presented here is the _only_ one capable of doing this. It
  occurred to me that there's a preprint that discusses how to stabilise
  the calculations of persistent homology (with respect to picking the
  _same_ creator simplex for a feature, for example):
    Bendich, Paul et al.
    Stabilizing the unstable output of persistent homology computations
    https://arxiv.org/abs/1512.01700
  It seems that one of the unique features of the method in this paper
  is that the definition of path stability aligns exactly with the other
  stability concept---the paths are thus indeed stable with respect to
  perturbations of the underlying data set.
  Maybe this could be mentioned as an interesting feature.
- The paragraph 'In Section 4.1 [...]' in the related work section is
  somewhat redundant; I would suggest putting it at the beginning of the
  respective experimental section and merging it with the existing
  description there.
- The definition of the Steinhaus distance strikes me as unnecessary
  complex; while it is mathematically pleasing to know that arbitrary
  measures could be used, it seems that this is never exploited anywhere
  in the paper.
- In the interest of clarity, maybe it would make sense to refer to the
  distance as a 'generalised Jaccard distance' instead.
- The proof of Theorem 2.5 could be moved to the appendix. Moreover,
  I think it could be simplified by mentioning that the nerve is
  constructed from the cover and that weights are assigned based on
  the number of intersections; then the main result would follow from
  monotonicity. (the current proof is of course fully correct, I just
  had to rephrase this result in terms of concepts that I found easier
  to grasp)
- Concerning the terminology, I find 'a most stable path' to be somewhat
  confusing at the first read. I now understand what is meant by it, but
  maybe 'highly stable' or 'maximally stable' would be better.
- In Section 3, the 1-skeleton should be explicitly mentioned.
- Moreover, the notion of 'shortest path', which _could_ also employ
  distances, should be distinguished from stable paths; if I understand
  the argumentation correctly, shortest paths are defined in terms of
  the number of edges, while stable paths are defined in terms of the
  weights along those edges. I would suggest spelling this out directly
  to make the concepts non-ambiguous.
- In Definition 3.1, it should be $d_{St}$, not $D_{St}$, I think.
- Definition 3.1 could also be intuitively summarised as 'the largest
  edge weight along a path', right?
- The algorithm in Figure 2 should include some comments that make the
  procedure  more understandable. I found the conceptual leap to Pareto
  frontiers confusing at first; maybe this could be motivated better.
- Figure 3 needs more explanations. In particular the connection to
  persistence could be elucidated---I get that there's an immediate
  connection (see the paper above) but this concept needs at least
  a brief introduction. Moreover, if this connection is relevant, it
  deserves to be elucidated in the paper explicitly.
- Figure 4 needs additional labels; I expect that 'distance/stability'
  are shown in the caption. Is this correct?
  In general, I found the path visualisation not so helpful; maybe it
  would be better to show the full 1-skeleton (or an excerpt) and
  highlight the corresponding paths?
- The language and motivation in Section 4 is somewhat informal; this is
  not an issue for me, but I wanted to mention it as a something that
  could potentially be rephrased.
- The additional application domains should be moved from Section 4 to
  a discussion section.
- The cover generation strategy could be understood more rapidly if
  a small illustration was provided. Do I get it correctly that every
  *cover set* constitutes a film, while the overlap between them is
  based on whether the same user also rated another film?
- The path selection procedure for the experiment in Section 4.1 needs
  to be explained formally. I get that it is akin to selecting a point
  on the 'elbow' of a curve, but this should be briefly explained.
- Figure 5 needs to be referenced more prominently in the text.
- Section 4.2 is glossing over many important details of the definition
  of Mapper. These are absolutely required, though, and any parameters
  selected here should also be analysed to learn whether they affect the
  results.
- I am not sure about the utility of the paths in Figure 7
- Figure 9 needs more explanations; as mentioned below, why not compare
  this path with a geometry-based path in the embedding? Moreover, why
  are there not the same number of images for all classes?
- The conclusion needs to be rewritten; the paper does *not* show
  stability properties in the main text
4. Experiments
- As a general question, I would like to see how stable the
  interpretations of the different sections are. There is always
  a degree of stochasticity when training a neural network, so does
  the output of Mapper stay stable?
- I am not sure about the relevance of experiment in Section 4.1; it
  does not really provide a link to an explainable model for me, but
  instead is more along the lines of 'gentle interpolation in a latent
  space'. I could see that this has its uses, but an evaluation requires
  more than one example. I would be particularly interested in knowing
  about cases in which result is unexpected. Do such cases exist?
- Please comment on the stability of the sampling procedure in Section 4.1
- I understand that Section 4.2 wants to show how algorithms like Mapper can
  act as a middle ground between black-box and white-box models. Is this
  correct? If so, the chosen example is _not_ sufficiently complex to be
  compelling. Would it not be equally appropriate to rewrite this
  experiment in terms of explaining paths in the latent space of
  autoencoders? In this context, I see more of a need for these paths as
  an alternative to paths that purely employ latent space geometry.
- Adding to this, the autoencoder context would it also make possible to
  compare paths generated using Mapper and the proposed as well as the
  paths generated by the model. Such an experiment would be more
  compelling, I think.
- In general, what are the implications of understanding a path as in
  Figure 9? How could the model updated to account for this? If the
  paper were to include an example of how to *fix* a model based on such
  information, it would a highly compelling use case.
- Moreover, would it be possible to detect 'problematic' regions such as
  the one shown in Figure 9 automatically? If so, it would again provide
  a highly compelling example. Otherwise, I think that the
  interpretation of a model depends again on humans and are thus
  restricted based on model complexity.
5. Minor style issues
- Mapper should be capitalised consistently
- I suggest removing the paragraph 'Organization' as it is redundant
- 'mod differences' --> 'modulo differences' (?)
- 'distanceof' --> distance of'
- 'on undirected graph' --> 'on an undirected graph'
- 'Principle component analysis' --> 'Principal component analysis'


Learning overcomplete representations finds many applications in machine learning and data analytics. In the past decade, despite the empirical success of heuristic methods, theoretical understandings and explanations of these algorithms are still far from satisfactory. In this work, we provide new theoretical insights for several important representation learning problems: learning (i) sparsely used overcomplete dictionaries and (ii) convolutional dictionaries. We formulate these problems as ℓ 4 -norm optimization problems over the sphere, and study the geometric properties of their nonconvex optimization landscapes. For both problems, we show the nonconvex objectives have benign (global) geometric structures, which enable development of efficient optimization methods finding the target solutions. Finally, our theoretical results are justified by numerical simulations.

The authors consider two problems: Overcomplete dictionary learning (ODL)
and convolution dictionary learning (CDL).
Dictionary learning learns a matrix factorization of the data
Y = A X
where A is the dictionary and X is the (known to be sparse) code.
Y consists of n rows (sample size) and p columns (dimension).
In the overcomplete version A is n x m where m > n, i.e. the number of learned
features is larger than the sample size.
The CDL problem is a special case of the ODL problem where the dictionary
matrix is known to consist of convolution filters instead of being unstructured.
The authors show that under a given set of assumptions local nonconvex
optimization can be used to find globally relevant solutions.
The basic assumptions are:
(i) unit norm tight frame
(ii) mu-incoherence
	(relates the angles of the columns of a, e.g.\ if columns are orthogonal,
	they are incoherent / have small mu)
(ii) stochastic model of the code X that says entries are Gaussian and sparse
	according to a Bernoulli random variable
The authors present the idea of maximizing the l^4 norm of A^T q in order to
find q as rows of A.
Apparently l^4 norm maximization leads to "spikiness" which is exactly
desirable under mu-incoherence.
The authors show (assuming p \to \infty) that the optimization nonconvex
landscape (constrained to the sphere) does not contain any stationary points
without negative curvature.
A saddle avoiding optimizer therefore converges to local minimizers from
random initialization.
The authors also show that the analysis extends to CDL via a preconditioned
initializer.
Finally, they go on to briefly show some experiments that further validate
the theory presented in the paper.
Overall, the authors present a rigorous technical analysis using powerful
mathematical tools for nonconvex optimization (which is relevant to many
machine learning problems).
I am recommending to accept based on the high quality of the work.
But I am not confident as to the accessibility of the paper to the wide
audience of ICLR as it is rather technical.
Perhaps, the complete contribution would be better suited as a journal article.
Notes:
It would have been useful to give some more intuition about what "spikiness"
of A^T q is, why spikiness exists under mu-incoherence and why l^4 norm
maximization improves spikiness.
I am not sure that the inclusion of the CDL problem is beneficial for a
converence paper and would rather have more space allocated to the intuition on
why the method works for ODL.


We consider the task of answering complex multi-hop questions using a corpus as a virtual knowledge base (KB). In particular, we describe a neural module, DrKIT, that traverses textual data like a virtual KB, softly following paths of relations between mentions of entities in the corpus. At each step the operation uses a combination of sparse-matrix TFIDF indices and maximum inner product search (MIPS) on a special index of contextual representations. This module is differentiable, so the full system can be trained completely end-to-end using gradient based methods, starting from natural language inputs. We also describe a pretraining scheme for the index mention encoder by generating hard negative examples using existing knowledge bases. We show that DrKIT improves accuracy by 9 points on 3-hop questions in the MetaQA dataset, cutting the gap between text-based and KB-based state-of-the-art by 70%. DrKIT is also very efficient, processing upto 10x more queries per second than existing state-of-the-art QA systems.

The paper studies scaling multi-hop QA to large document collections, rather than working with small candidate lists of  document/paragraphs  (as done in most of the previous work), a very important, practical and challenging direction.
They start with linking mentions to entities in a knowledge base. Every iteration of their mult-hop system produces a set of entities Z_t, relying on entities predicted on the first representation Z_{t-1} and the question representation.   In order to make training tractable, they mask 'attention'  between Z_{t-1}  and Z_t (actually mentions corresponding to Z_t).  They also use top-K relevant mentions at train and test time. As the attention score is based on dot-product, they can plug-in the approximate Maximum Inner Product Search to avoid computing the attention score for every mention in the collection. The architecture is essentially end-to-end trainable (except for specialized pretraining discussed below).
Whereas itself the architecture is not overly novel (e.g., the architecture does feel a lot similar to models in KB context, and also graph convolution networks applied to QA), there is a lot of clever engineering and the novelty is really in showing that it can work without candidate preselection.
My main worry is pretraining. In both experiments (MetaQA and the new Wikidata Slot Filling task), they pretrain the encoders using a knowledge base, and the knowledge base directly corresponds to the QA task.   E.g., for MetaQA the questions are answerable using the knowledge bases, so relation types in the knowledge base presumably correspond to relations that need to be captured in the QA multihop learning.  This specialized pretraining appears to be crucial (88% for pretraining vs 55% with BER), presumably because of the top K pruning. Though a nice trick, it is likely limiting as a knowledge base needs to be available + it probably constraints the types of mult-hop questions the apporach can handle.  Also, some of the baselines do not benefit from using the KB, and, in principle, if it is used in training, why not use the KB at test time?  (I see though that for the second dataset pretraining seems to be done on a different part of Wikipedia, I guess, to address these concerns).
I was not sure how the number of hops T was selected for the model, it does not seem to be defined in the paper. Do you pretend that you know the true number of hops for each given question?
The authors experiment with reducing the size of a KB for pretraining. It apparently does not harm the first 1 hop questions, but 2 and 3-hop. Do the authors have any explanation for this?  Related to the previous question, does it mean that the model does not learn to exploit the hops for t > 1?
The evaluation is on MetaQA and on the newly introduced Wikidata task, whereas most (?) recent multi-hop QA work has focused on HotpotQA (and to certain degree WikiHop). Is the reason for not using (additionally) HotpotQA?  Is the model suitable for HotpotQA? If not, does this have to do with pretraining or the types of questions in HotpotQA?
The model definition in section 2.1 is not very easy to follow. E.g., it is not immediately clear if the model applied at every hop is the same model, and not clear how the model is made aware of the current search state (e.g., which part of the question has already processed / how the history is encoded) or even of the hop id.
I would really like to see more analysis of what the model learns at every hop.
--
After the rebuttal -- I appreciate the detailed feedback, extra experiments, and analysis. `I increased my score.


Temporal difference (TD) learning is a popular algorithm for policy evaluation in reinforcement learning, but the vanilla TD can substantially suffer from the inherent optimization variance. A variance reduced TD (VRTD) algorithm was proposed by Korda and La (2015), which applies the variance reduction technique directly to the online TD learning with Markovian samples. In this work, we first point out the technical errors in the analysis of VRTD in Korda and La (2015), and then provide a mathematically solid analysis of the non-asymptotic convergence of VRTD and its variance reduction performance. We show that VRTD is guaranteed to converge to a neighborhood of the fixed-point solution of TD at a linear convergence rate. Furthermore, the variance error (for both i.i.d. and Markovian sampling) and the bias error (for Markovian sampling) of VRTD are significantly reduced by the batch size of variance reduction in comparison to those of vanilla TD.

This paper presents a non-asymptotic analysis of Variance Reduced TD (VRTD), proposed by Korda and La (2015), to apply variance reduction ideas to temporal difference learning, specifically TD(0) with linear function approximation. The algorithm closely follows ideas of stochastic variance reduction (SVRG) which is widely used for large scale empirical risk minimization.
Specifically, the authors show VRTD converges (in expectation) to a neighborhood of the limit point using a constant step-size. Interestingly, this neighborhood can be made small using a large batch size M in both the IID and Markovian sampling regimes. From a technical standpoint, the work seems novel with interesting results although I am not sure if VRTD offers practical performance gains.
Main points:
1) I am a bit confused by the presence of a constant variance error term in the results. Intuitively, with variance reduction, I was expecting convergence in expectation; not just to a neighborhood with constant error. In other words, the variance error term decaying with $m$ (and not M). This would imply a stronger result than what authors have currently.
Note that vanilla SGD for strongly convex objectives also suffers from a constant variance error term (see for example Chapter 4 in Bottou et al., 2018). However, analysis of SVRG (Johnson and Zhang 2013) shows linear convergence for strongly convex objectives and claims sublinear convergence for convex objectives. While I appreciate the technical challenge in analyzing a \textit{semi-gradient} method like TD vis-a-vis  analyzing gradient descent for convex objectives, my understanding is that (Bhandari et al., 2018) showed a connection between the two. This connection makes me think if variance reduction can also help \textit{get rid} of the constant variance error term when analyzing TD(0) with constant step-sizes. Can the authors clarify?
Also, the authors might find it useful to look at (Lakshminarayanan and Szepesvari, 2018) which does in fact show convergence with a constant (problem instance independent) step-size for the IID case. I think that result only applies with iterate averaging but it might still be useful (certainly as a citation suggestion).
2) As a practical proposal, I wonder if VRTD has a potential benefit over vanilla TD(0). Essentially, the rates showed in this paper require $O(mM)$ samples. I wonder how vanilla TD would perform with $O(mM)$ samples coupled with a simple strategy of reducing the step-size by half when the value-function estimates stop changing. See Chapter 4 in (Bottou et al., 2018) for details. That would probably be a fairer comparison to help convince audience of VRTD as a practical alternative to TD(0).
Minor points:
- The counter example in Section 3.2 seems out of place. While it is important to point out the errors in previous analysis, I suggest the authors to flesh out the details in an Appendix section.
- $R_{\theta}$ and $r_{max}$ seem undefined in the main body of the paper.
- The constants in Theorem 1 and 2 seem complicated. Is there a way to simplify these for presentation purposes?
Response to author comments.
I thank the authors for their comments and changes to the draft.
1) The explanation makes a lot of sense. Since we cannot exactly estimate the population gradient, the variance error seems unavoidable with constant stepsize. I like the clarifications in Section 3.2
2) Thanks for clarifying that all the comparisons are done in terms of a fixed number of total gradient computations. Please add this clearly in the paper as well. I think the additional experiments and comparison with the new scheme I suggested above does a good job of convincing the readers of the practical usefulness of variance reduction.
Although I am not sure how likely practitioners are to use this (both $\alpha$ and M are problem dependent), in my opinion the paper presents a novel analysis of an important idea of variance reduction applied to Online TD. The experiments seem reasonable to suggest benefits. I’ll be happy to see this paper accepted. I have also updated my score.
Some minor comments and suggestions:
1) In Theorem 1, constant D_2 depends on $R_{\theta}$ but there Algorithm 1 has no projection step. I think the authors assume that $\norm{\theta^*} \leq R_{\theta}$. If so, please state that somewhere. If not, maybe I missed something and would like a clarification.
2) Page 6 should have $e(\tilde{theta}_m-1)$ instead of $e(\tilde{theta}_m)$.
3) Before stating the main results in Section 4, please restate the definition of $\lambda_A$ for better readability.
4) Constant G seems to be undefined for Theorem 2. In fact, it seems that $C_4 = G^2 [1 + \frac{2\rho \kappa}{1-\rho}]$. This is what I meant by simplifying constants. Ensuring that these constants are optimized for presentation helps.
5) Theorem 2 has a dependence on $d$. Maybe the authors should explicitly mention and discuss that.
6) The result for Markovian case requires the bias term to not dominate. This seems novel to me and would benefit from emphasis in the main body of the paper.
7) Equation 5 should have ’m’ instead of ’n’.
8) I always find large figures and clear (and) large axis labels very useful. Can the ‘iteration process’ graphs in Figure 1 be made more reader friendly? I’d  like to see the behavior of vanilla TD more clearly.
9) Section A in the Appendix might benefit from more background. For example, what is $v$. I am thinking this section to be more self contained with the readers not being compelled to read the proof in Korda and La (2015).


sive disease that affects over 5 million patients in the US alone, and is the 6th leading cause of death. Early detection of AD can significantly improve or extend a patient's life so it is critical to use all available information about patients. We propose an unsupervised method to learn a consistent representation by utilizing inconsistent data through minimizing the ratio of p-Order Principal Components Analysis (PCA) and Locality Preserving Projections (LPP). Our method's representation can outperform the use of consistent data alone and does not require the use of complex tensor-specific approaches. We run experiments on patient data from the Alzheimer's Disease Neuroimaging Initiative (ADNI), which consists of inconsistent data, to predict patients' diagnosis. Our results show that we can utilize additional data about patients, even when they are inconsistent, to improve their diagnosis accuracy thereby improving or lengthening their lifetime. We also identify key biomarkers predictive of AD consistent with clinical research.

This manuscript develops a metric-learning with non-Euclidean error terms for robustness and applies in to data reduction to learn diagnostic models of Alzheimer's disease from brain images. The manuscript discusses a metric-learning formulation as a quotient for reconstruction-error terms, how to optimize the quotient based on results from Wang et 2014, an iterated reweighted approach to circumvent the non-smooth part of the l1 loss in zero, and experiments on brain images of Alzeimer's disease.
My two main issues with the manuscript is that the theoretical part is written very imprecisely and that the experiments are not convincing due to the lack of good baselines and of statistical power.
With regards to the theoretical contributions, a fraction of the results in the present manuscript are trivial consequences or Wang2014, and yet it comes with errors in the statements. For instance, in equation (9), the present manuscript writes greater or equal, while I believe that it should be strictly greater, as in Wang. Theorem 1 and 4 seem almost the same thing, though with a contradiction between the two. Other statements are inaccurate: the authors claim some results on reaching global optima, while I believe that they can only claim that they reach stationary points. Theorem 2 and 5 seem to be the same thing.
Concerning the iterated reweighted approach, I believe that this is non smooth only for g(x)=0, which is not covered by the theorems of Wang 2014. Is this algorithm needed? Note that Wang apply their algorithm with an l1 norm, ie non-smooth in zero, and do not report problems with out. The manuscript mentions that "to inverted matrices that divide 0s, which routinely lead to inferior learning performance.". I am not exactly sure what that means and I would need to understand better the problem. Also, the theoretical contribution that with the added the delta the algortihm converges, seems quite trivial: it seems to me that it is the eta trick.
Minor comments: in algorithm 3, it would be useful to write the full expression of the equations, rather than just reference the numbers. Also, the computational cost of the eigenvectors at each iteration seems quite prohibitive.
How was the value r=3 selected?
Figure 2 seem to choose that approaches have not converged: they final value is larger than intermediate values?
How were the p-values between cross-validation assessment of estimators computed? If it was done using standard paired t-test, this is incorrect are the folds are not independent.
With regards to the experiments, I worry that the model is not compared against simple baselines, such as a PCA.


We propose vq-wav2vec to learn discrete representations of audio segments through a wav2vec-style self-supervised context prediction task. The algorithm uses either a Gumbel-Softmax or online k-means clustering to quantize the dense representations. Discretization enables the direct application of algorithms from the NLP community which require discrete inputs. Experiments show that BERT pre-training achieves a new state of the art on TIMIT phoneme classification and WSJ speech recognition.

Overview:
This paper considers unsupervised (or self-supervised) discrete representation learning of speech using a combination of a recent vector quantized neural network discritization method and future time step prediction. Discrete representations are fine-tuned by using these as input to a BERT model; the resulting representations are then used instead of conventional speech features as the input to speech recognition models. New state-of-the-art results are achieved on two datasets.
Strengths:
The core strength of this paper is in the results that are achieved on standard speech recognition benchmarks. The results indicate that, while discritization in itself does not give improvements, coupling this with the BERT-objective results in speech features which are better in downstream speech recognition than standard features. I think the main technical novelty is in combining discritization with future time step prediction (but see the weakness below).
Weaknesses:
The main weakness of the paper is that it does not situate itself within existing literature in this area. Over the last few years, researchers in the speech community have invested significant effort in learning better speech representations, and this is not discussed. See e.g. [1]. Even more importantly, very recently there has been a number of papers investigating discrete representations of speech; see the review [2]. Some of these papers specifically use VQ-VAEs [3]. [4] actually compares VQ-VAE and the Gumbel-Softmax approach. These studies should be mentioned. This paper is different in that it incorporates future time step prediction. But context prediction has also been considered before, also for speech [5, 6, 7]. This paper can be situated as a new contribution combining these two strands of research. In the longer run it would be extremely beneficial to the community if this approach is applied to the standard benchmarks as set out in [2].
As a minor weakness, some parts of the paper is not described in enough detail and the motivation is weak or not exactly clear (see detailed comments below).
Overall assessment:
I think the results as well as the new combination of existing approaches in the paper warrants publication. But it should be amended significantly to situate itself within the existing literature. I therefore award a "weak accept".
Detailed questions and suggestions:
- Section 1: As motivation for this work, it is stated that "we aim to make well performing NLP algorithms more widely applicable". As noted above, some NLP-like ideas (such as prediction of future speech segments, stemming from text-based language modelling) have already been considered within the speech community. Rather than motivating the work in this way, it might be helpful to focus the contribution as a combination of future time step prediction and discretization (both of which have been considered in previous work, but not in combination).
- Section 4: Would it be possible to train the vq-wav2vec model jointly with BERT, i.e. as one model? I suspect it would be difficult since, for the masking objective, the discrete units are already required, but maybe there is a scheme where this could work.
- Section 2.2: Similarly to the above question, would there be a way to incorporate the BERT principles directly into an end-to-end model, e.g. by randomly masking some of the continuous input speech?
- Section 3.3: What exactly does "mode collapse" refer to in this context? Would this be using only one codebook entry, for instance?
- Section 6: It seems that in all cases to obtain improvements from discritization, BERT is required on top of the vq-wav2vec discrete symbols. Is it possible that the output acoustic model is simply better-matched to continuous rather than discrete input (direct vq-wav2vec gives discrete while BERT gives continuous)? Would it make sense to train the wav2vec acoustic model on top of the vqvae codebook entries (e) instead of directly on the symbols?
Typos, grammar and style:
- "gumbel" -> "Gumbel" (throughout; or just be consistent in capitalization)
- "which can be mitigated my workarounds" -> "which can be mitigated *by* workarounds"
- "work around" -> "workaround"
Missing references:
1. Versteegh, M., Anguera, X., Jansen, A. & Dupoux, E. (2016). The Zero Resource Speech Challenge 2015: Proposed Approaches and Results. In SLTU-2016 Procedia Computer Science, 81, (pp 67-72).
2. https://arxiv.org/abs/1904.11469
3. https://arxiv.org/abs/1905.11449
4. https://arxiv.org/abs/1904.07556
5. https://arxiv.org/abs/1904.03240
6. https://arxiv.org/abs/1807.03748 (this paper is cited)
7. https://arxiv.org/abs/1803.08976
Edit: Based on the feedback from the authors, I changed my rating from a 'weak accept' to an 'accept'.


Recent work has described neural-network-based agents that are trained to execute language-like commands in simulated worlds, as a step towards an intelligent agent or robot that can be instructed by human users. However, the instructions that such agents are trained to follow are typically generated from templates (by an environment simulator), and do not reflect the varied or ambiguous expressions used by real people. We address this issue by integrating language encoders that are pretrained on large text corpora into a situated, instruction-following agent. In a procedurally-randomized first-person 3D world, we first train agents to follow synthetic instructions requiring the identification, manipulation and relative positioning of visually-realistic object models. We then show how these abilities can transfer to a context where humans provide instructions in natural language, but only when agents are endowed with language encoding components that were pretrained on text-data. We explore techniques for integrating text-trained and environment-trained components into an agent, observing clear advantages for the fully-contextual phrase representations computed by the well-known BERT model, and additional gains by integrating a self-attention operation optimized to adapt BERT's representations for the agent's tasks and environment. These results bridge the gap between two successful strands of recent AI research: agent-centric behavior optimization and text-based representation learning.

The authors present a method of transferring template-based instruction following agents to natural language instructions by using language encoders trained on large text corpora. They explore different ways of combining text-based language encoders with visual representations and compare them. They find that contextual phrase-based representations learned by BERT significantly improve the performance on natural language instructions.
Strengths:
- The paper is written well, it is easy to understand and follow.
- The task setup is good, authors collect natural language instruction data from humans.
- The paper presents several language encoding methods for the task and systematically evaluates them in a scientific manner.
- The experimental results indicate that it is possible to transfer an agent trained on template-based instructions to natural language instructions using language models trained on large text corpora. It is not necessary to train the agent on natural language instructions. I find this result important and useful.
Weaknesses:
- The paper lacks significant technical novelty. It essentially combines known reinforcement learning based instruction following agents with known language models. The different ways of combining language encoding with visual representations are either trivial or adapted from prior work.
- A major concern is that the natural language instructions considered in the paper do not have much diversity with respect to language. The paper only considers lifting and putting tasks and trains a separate model for both the tasks.
-- The lifting task always uses the verb 'lift' and replaces the object word with synonyms or referring expressions. There are 80 objects in the lifting task and I suspect there are very few referring expressions for these objects and they mostly involve a synonym. Furthermore, at test time, the agent only needs to distinguish between 2 objects. The performance with random embedding is around 50% for this task and the best model is around 76% which means the agent is not recognizing the correct object around 50% of the time.
-- For the putting task, authors consider synonyms for object words and natural instructions which involve changing the verb ‘put’. It seems like humans mostly use only 4 verb words for this task, ‘put’, ‘keep’, ‘move’, ‘place’. This might be an artifact of the examples given to the human annotators. In any case, this word is inconsequential as the agent always lifts one of 3 available objects on one of 2 fixed objects.
- It seems like the most diversity is coming from synonyms which can probably be handled with a dictionary or wordnet rather than requiring a language model. There is also some prior work on handling synonyms (https://arxiv.org/pdf/1902.04546.pdf). I would have liked to see many more tasks and a multi-task learning model which is also able to distinguish between the task based on natural language instructions in addition to understanding object word synonyms and referring expressions. More objects would also help.
- The authors claim to tackle “more behavioural and environmental challenges than previous work”. I do not agree with this claim. It is true that this paper handles object interaction and natural language instructions in a partially observable setting, however, previous work has tackled other challenges which this work does not tackle. For example, Oh et al. 2017 generalize to new sequence of instructions, Hermann et al. 2017 and Chaplot et al. 2018 also handle compositionality and generalize to unseen instructions referring to new objects, Hermann et al. 2017 handle negation, Chaplot et al. 2018 handle instructions involving ‘largest’ or ‘smallest’ objects, Misra et al. 2018 handle more diverse natural language and so on.
- I wouldn’t call moving objects using high-level symbolic actions as ‘manipulation’. This is a whole research area in robotics involving taking low-level actions to move an object. Also, the environment used in the paper is not visually realistic in my opinion. It looks game-like and visual encoders trained in this environment are unlikely to generalize to the real world. This is fine as it is mostly irrelevant to handling natural language instructions, but authors should not claim visual realism and object manipulation in my opinion.
Comments/Questions
- I do not understand the meaning and purpose of some actions. Why are there GRAB + actions? Doesn’t the object move with the agent once it is grabbed? What is SPIN_OBJECT? Why is it needed? It seems like there is no ‘place’ action, how does the agent place the object? I am guessing when the agent stops taking GRAB+ actions. If that is the case, then wouldn’t it be easier to just have Grab and Place actions rather than 16 GRAB+ actions?
- The meaning of ‘(sub)-’ in (sub)-word is not described.
- What is the probability of typo noise in the experiments?
- Many experimental details are missing. How long was the model trained for both the tasks? How many training samples/episodes? What were the hyperparameters used for reinforcement learning? Learning rate, optimizer, discount value and so on.
Updates after author response:
I have examined the author response and additional experiments carefully. I am maintaining my score due to the following reasons:
- The authors seem to agree that the paper does not provide any 'substantial algorithmic advance'.
-  The authors argue that the number of objects in the paper is much more than prior work. However, the focus of prior work (referenced by authors) was not to tackle natural language. Since the focus of this paper is to tackle natural language, I believe the number and diversity of objects and tasks need to be much higher than 80 objects and 2 tasks used in the paper.
- "We would language referring to an even wider range of motor-behaviours, e.g. more 'verbs'" -> I do not understand what the authors are trying to say here, but if "learning the motor programmes for such concepts in an environment" is not "the focus of the present paper", I believe the focus is only handling synonyms and referring expressions for objects. In my opinion, these are relatively easier to tackle, (for example using a dictionary or wordnet) than grounding 'verbs' into sequence of actions, which limits the scope of this paper further.
- The authors still claim to tackle object manipulation and visual realism which I do not agree with. I do not believe taking high-level "GRAB" action can be called object manipulation. The objects are taken from shape net with relatively realistic shapes, but neither the appearance of objects (textures, shadows) nor the relative arrangement of objects is realistic. In my opinion, a model trained in this environment has no hope of generalizing to the real-world.
- I do not agree with the authors' argument against chance performance in the lifting task. From my experience, I believe an RL agent trained without any language input would perform at 50% if it receives a reward for lifting one of the two available objects. I do not understand why the authors chose to put only 2 objects in the environment. Why not put 5 or more objects?
- I appreciate authors' efforts towards multi-task learning results, however, tackling only 2 tasks is not convincing enough.


Good methods of performing anomaly detection on high-dimensional data sets are needed, since algorithms which are trained on data are only expected to perform well on data that is similar to the training data. There are theoretical results on the ability to detect if a population of data is likely to come from a known base distribution, which is known as the goodness-of-fit problem, but those results require knowing a model of the base distribution. The ability to correctly reject anomalous data hinges on the accuracy of the model of the base distribution. For high dimensional data, learning an accurate-enough model of the base distribution such that anomaly detection works reliably is very challenging, as many researchers have noted in recent years. Existing methods for the goodness-of-fit problem do not account for the fact that a model of the base distribution is learned. To address that gap, we offer a theoretically motivated approach to account for the density learning procedure. In particular, we propose training an ensemble of density models, considering data to be anomalous if the data is anomalous with respect to any member of the ensemble. We provide a theoretical justification for this approach, proving first that a test on typicality is a valid approach to the goodness-of-fit problem, and then proving that for a correctly constructed ensemble of models, the intersection of typical sets of the models lies in the interior of the typical set of the base distribution. We present our method in the context of an example on synthetic data in which the effects we consider can easily be seen.

Summary:
I machine learning, we often have training data representative of an underlying distribution, and we want to test whether additional data come from the same distribution as the training data (e.g. for outlier/anomaly detection, or model checking). One way to do this is to learn a model of the underlying distribution, and test whether the additional data fall within the typical set of the model. This paper points out that the typical set of the model may be very different from the typical set of the underlying distribution if the model is learned by maximum likelihood, in which case a test of typicality with respect to the model would be a poor test of typicality with respect to the underlying distribution. The paper shows theoretically that the intersection of the typical sets of an ensemble of models lies within the typical set of the underlying distribution, provided that (a) each model is a good enough approximation to the underlying distribution, and (b) the models are all sufficiently different from each other. Based on that, the paper argues that a better test of typicality would be to test whether the additional data fall within the intersection of the typical sets of the ensemble of models.
Pros:
The paper addresses an interesting problem in a sound and well motivated way. There is a lot of work on outlier/anomaly detection that uses the model's probability density to determine whether a dataset is out-of-distribution or not, which is known to not be a good proxy for typicality, because atypical data can have high probability density. In contrast, this paper uses a well-founded notion of typicality based on the information-theoretic definition of a typical set.
The toy example that is used to illustrate the problem is clear and illuminating, and motivates the paper well. In particular, the example clearly illustrates the issue of local minima when training models, and the mass-covering behaviour of maximum-likelihood training.
The idea of using the intersection of the typical sets of an ensemble of models is interesting and clever, and backed by strong theoretical results.
Cons:
Even though I appreciate the paper's theoretical contribution, there are no empirical results other than the motivating example. In particular, the paper proposes an idea and theory to back it up, but it doesn't really propose a practical method, and as a result it doesn't test the theory in practice.
Theorems 2 and 3 provide a solid foundation for the proposed idea, but it's not clear how they can be used in practice. Specifically:
- How can we verify that in practice the KL between the models and the underlying distribution is small enough as required by theorem 2 when we can't usually evaluate it?
- In practice, how should we construct an ensemble such that the individual models in the ensemble are different enough from each other as required by theorem 3?
- Both theorem 2 and 3 are valid "for large enough n". However, in practice we may want to check e.g. individual datapoints for typicality (in which case n=1). Are the theorems relevant for small n?
The paper is generally well written, but some statements made are either inaccurate or subjective, and I worry that they might mislead readers. Later in my review I will point out exactly which statements I'm referring to. I strongly encourage the authors to fix or moderate these statements before the paper is published.
Decision:
I believe the paper to be an important contribution, but the work is clearly incomplete. For this reason, my recommendation is weak accept, with an encouragement to the authors to continue the good work.
Inaccuracies or subjective statements that I encourage the authors to fix/moderate:
"we are still bad at reliably predicting when those models will fail"
"we are unable to detect when the models are presented with out-of-distribution data"
These statements may come across as too strong. I suggest making the statements about our current methods, rather than about the ability of the research community, and be more specific in what ways the current methods are inadequate.
"detecting out-of-distribution data [...] is formally known as the goodness-of-fit problem"
I'm not sure that detecting our-of-distribution data and goodness-of-fit are synonymous. Goodness-of-fit testing can be used in situations other than outlier detection, e.g. for testing whether a proposed model is a good fit to a dataset.
(Second bullet-point of section 1) "distributions having low KL divergence must have non-zero intersection"
To be more precise, the typical sets must have non-zero intersection, not the distributions.
"determining which of two hypotheses are more probable"
"H0 is deemed more probable"
Classical hypothesis testing does not assign a probability to a hypothesis, which would be a Bayesian approach instead. Therefore, it's technically incorrect to talk about the probability of a hypothesis in this context.
"which accepts the null-hypothesis"
"f correctly accepting H0"
Hypothesis testing doesn't accept a hypothesis, it merely decides whether to reject the null hypothesis in favour of the alternative hypothesis. Therefore, it may "fail to reject" the null hypothesis, but it never accepts it.
"the KL-divergence is equal to zero if and only if p(x) = q(x; θ) ∀x ∈ X"
The KL is equal to zero if and only if the distributions are equal, but the densities may still differ in at most a set of measure zero. Therefore, it's not a requirement that the densities match for all x for the KL to be zero.
"For example, by looking at the form of the KL-divergence, there is no direct penalty for q(x; θ) in assigning a high density to points far away from any of the ¯xi’s"
The problem that this statement is talking about is the problem of overfitting, which is the problem of the model learning the specifics of the training data rather than the underlying distribution. However, the statement preceding the above is about the problem of local minima when optimizing then parameters of a model. These two problems are distinct and shouldn't be conflated, as they are here.
"this requires direct knowledge of p(x) to evaluate the objective"
However we can evaluate the objective up to an additive constant when p(x) is known up to a multiplicative constant, which is enough to optimize it.
"as do all divergences other than the forward KL, to the best of our knowledge"
"This makes the forward KL-divergence special in that it is the only divergence which can directly be optimized for."
I don't think this is true. For example, the Maximum Mean Discrepancy is a divergence, since it's non-negative and zero if and only if the two distributions are equal, but it only involves expectations under p(x) and can be directly optimized over the parameters of q(x; \theta). Moreover, the second statement doesn't follow from the first: it's incorrect to conclude that the forward KL is the only one that can be directly optimized for, based only on one's state of knowledge.
"Variational Auto-encoders [...] map a lower-dimensional, latent random variable"
There is no fundamental reason why the latent variable of a VAE has to be low-dimensional. We may do this often in practice, but a VAE with a high-dimensional latent variable may also be used.
"Because the image of any non-surjective function necessarily has measure zero"
This is not true; the absolute-value function is not surjective but its image doesn't have measure zero in the set of real numbers. I understand what the statement is trying to say, but it's important that it's said accurately.
"autoregressive models, such as PixelCNN"
Autoregressive models can also be used to model discrete variables in which case they can't be thought of as flows. In fact, PixelCNN as first proposed is a model of discrete variables.
"all of these models rely on optimizing the forward KL-divergence in order to learn their parameters"
Not necessarily, flow-based models don't have to be optimized by minimizing the forward KL. For example, they can be trained adversarially in the same way as GANs, and in principle can be trained with other divergences or integral probability metrics. The model and the loss are (at least in principle) orthogonal choices.
"advancements in the expressivity of the models are unlikely to fix the undesired effects"
This is a subjective assessment, and is not sufficiently backed by arguments where it first appears. I understand that the arguments are presented later in section 3, so I would at least suggest that a forward reference to the argumentation in section 3 is given here.
Figure 5 gives the impression that the model samples have less variance than the ground-truth samples. Isn't that surprising given that the problem is that minimizing the forward KL leads to mass-covering behaviour? I suspect that the problem here is that there are more ground-truth samples than model samples, and the ground-truth samples saturate the scatter plot. If that's the case, I believe that figure 5 is very misleading.
"we see that the learning procedure converged"
We know however that the learning procedure hasn't really converged, instead it is stuck at a saddle point (where the model is using a single mode to cover two modes of the underlying distribution). In other words, it appears to us that the learning procedure has converged, even though it hasn't, and possibly if we wait for long enough we will see rapid improvement when the procedure escapes the saddle point. Therefore, I would at least say "we see that the learning procedure has appeared to converge".
I would expect the bottom-right entry of table 1 to be higher than 90% like the other diagonal elements, so I suspect that it might be a typo.
In eq. (7), shouldn't each log q_k be divided by n?
"in practice we find that it is much easier to find an ensemble of models such that the multi-typical set approximates the ground-truth typical set than the bounds require"
There is no empirical evidence presented in the paper in support of this statement.
"least probable density"
"least typical density"
I understand what the intended meaning of these terms is, but these terms make little sense mathematically nevertheless. I would suggest that the statement is rewritten in a more precise and direct way.
"This measure only corresponds to measuring typicality if the bijection is volume preserving"
I'm not sure that the distance from a Gaussian mean is a valid measure of typicality. In high dimensions, the region around the mean is very atypical.
Minor errors, typos, and suggestions for improvement:
The phrase "the authors in Smith et al. (2019) propose" is a bit awkward. Better say "Smith et al. (2019) propose", as Smith et al are indeed the authors.
Missing full stop in first bullet-point of section 1.
It would be good to provide more details of the experiment in section 3. Specifically:
- What training algorithm was used to maximize the likelihood? SGD or EM?
- How many training datapoints were used?
"to index the 5 experiments ran" --> run
"refer the k-th learned density" --> refer to
interestig --> interesting
Missing closing bracket in point 1 of section 4.
Capital C in "Consider" in theorem 2.
"if every model in a density of learned distributions" --> an ensemble of learned distributions
"where as the method we propose" --> whereas
"can be found in in", double "in"