Adaptive optimal training of animal behavior
Neuroscience experiments often require training animals to perform tasks designed to elicit various sensory, cognitive, and motor behaviors. Training typically involves a series of gradual adjustments of stimulus conditions and rewards in order to bring about learning. However, training protocols are usually hand-designed, relying on a combination of intuition, guesswork, and trial-and-error, and often require weeks or months to achieve a desired level of task performance. Here we combine ideas from reinforcement learning and adaptive optimal experimental design to formulate methods for adaptive optimal training of animal behavior. Our work addresses two intriguing problems at once: first, it seeks to infer the learning rules underlying an animal's behavioral changes during training; second, it seeks to exploit these rules to select stimuli that will maximize the rate of learning toward a desired objective. We develop and test these methods using data collected from rats during training on a two-interval sensory discrimination task. We show that we can accurately infer the parameters of a policy-gradient-based learning algorithm that describes how the animal's internal model of the task evolves over the course of training. We then formulate a theory for optimal training, which involves selecting sequences of stimuli that will drive the animal's internal policy toward a desired location in the parameter space. Simulations show that our method can in theory provide a substantial speedup over standard training methods. We feel these results will hold considerable theoretical and practical implications both for researchers in reinforcement learning and for experimentalists seeking to train animals.

The authors have tried to accomplish two goals in this paper. First, they develop a technique to dynamically track the (time-varying) psychometric function of an animal while it learns a new task. Then they propose an algorithm to choose an optimal stimulus sequence that would most quickly drive the animal towards a desired (target) psychometric function. For simplicity, the focus of the paper is restricted to binary choice tasks in which the psychometric model is assumed to be described by a logistic function. The model is somewhat enriched by incorporating additional parameters (“stickiness coefficients”) to allow for possible history-dependence in choice behavior typically observed in the laboratory. The bulk of the results are presented in sections 3-5.  In section 3, the authors develop a general framework to estimate time-varying parameters of the logistic model when the changes in parameter values are purely stochastic (follow a gaussian random walk). This is achieved by first using an empirical Bayes approach to determine the hyperparameter that characterises the variability of the random walk, followed by standard MAP estimation of the model parameters for the given dataset. The approach is validated through simulations and applied to data collected from training rodents on an auditory task.  This exercise is repeated in section 4, but now after adding a non-random component to the temporal dynamics of the model parameters to mimic the effect of learning. This deterministic component is assumed to push the model parameters along the gradient of the expected reward with a learning rate which enters the model as a second hyperparameter. They call this the “RewardMax” model of learning. This section ends with a comparison of the two hyperpameters fit to behavioural data from rodents, and the authors make a point about how these results imply that the rodents are indeed behaving in a regime where the effect of learning trumps that of random noise. In section 5, the authors present an algorithm to select a stimulus sequence that would drive the psychometric function of the trainee towards a desired shape as quickly as possible. This is achieved using an “AlignMax” algorithm in which an upcoming stimulus is chosen as the one that maximally aligns the expected change in model parameters along the direction of the target model parameters. A comparison of the evolution of model-parameters using optimal and random stimulus sequences revealed that the former is indeed faster albeit only marginally. The more remarkable observation (although strangely this is not emphasised) was the fact that employing optimal stimulus sequence completely eliminated bias and history-dependence whereas random stimuli failed to achieve this even after training for several thousand trials. The authors then highlight key features of the resulting optimal stimulus sequence, and explain how their algorithm exploits a large stimulus space to optimise training.
This is an innovative study that establishes a nice model-based method to optimize animal learning of a task. It addresses a simple task and simple model but this is challenging enough and a good foundation from which to build. The authors don’t yet applying this method to real experiments, but I’m sure that’s coming soon, and it’s not needed here to demonstrate their technical advance. - There should be some discussion of the impact of the restriction to a memory-less model of reward (tau = 0). - Figure 4 – define success rate. Better explain why success is so low (because you’re giving essentially adversarial examples?). Why is expected reward worse than for the random stimulus presentation schedule? - Figure 4 – The performance measures are interesting, but the how well does the model learn the desired weights (which is the expressed goal of the model)? - L is likelihood of data or likelihood of weights? - Figure 2 - dw of different realisations are not identical but there is a strong correlation. - Define “lazy fluke”. - It would be appropriate to address/cite Christian Machens’ work on optimal stimulus ensembles (http://www.ncbi.nlm.nih.gov/pubmed/16055067) and, especially, Liam Paninski’s work on optimizing stimuli to maximize information about an underlying model (http://www.stat.columbia.edu/~liam/research/pubs/pbr.pdf).


Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation
Segmentation of 3D images is a fundamental problem in biomedical image analysis. Deep learning (DL) approaches have achieved state-of-the-art segmentation performance. To exploit the 3D contexts using neural networks, known DL segmentation methods, including 3D convolution, 2D convolution on planes orthogonal to 2D image slices, and LSTM in multiple directions, all suffer incompatibility with the highly anisotropic dimensions in common 3D biomedical images. In this paper, we propose a new DL framework for 3D image segmentation, based on a combination of a fully convolutional network (FCN) and a recurrent neural network (RNN), which are responsible for exploiting the intra-slice and inter-slice contexts, respectively. To our best knowledge, this is the first DL framework for 3D image segmentation that explicitly leverages 3D image anisotropism. Evaluating using a dataset from the ISBI Neuronal Structure Segmentation Challenge and in-house image stacks for 3D fungus segmentation, our approach achieves promising results comparing to the known DL-based 3D segmentation approaches.

This paper proposes a new model for biomedical image segmentation, which extends existing works in two distinct way: First, it generalizes the well-known U-Net architecture to a multi-scale setting (called kU-net), while remaining fundamentally within a 2D fully-convolutional scheme. Second, it combines several such kU-Net layers into a bi-directional convolutional LSTM (called BDC-LSTM), which can account for context given by the third dimension, while elegantly handling the common anisotropy of biomedical stacks (wherein the z-dimension is sampled at a lower spatial resolution than the xy-plane).
The paper is generally well written and easy to understand.  I quite like the proposed model: kU-net provides an answer to the ability to capture multi-scale features within a medical image, and the bi-directional LSTM scheme is an elegant way to account for broader context from the z-dimention.  However, I offer a few reservations to the paper as it currently stands.  Standard ways of dealing with anisotropy include resampling (e.g. trilinear) to isotropic voxels, or of using non-cubic kernels in fully convolutional 3D models [1,2]. For datasets in which the across-plane resolution is reasonably close to the within-plane one (e.g. fungus), this standard preprocessing would have been expected as a baseline. For models in which the across-plane resolution is much less than the within-plane one (e.g. Neuron), one can expect the impact of across-plane z-context to be small compared to the within-plane influence. It may be for this reason that the performance improvements of using RNNs for this dataset are so small. This might warrant a more extended discussion.  Much case is made of the fact that the model can handle anisotropic images; however, the model as presented can only deal with training and testing images that all exhibit the same sampling resolution. The real problem with anisotropy is that datasets are acquired on heterogenous equipment (e.g. scanners from different manufacturers) such that all images are collected at different resolution. As noted, current practice is to carry out a crude but effective resampling to a common resolution; however, it would be really nice to be able to have the model directly handle this heterogeneity.  The model’s multi-scale ability arise from a single downsampling (by a factor of 2) in the kU-net layer, with the concatenation of features at the two levels. Much simpler yet effective mechanisms have been proposed (e.g. [3]), and it would have been nice to understand whether they could serve the same purpose within the studied context.  The experimental results appear fairly limited. Although the proposed models seem to show improvements to commonly-reported metrics, the evaluation is limited to two very small datasets, and give no hint at how the model would perform on more general tasks (e.g. on anatomical data such as BRATS).  Moreover, the improvements appear relatively small (especially for Neuron dataset) given the enormous complexity of the model compared to baseline U-Net (both in terms of architecture and number of parameters). This at least warrants a discussion of whether the additional effort is really worth it.  Detailed Comments: - Line 85: this arrangement is also similar to [3] - Line 164-165: the argument that BDC-LSTM is more efficient at exploiting inter-splice contexts than Pyramid-LSTM needs to be better explained and substantiated. - Line 186: define deconvolution: is it simple upsampling or a real deconvolution? - Lines 211-212: it is not clear from the paper if decoupled training is indeed used in the experiments. What are the implications for model initialization? Line 330: lstm ==> LSTM Line 332: lstm ==> LSTM  References  [1] Dou, Q., Chen, H., Yu, L., Zhao, L., Qin, J., Wang, D., ... & Heng, P. A. (2016). Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks. IEEE transactions on medical imaging,35(5), 1182-1195.  [2] A. A. A. Setio et al. (2016). Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks. in IEEE Transactions on Medical Imaging, 35(5), pp. 1160-1169.  [3] Shen, W., Zhou, M., Yang, F., Yang, C., & Tian, J. (2015, June). Multi-scale convolutional neural networks for lung nodule classification. In Information Processing in Medical Imaging (pp. 588-599). Springer International Publishing.


Showing versus Doing: Teaching by Demonstration
People often learn from others' demonstrations, and inverse reinforcement learning (IRL) techniques have realized this capacity in machines. In contrast, teaching by demonstration has been less well studied computationally. Here, we develop a Bayesian model for teaching by demonstration. Stark differences arise when demonstrators are intentionally teaching (i.e. showing) a task versus simply performing (i.e. doing) a task. In two experiments, we show that human participants modify their teaching behavior consistent with the predictions of our model. Further, we show that even standard IRL algorithms benefit when learning from showing versus doing.

This paper addresses the subject of teaching by demonstration. In particular, it asks the following question: When teaching from demonstration, is there a better way to form trajectories than following the optimal policy?  In addressing this question, the authors combine two earlier strands of research in the literature, inverse reinforcement learning and Bayesian pedagogy, to introduce a new model called pedagogical inverse reinforcement learning.  They perform two experiments with human subjects on two simple, deterministic gridworld problems, using a tabular and a feature-based reward function. The authors observe that people differ in the trajectories they produce when performing the task for themselves (to maximize return) and when demonstrating the task to others (so that they can maximize return as well).
The subject of this paper (teaching by demonstration) is important and relevant for NIPS.   The primary questions that the authors ask are the following:  -- How might showing how to do a task differ from merely doing it?   -- How can inverse RL algorithms benefit from intentional teaching?   These questions are novel and this line of research can potentially have high impact.  The model introduced by the authors is a straightforward combination of two pieces of earlier work on inverse reinforcement learning and on Bayesian pedagogy. From this model, the authors make certain predictions, then conduct two experiments with human subjects to evaluate whether these predictions are observed in behavioral data.   In the paragraph that starts on line 124, the authors state that the pedagogical model makes two types of qualitative predictions. The first prediction is that certain trajectories show "clearer" paths than others. The second prediction is that certain "looping" behaviors will be observed. I have a number of concerns regarding these qualitative predictions.   My first concern is that these predictions are stated only verbally, not mathematically, and it is not entirely clear what the authors mean by them. For example, given two paths to the goal, how can we determine which one is a "clearer" path? Does the answer generalize to stochastic domains? Second, I do not follow how these predictions follow from the model in section 2.3. It would be useful if the authors can show how their model leads to these predictions (for example, can they derive them analytically?). It would also improve the paper to state the predictions clearly with no ambiguity. Third, it is not clear how the two types of predicted behaviors are actually useful for learning from demonstrations. For example, given that the goal states are distinct and far from each other, why is it desirable that the paths do not intersect?   The analysis of the two experiments are not as rigorous as one would like, and some experimental details are left out, which makes it difficult to interpret the results. Detailed comments below.   EXPERIMENT 1  1. What was the role of the learning block? From the text, I understand that in both the "Do" and "Show" conditions, the subjects were explicitly shown where the reward is. And if there were any participants learning from demonstration (the text is not clear on this), these participants observed deterministic trajectories generated by people who knew where the reward is, which means that they could observe the reward location indirectly, without having to learn.  2. Was the length of the trajectory important? That is, were shorter trajectories to the goal state more rewarding than longer trajectories? How was that information conveyed to the subjects? More generally, what exactly was the underlying reward function?  3. On line 148, the authors write that the participants in the "show" condition were more likely to choose paths that helped disambiguate their goal as compared to those in the "do" condition. As noted above, it is not clear why this type of disambiguation is useful. Was it indeed useful to the participants who learned by demonstration (if they did exist; see comment 1 above)? That is, was there a difference in the success rates of the participants who learned from demonstration when presented with the different trajectories?  4. What does M denote on line 151 (also used later)? I assume it is mean of something, but of what?  5. I do not follow the sentence that starts on line 152: "For trials where..." How does the model predict marginal significance?  6. Based on lines 145-146, I conclude that each participant completed 9 trials and that these nine trials used the 3 tasks shown in Figure 1, repeating each task 3 times. Is this correct? If yes, what is the rest of the data (Figure 1 shows only 3 trials)? If not, the paper should describe all 9 trials.  7. Why does the model predict loping behavior when % is the goal? (as stated in Figure 1 caption)  8. I do not see an analysis of the % goal. The difference  between the "do" and "show" conditions in this condition is much smaller than in the & condition. Why is that? Could there be an order effect?  9. Figure 1 left-hand column is showing trajectories from only 2 of the standard planning algorithms. This is misleading because these two trajectories are not special in any way. Given that the domain is deterministic, a number of other trajectories exist that collect exactly the same return as the two trajectories that are shown. On a related note, on the second column from the left, only the two model trajectories with the highest probabilities are presented --- it would be much more informative to see a larger number of trajectories, along with the actual probabilities assigned to them by the model. This information would show, for instance, whether there is a third trajectory with identical probability or whether there is a large gap in probability between the first and the second trajectory.  10. In section 3.3, it would be useful to see a high-level discussion of the overlap between model predictions and human behavior.   EXPERIMENT 2  11. There is an inconsistency in the terminal reward given. Is it 5 (line 196) or 10 (line 178)?  12. It would be nice to see a more direct link from the pedagogical model to the specific types of trajectories described in lines 187-190. (I have made similar comments earlier regarding Experiment 1.)  13. Columns 2 and 3 in Figure 2: See comment 9 above.   14. Line 211: "potentially suboptimal paths": Longer paths on safe tiles are *not* suboptimal; they collect exactly the same return as the shorter paths in safe tiles (unless it was communicated to the participants in some way that shorter trajectories are better, but I see no indication of this)  15. Assertions in lines 212-214 should be backed up with a quantitative analysis.  16. Line 217: The choice of alpha=1.5 should be discussed. In addition, it would be useful to have an analysis of the sensitivity of the results to the value of this parameter.  17. Lines 217-221: This introduces a difficulty in interpreting the results and should be discussed in some detail. In addition, using gamma=1 but adding a small negative reward for each action would place the two approaches at equal footing, is this not correct?    18. Most numbers in Table 1 are very small (except for the 0A condition), indicating that the observed trajectories are not a good fit to either model. Furthermore, the overall difference between "show" and "do" conditions is very small (as deduced from the last column in Table 1: about 0.02). Consequently, I do not think that the statements in lines 222-228 are a fair and accurate description of the results in Table 1.  19. Lines 237-239: Text is inconsistent with the numbers shown in parenthesis.   OTHER  -- Mathematical symbols in Equation 2 should be defined.  -- Algorithm 1, line 5: set pi (initialized on line 1) is still empty so this line would lead to j being assigned to the empty set.


Deep Learning without Poor Local Minima
In this paper, we prove a conjecture published in 1989 and also partially address an open problem announced at the Conference on Learning Theory (COLT) 2015.
With no unrealistic assumption, we first prove the following statements for the squared loss function of deep linear neural networks with any depth and any widths: 1) the function is non-convex and non-concave, 2) every local minimum is a global minimum, 3) every critical point that is not a global minimum is a saddle point, and 4) there exist "bad" saddle points (where the Hessian has no negative eigenvalue) for the deeper networks (with more than three layers), whereas there is no bad saddle point for the shallow networks (with three layers). Moreover, for deep nonlinear neural networks, we prove the same four statements via a reduction to a deep linear model under the independence assumption adopted from recent work. As a result, we present an instance, for which we can answer the following question: how difficult is it to directly train a deep model in theory? It is more difficult than the classical machine learning models (because of the non-convexity), but not too difficult (because of the nonexistence of poor local minima). Furthermore, the mathematically proven existence of bad saddle points for deeper models would suggest a possible open problem. We note that even though we have advanced the theoretical foundations of deep learning and non-convex optimization, there is still a gap between theory and practice.

The paper proves two theorems: one unconditional result about the loss landscape of deep linear networks (a chain of matrices) and one assumption-based calculation for deep relu networks.  Both answer questions raised in previous work: the first is a strengthening of a conjecture and the second uses fewer assumptions than a previous result.  The results provide excellent intuition towards the robustness of neural network optimization.  Although the proofs are a fairly technical calculation, the theorems themselves are extremely intuitive.  The paper is likely to benefit researchers trying to understand why simple algorithms (SGD) suffices for technically NP-hard optimization problems, and may lead to further strengthenings or similar results.
 The paper makes a significant dent in the central mystery of deep learning: why are neural networks so easy to optimize?  The precision of the linear result is quite impressive!  The nonlinear result still makes a implausible assumption, but is stronger than previous work and the implausibility is carefully discussed.  The paper itself was a delight to read, though I will admit that the full proofs were a bit of a slog.  My main question is whether the proofs could be simplified.  Two approaches come to mind: use random perturbation tricks to avoid some of the length devoted to singularity, and constructing a monotonic descent path from any point to the global minimum.  Here's a sketch of how to construct a descent path.  First, I claim that the global minimum is    min_{rank W = p} 1/2 || Y - W X ||^2  This is clearly at least as good as the global minimum, and can be achieved by decomposing the above optimal W as     W = U D V' = Ue 1 ... 1 De 1 ... 1 Ve' = W_{H+1} ... W_1  where U D V' is the SVD of W, Ue/De/Ve are zero extended versions of U/D/V, and the 1's are rectangular extended identity matrices.  Now consider an arbitrary point W = W_{H+1} ... W_1, and decompose each factor via SVD.  Given a matrix decomposition D U V (resp. U V D) for diagonal D and orthogonal U/V, there is a smooth path to a decomposition of the form D 1 V (resp. U 1 D) that does not change the value of the product.  Similarly, D U E for diagonal D/E can be transformed into the form 1 U E without changing the product.  Using a series of these transforms chained together (which will be continuous if not smooth), we can find a path ending in the form    Ue 1 ... 1 De 1 ... 1 Ve'  A final path changing Ue/De/Ve' can be constructed to reach the global minimum, where the loss decreases monotonically.  Note that the path can also be made to increase after staying constant.  If correct, this is a much simpler proof of 2.3(i) and (ii).  The above shows that every point W can be monotonically decreased to a global minimum.  However, I believe the above proof can be extended to show that every nonglobal minimum is a saddle point, essentially by rearranging the path construction to change singular values immediately.  This would prove 2.3(iii).  I am not sure if there is a similarly easy proof of 2.3(iv).  Additional comments:  1. Does Theorem 2.3 need a size > 0 condition to ensure (i)?  2. "scaler" to "scalar" in the proof of Lemma 4.6.  3. It's good that A.7 avoids discussing the other direction, since it is false. For example, if $A = 0 \kronecker 0$ then all matrices are $A^-$, but only some matrices are Kronecker products.


Tensor Switching Networks
We present a novel neural network algorithm, the Tensor Switching (TS) network, which generalizes the Rectified Linear Unit (ReLU) nonlinearity to tensor-valued hidden units. The TS network copies its entire input vector to different locations in an expanded representation, with the location determined by its hidden unit activity. In this way, even a simple linear readout from the TS representation can implement a highly expressive deep-network-like function. The TS network hence avoids the vanishing gradient problem by construction, at the cost of larger representation size. We develop several methods to train the TS network, including equivalent kernels for infinitely wide and deep TS networks, a one-pass linear learning algorithm, and two backpropagation-inspired representation learning algorithms. Our experimental results demonstrate that the TS network is indeed more expressive and consistently learns faster than standard ReLU networks.

The paper proposes a new network architecture that, like ReLU networks, is conditionally linear.  The novelty in the proposed architecture is that the weights that determine which units are active (called activation weights in the paper) are decoupled from the weights that map the input to the output (called analysis weights in the paper).  In the proposed architecture, active units copy their full input to the output, while inactive units output zeros.  As a consequence of this design, the output of each layer is a tensor, and each successive layer in the network increases the order of the tensor by one.  The analysis weights are applied only to the final output.  Three different algorithms for training these tensor switching networks are described in the paper.  The first derives a kernel function that is equivalent to one or more tensor switching layers that are infinitely wide and have activation weights drawn from a standard normal distribution.  Such kernels may be used with small data sets to learn SVMs, and comparisons between the tensor switching kernel and the arccosine kernel (which is the equivalent kernel for deep ReLU networks) provide insights into the differences between tensor switching and ReLU networks.  The second training algorithm draws the activation weights randomly from a given distribution and then uses regularized ridge regression to learn the analysis weights using only a single pass over the training data.  The third training algorithm introduces introduces a set of auxiliary variables such that the derivatives of the loss with respect to the auxiliary variables serve as proxies for the derivatives of the loss with respect to the unit activation weights, which are zero or undefined everywhere.  Experiments on variations of MNIST show that the combination of tensor switching networks and the second training algorithm provide the best performance when learning is done in a single pass over the training data.
This is a creative piece of work, and the provision of three different learning algorithms that work and help provide insights into the proposed tensor switching architecture is impressive.  I have only a few minor comments that, if addressed, would improve the paper.  Figures 2a, b, and c would be clearer if the x axes were labeled in terms of fractions of pi.  Page 7, lines 242-243: "In terms of running time, the SS network enjoys a roughly 8× improvement in wall-clock training speed over the SS network for the 1 layer MLP." Should this be "...the RR network enjoys..."?  Page 7, lines 247-248: "the TS net architecture, [which] attains much lower asymptotic training error than the SS net but similar test errors" -- basically, TS networks overfit more than the SS networks. But this isn't a surprise, is it? The TS networks have far more parameters than the SS networks with the same architecture. One way of looking at this is that the TS networks trade off a large number of parameters for ease and speed of training.  How were the sizes of the MLPs in the experiments selected?  The one-pass training algorithm with ridge regression is somewhat reminiscent of one-pass training using randomized decision trees. The theory developed for randomized decision trees might provide a useful path for theoretical analysis of TS networks with random activation weights.


Stochastic Gradient
Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) algorithms have become increasingly popular for Bayesian inference in large-scale applications. Even though these methods have proved useful in several scenarios, their performance is often limited by their bias. In this study, we propose a novel sampling algorithm that aims to reduce the bias of SG-MCMC while keeping the variance at a reasonable level. Our approach is based on a numerical sequence acceleration method, namely the Richardson-Romberg extrapolation, which simply boils down to running almost the same SG-MCMC algorithm twice in parallel with different step sizes. We illustrate our framework on the popular Stochastic Gradient Langevin Dynamics (SGLD) algorithm and propose a novel SG-MCMC algorithm referred to as Stochastic Gradient Richardson-Romberg Langevin Dynamics (SGRRLD). We provide formal theoretical analysis and show that SGRRLD is asymptotically consistent, satisfies a central limit theorem, and its non-asymptotic bias and the mean squared-error can be bounded. Our results show that SGRRLD attains higher rates of convergence than SGLD in both finite-time and asymptotically, and it achieves the theoretical accuracy of the methods that are based on higher-order integrators. We support our findings using both synthetic and real data experiments.

Stochastic Gradient MCMC (SG-MCMC) methods provide an easy way to perform Bayesian posterior inference in large datasets, and have received a lot of interest from the research community in the last few years. One drawback of these methods is that they are asymptotically biased, meaning they do not converge exactly to the posterior distribution.  This paper shows that the bias can be reduced by a technique called Richardson-Romberg extrapolation.   The basic idea is very simple: Two SG-MCMC chains are run in parallel, with Chain 2 having half the step size of Chain 1. Also chain 2 is run for twice the number of steps as Chain 1. The injected Gaussian noise in the chains are correlated, with alpha_k = (beta_{2*k-1) + beta_{2*k})/sqrt(2) where alpha_k and beta_k are the injected Gaussian noise at time step k in Chains 1 and 2 respectively. Then, to compute the expectation of a function f w.r.t. the target distribution of the chains, one simply subtracts the estimate computed using samples from Chain 1 from the estimate computed using Chain 2. The resulting estimate is shown to have lower bias O(gamma^2) compared to O(gamma) obtained from Chain 1 or 2 alone (where gamma is the step size). They also prove a similar reduction in MSE of the estimates.
I like the fact that the method is very simple to understand and implement (see my summary), and does not require any major changes to the base SG-MCMC algorithm. Also, this seems very general and applies to a large class of SG-MCMC algorithms, and is therefore potentially very impactful to the Stochastic Gradient MCMC community.  Novelty: Although Richardson-Romberg extrapolation is well known in numerical analysis, it is not widely known in the machine learning / stochastic gradient MCMC community.   Clarity: The paper is well written and the presentation is clear.   Comments/questions:  - Can this technique be directly applied to all SG-MCMC methods? If not, are there specific conditions other than that the SG MCMC algorithm satisfies the bound in section 2.2: pi_gamma(f) = pi(f) + C * gamma + O(gamma^2)?  - Can this method be applied to other non-stochastic gradient, but approximate MCMC schemes, e.g. sub-sampling based Metropolis-Hastings tests? What about Distributed SGLD? - Since one chain has to run for twice as long as the other, there is a lot of wasted computation where the chain with the bigger step size stays idle. Instead, is it possible to use 3 chains, one chain with a step size of gamma, and two chains with step sizes gamma/2 each, and run all of them for the same number of steps? Does this increase the variance of the estimate? - Is there an extension of this method to more than 2 step sizes? - The injected noise in the chains have to be correlated, but I didnt quite understand the effect of correlation between mini-batches . Is the variance lowest when using the same batch for the k^th iteration of chain 1, and the 2k-1^th and the 2k^th iteration of chain 2? Or is the effect of this correlation negligible? - I think the matrix factorization experiment (figure 5) should be run until convergence, and you should also compare to SGD. The current plot shows that your method works better than SGLD, but it would be nice to show that your method works better than SGLD in a setting where being Bayesian is advantageous (i.e. better than SGD). Bayesian matrix factorization is not widely used in practice, so showing some numbers will convince more people to adopt your method. Also having an SGD plot as control will put the final errors of SGRRLD and SGLD in better perspective. - Although the method is easy and clearly presented, an algorithm box could make the paper more accessible to a wider audience. I believe many practitioners will not read through the more theoretical parts, so pseudo code presented early on in the paper will enable readers who are not very familiar with this area to try out your method without understanding the more challenging parts.


Structured Matrix Recovery via the Generalized Dantzig Selector
In recent years, structured matrix recovery problems have gained considerable attention for its real world applications, such as recommender systems and computer vision. Much of the existing work has focused on matrices with low-rank structure, and limited progress has been made on matrices with other types of structure. In this paper we present non-asymptotic analysis for estimation of generally structured matrices via the generalized Dantzig selector based on sub-Gaussian measurements. We show that the estimation error can always be succinctly expressed in terms of a few geometric measures such as Gaussian widths of suitable sets associated with the structure of the underlying true matrix. Further, we derive general bounds on these geometric measures for structures characterized by unitarily invariant norms, a large family covering most matrix norms of practical interest. Examples are provided to illustrate the utility of our theoretical development.

The paper under review consider recovery of general structured measurements with the Danzig selector from linear measurements.   Specifically, the paper assumes measurements y_i = < \theta,X_i > + w_i, where X_i are measurement matrices and w_i is a scalar corresponding to noise.  The paper considers the estimator   arg min_\theta R(\theta)  subject to R*(\sum_i ( < \theta,X_i > - y_i ) X_i ) \leq \lambda_n,  where R is an arbitrary matrix norm and R* is its dual.   The paper assumes sub-Gaussian measurement matrices and sub-Gaussian noise, which is a very general setup.   The main result of the paper appears to be what is stated in the text in the first paragraph of Section 5:   If the number of measurements is larger than the squared Gaussian width of the tangent cone with respect to the norm R, then the recovery error is bounded in a certain sense.  This is similar to [Cor. 3.3, 10], which pertains to Gaussian measurements and bounded noise.
1. In essence the paper provides generalizations of known results: It shows that matrices with structure (e.g., low rank) can be recovered from sub-Gaussian measurements.  What appears to be new is the very general problem setup.  Specifically the paper considers general norms R as well as the measurement matrices and the noise are assumed to be sub-Gaussian instead of Gaussian, for which corresponding results are already available.   However, several works have shown already that Gaussian measurement matrices can be substituted by sub-Gaussian measurement matrices; this is essentially a consequence of theory developed in [10] and Gordan's lemma holding for more general than Gaussian matrices. See for example the work by Dirksen ``Dimensionality Reduction with Subgaussian Matrices" for a concrete example of this line of work.   While I understand that it is not easy to prove such results, the paper would need to explain if and why the results are novel with respect to the literature mentioned above, and why and if they provide a substantial improvement with respect to the state of the art.   It might be for example that there are interesting choices of the norm R that are relevant and not covered by results in the literature yet.   2. After having read the introduction several times, it is not clear what the main contribution and the specific problem considered of the paper is. Specifically, the paper does not elaborate what it means with ``generally structured matrices''. As a consequence, it is not clear whether the problem of recovering ``generally structured matrices'' includes interesting examples beyond the usual low rank models covered by the literature already.   3. In general the writing could be improved significantly; the paper contains many typos (e.g., only on the bottom of page 5: Lemma 2 ends without colon, Base-> Based; supplementary -> supplementary material; etc), and more importantly, the statements have to be combined by the reader himself to obtain the most interesting results of the paper.   Specifically, the current structure is the following:  Section 2 starts with a deterministic recovery condition (Theorem 1) that expresses the estimation error in Frobenius norm in terms of the geometry of the tangent cone at \theta*.   Section 4 bounds quantities relevant for the deterministic recovery condition in Theorem 1 for sub-Gaussian measurement matrices and sub-Gaussian noise.   Finally, in Section 5, the paper states in words what the results in Section 2 and 4 combined yield, and some implications. This is one of the most interesting conclusions of the paper, and appears to be the main result of the paper (see above).   It would be great if the paper would be reorganized so that a main result, e.g., what is stated in the first paragraph of Section 5, would be stated and discussed in detail early in the paper.  Moreover, additional explanations on how this result improves the state of the art, and pointing to special case already treated by the literature (e.g. [Cor. 3.3, 10]) would be helpful.   As it is now, Section 4 contains several theorems that bound and relate quantities relevant for the deterministic recovery condition. E.g., Theorem 4 lower bounds the condition in terms of the Gaussian width of the tangent cone (note that this result uses generic chaining and is stated for subgaussian matrices; however for Gaussian matrices this is well know, see [10]) Only combined with Theorem 1 those statements yield a recovery result; if this is what the paper sets out to do, then it would be significantly more clear if all those Theorems 3-7 were stated as lemmas, and only one central Theorem were stated and discussed.     Minor comments:   1. Sec. 2: ``The recovery bound assumes not knowledge of R'': It is not clear what is meant with this statement. The recovery bound depends on the norm R, and the convex program assumes knowledge of R.   2. Sec. 4 ``X_i's are copies of a random vector'': Those are matrices.


Optimal Sparse Linear Encoders and Sparse PCA
Principal components analysis (PCA) is the optimal linear encoder of data. Sparse linear encoders (e.g., sparse PCA) produce more interpretable features that can promote better generalization. (i) Given a level of sparsity, what is the best approximation to PCA? (ii) Are there efficient algorithms which can achieve this optimal combinatorial tradeoff? We answer both questions by providing the first polynomial-time algorithms to construct optimal sparse linear auto-encoders; additionally, we demonstrate the performance of our algorithms on real data.

The paper considers the problem of designing “sparse linear encoders”, i.e., a linear transformation to obtain a low dimensional representation of the input data that preserves as much information as possible while satisfying additional sparsity constraints. In particular, consider a d-by-k matrix H whose columns have at most r nonzero entries. Every d-dimensional data point x is represented by a k-dimensional (encoded) vector z = H’x. The matrix H is the “encoder”. For a given H, we can also specify a “decoder” G, that maps points from k dimensions back to the original d-dimensional space, while minimizing the loss over all datapoints: ||X-XHG||_F (points are rows of X). The question is then how to design the optimal sparse encoder H.  The main result is an algorithm for designing a sparse linear encoder H, using an existing algorithm for the Column Subset Selection problem as a black box. The authors establish a reduction and are able to give approximation guarantees for the linear encoding problem based on the guarantees of the CSS algorithm.  They also propose an alternative approach based on the same algorithm, but computing the components (columns) of the encoder one by one. They also provide the first theoretical analysis (i.e. approximation guarantees) for this case as well.  They conclude with experiments comparing their approach against existing approaches for sparse PCA which focus on the objective of maximizing variance. They show that the two objectives can result to different linear encoders which is interesting.
The paper has an interesting theoretical contribution.  The objective of obtaining a low-dimensional representation of the data that minimizes a Frobenius representation error is common. The paper draws connections of the above problem with sparse PCA showing that it can be seen in fact as a generalization of the latter. The authors also have a useful discussion explaining the above “loss minimization” objective differs from (and is perhaps of greater interest to the ML community compared to) the “explained variance maximization” viewpoint for PCA (even though the two objectives are equivalent in the case of  PCA).  As mentioned above, the authors propose an algorithm for designing a sparse linear encoder utilizing a reduction to the CSS problem and using an existing CSS algorithm as a black box. The obtained linear encoder has the restriction (or property) that all k dimensions (columns of H) have the same r nonzero coordinates. The theoretical guarantees are interesting. The main theorem (Thm 7) gives a multiplicative factor guarantee for the expected achieved error with a factor that depends on the choice of the sparsity parameter r. For instance, the theorem implies that for sparsity r = Omega(k/epsilon), the linear encoder achieves a low-dimensional representation with error comparable to the best (unconstrained) rank-k approximation.  The lower bound in Thm 8 also nicely supplements the previous result. The subsequent segment sketching the proof of Thm 8 could be improved. Among others, what is B and Y in Lemma 9?  In section 3, the authors describe that their approach can be used to compute the k columns of H sequentially and also provide theoretical approximation guarantees. Of course the analysis of the sequential/greedy approach is more challenging and those guarantees are much weaker compared to those for the batch algorithm. For example, the error bound in Thm 11 includes an additive term that depends on the sum of the top singular values sigma_2^2 +...+ sigma_\ell^2. This error term would scale proportionally with the dimension of the input data. Does the “small additive term” in the subsequent comments refer to this error term?  They conclude with experiments comparing their algorithm with algorithms for sparse PCA. They show that their algorithms gives low dimensional representations with lower reconstruction (Frob) error, but perform worse with respect to the “variance” metric, supporting the claim that the two objectives result in different encoders.  Minor: Line 48: sentence ends at the comma. In the comments under Thm 7, there is a stray “2.” Lines 159-160 seem to repeat the comment in 155. “Matrrial” in line 212


Deep ADMM-Net for Compressive Sensing MRI
Compressive Sensing (CS) is an effective approach for fast Magnetic Resonance Imaging (MRI). It aims at reconstructing MR image from a small number of undersampled data in k-space, and accelerating the data acquisition in MRI. To improve the current MRI system in reconstruction accuracy and computational speed, in this paper, we propose a novel deep architecture, dubbed ADMM-Net. ADMMNet is defined over a data flow graph, which is derived from the iterative procedures in Alternating Direction Method of Multipliers (ADMM) algorithm for optimizing a CS-based MRI model. In the training phase, all parameters of the net, e.g., image transforms, shrinkage functions, etc., are discriminatively trained end-to-end using L-BFGS algorithm. In the testing phase, it has computational overhead similar to ADMM but uses optimized parameters learned from the training data for CS-based reconstruction task. Experiments on MRI image reconstruction under different sampling ratios in k-space demonstrate that it significantly improves the baseline ADMM algorithm and achieves high reconstruction accuracies with fast computational speed.

The authors present a deep architecture that implements an ADMM for compressed sensing applied to MRI recovery. The ADMM algorithm has proven to be useful for solving problems with differentiable and non-differentiable terms, and therefore has a clear link with compressed sensing. Experiments prove some gain in performance with respect to the state of the art, specially in terms of computational cost at test time.
The idea of reimplementing an iterative algorithm in a deep architecture is not new, but solving a CS problem for MRI recovery with a Deep-ADMM architecture is novel as far as I can say.  I am a bit worried about the correspondence between the "classical" and the "deep" learning problems. Indeed, the classical problem is very well stated as minimizing the augmented Lagrangian. However, when discussing the implementation of the convolutional layer, the authors say that they increase the network capacity by decoupling the transform matrix of the convolutional from the one of the reconstruction layer. Therefore, we are not solving for the augmented Lagrangian anymore, but for something else. What is the interpretation in terms of "classical" learning? Can you rewrite the optimization problem you are actually solving for and discuss similarities and differenceswith the original augmented Lagrangian?  The authors claim something before equation (8) that is not clear to me and would require explanation or proof. When Implementing the non-linear transform layer (I.e. the shrinkage), how do you know that using a piecewise linear function is more general than trying to implement the shrinkage. I mean, I understand that fitting a piecewise linear function can approximate many other functions and therefore has some flexibility: but can it implement any shrinkage operator with piecewise linear functions (recall that the control points are predefined). My understanding is that even if more flexible, this layer is perhaps not more general than the original shrinkage.  When writing the gradients used in the back-propagation, I'd rather see in the main text the actual gradients and not general chain rules that do not provide any insight on the effects of each back-propagation step.  The experimental results should be sharpened-up. It is not really convincing to me that ADMM-NET-14 does not reach state-of-the-art performance, but ADMM-NET-15 does. It is very intuitive, since you get one iteration more of the ADMM. But how can I choose the number of iterations a priori? In regular ADMM different convergence criteria can be evaluated, but with your approach the number of iterations has to be fixed in advance. Importantly, this is critical, because the number of iterations defines the number of layers, and therefore the amount of parameters of the network. Therefore, while for classical ADMM, the number of parameters is fixed with the number of iterations, here it grows. Therefore, the number of data must grow with the number of iterations. Can you please comment on that and provide some insights? In this topic, I would also appreciate to see that the parameters also converge (as the MMSE does in Fig. 6b).   Typos: - Generally speaking, check the usage of the definite article "the" very carefully. -P1L19 techniques are -P1L22 CS-MRI (without the) -P1L29 uses groupS -P1L36 proven (not proved) -P2L56 non-local techniqueS -P1L58 propose (without d)


Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles
Many practical perception systems exist within larger processes that include interactions with users or additional components capable of evaluating the quality of predicted solutions. In these contexts, it is beneficial to provide these oracle mechanisms with multiple highly likely hypotheses rather than a single prediction. In this work, we pose the task of producing multiple outputs as a learning problem over an ensemble of deep networks -introducing a novel stochastic gradient descent based approach to minimize the loss with respect to an oracle. Our method is simple to implement, agnostic to both architecture and loss function, and parameter-free. Our approach achieves lower oracle error compared to existing methods on a wide range of tasks and deep architectures. We also show qualitatively that the diverse solutions produced often provide interpretable representations of task ambiguity.

The paper proposes a new method for training ensemble of Deep Neural Networks (DNN) (or any other gradient-based approach), by using an oracle (that knows with member of the ensemble is optimal for the current instance) over each data instances used for training, applying backprop only on the network with the output selected. The approach has the advantage of letting the networks to specialized themeselves, giving ensemble that are covering well all (or most) possible output of interest. Results are reported on a variety of domains, with overall great performances.
I think that's a pretty good paper. It presents well an interesting idea of training an ensemble of DNN with an oracle, which oracle is guiding backprop update to the net with the best output (according to the true label). It appears to enforce specialization quite well in a classification context. The overall presentation is really good and the experiments are done on several domains. Globally, a strong work that deserves to be published. I would have considered this paper as outstanding if the issue exploitation of ensemble of DNNs trained with the proposed approach would also been addressed. Currently, training is done with the oracle, but the use of the resulting ensemble is not developped further, outside saying it can provide a broad pool of possible results and that the approach is coherent with the top-k metric proposed for some problems (i.e., ImageNet). This is an important missing part for making the proposed approach fully usable in practice. Nevertheless, the contribution is there in the current stage, the missing part being relatively problem-specific. Fig. 2 presents pseudo-code that is in fact raster images embedded in the paper. Why not making the pseudo-code directly within the paper, or at least embed the pseudo-code figure as a vectorial image?


Global Analysis of Expectation Maximization for Mixtures of Two Gaussians
Expectation Maximization (EM) is among the most popular algorithms for estimating parameters of statistical models. However, EM, which is an iterative algorithm based on the maximum likelihood principle, is generally only guaranteed to find stationary points of the likelihood objective, and these points may be far from any maximizer. This article addresses this disconnect between the statistical principles behind EM and its algorithmic properties. Specifically, it provides a global analysis of EM for specific models in which the observations comprise an i.i.d. sample from a mixture of two Gaussians. This is achieved by (i) studying the sequence of parameters from idealized execution of EM in the infinite sample limit, and fully characterizing the limit points of the sequence in terms of the initial parameters; and then (ii) based on this convergence analysis, establishing statistical consistency (or lack thereof) for the actual sequence of parameters produced by EM.
In many applications, each step is intuitive and can be performed very efficiently.
Despite the popularity of EM, as well as the numerous theoretical studies of its behavior, many important questions about its performance-such as its convergence rate and accuracy-have remained unanswered. The goal of this paper is to address these questions for specific models (described in Section 1.2) in which the observation Y is an i.i.d. sample from a mixture of two Gaussians.
Towards this goal, we study an idealized execution of EM in the large sample limit, where the E-step is modified to be computed over an infinitely large i.i.d. sample from a Gaussian mixture distribution in the model. In effect, in the formula forQ(η |η t ), we replace the observed data Y with a random variable Y ∼ f (y; η ) for some Gaussian mixture parameters η and then take its expectation. The resulting E-and M-steps in iteration t are E-step:
M-step:
This sequence of parameters (η t ) t≥0 is fully determined by the initial setting η 0 . We refer to this idealization as Population EM, a procedure considered in previous works of Srebro (2007) and Balakrishnan et al. (2014). Not only does Population EM shed light on the dynamics of EM in the large sample limit, but it can also reveal some of the fundamental limitations of EM. Indeed, if Population EM cannot provide an accurate estimate for the parameters η , then intuitively, one would not expect the EM algorithm with a finite sample size to do so either. (To avoid confusion, we refer the original EM algorithm run with a finite sample as Sample-based EM.)

The authors provide an analysis of the EM algorithm for two toy Gaussian mixture models characterizing the initial parameter values that lead to convergence. To this end, they use the true Q function-- obtained as an expectation instead of doing a sample average-- to give the ideal parameter sequence from a given initial value. This sequence is used for the characterization of the initial parameter values that lead to the true parameter. They also demonstrate that the actual sequence of real run converges to the idealized sequence in probability. Later they use this analysis to explore the shape of the expected log likelihood in the neighborhood of its stationary points.
Congratulations to the authors for a good paper!  Quality: The paper is technically sound with non trivial results. The conclusions of the paper are well supported by the theory. The condition on the initial parameter that leads to convergence to the the true parameter are particularly interesting.   Clarity: This is a very well written paper and it reads well. The math, though not trivial, is very accessible because of the presentation. The authors have provided adequate commentary that aids intuition and understanding.   Originality: The idea of the paper to analyze an idealized execution of the EM algorithm, by using the true Q function, instead of the one obtained by sample average is quite novel. The authors have provided adequate discussion of the relevant literature to inform the reader about the state of the art and evaluate their work in the right context.   Significance: The results of the paper are very interesting, they help understand the sensitivity of EM  (Gaussian mixtures) to initial parameters. Though the models considered are too restrictive for real world data, but they work more like a proof of concept for the analysis technique. It remains to see if the technique would work for more general models.   Strengths: Solid analysis  Weakness: Restrictive model.   Comments: In line 188-192 authors claim that their analysis for EM provides understanding for other ascent algorithm. However, in the justification that follows the conclusions about G(\eta) are still made using the Q function, which is in the EM framework. It is not clear what the authors want to claim about other algorithms.  Overall, I would recommend accepting the paper without reservations.


Threshold Bandit, With and Without Censored Feedback
We consider the Threshold Bandit setting, a variant of the classical multi-armed bandit problem in which the reward on each round depends on a piece of side information known as a threshold value. The learner selects one of K actions (arms), this action generates a random sample from a fixed distribution, and the action then receives a unit payoff in the event that this sample exceeds the threshold value. We consider two versions of this problem, the uncensored and censored case, that determine whether the sample is always observed or only when the threshold is not met. Using new tools to understand the popular UCB algorithm, we show that the uncensored case is essentially no more difficult than the classical multi-armed bandit setting. Finally we show that the censored case exhibits more challenges, but we give guarantees in the event that the sequence of threshold values is generated optimistically.

The authors begin by providing an alternative proof for the regret bound of UCB. The endeavor of this seems to be to introduce the "potential trick" which the authors later use in the threshold setting. To my understanding, the threshold setting considered in this paper is a generalization of the UCB setting provided that the reward functions are all Bernoulli 0-1. To see this, one can fix c^t=c for all t. Then clearly the dominating arm is the one that maximizes $P(X^t_i \ge c)$. The distribution of the reward function $D_i$ is $1 w.p. P(X^t_i \ge c)$ and 0 w.p. 1- $P(X^t_i \ge c)$. I was wondering if a more appropriate generalization would be to reward $X^t_j$ provided it crosses the threshold. Probably in the setting of Dark pool brokerage this setting may find an application.  Coming back to the threshold UCB described in the paper, supposing $c^t=c$ for all $t$ then we are back in the UCB setting. When the sequence of thresholds is predetermined, one does not expect the analysis to deviate too much from Auer et. al. Having said that, as the authors state, the potential trick certainly fits in well.  The non-trivial threshold setting (uncensored and censored) is when $c^t$ is non-deterministic. Provided that $c^t$ takes values from a finite set the authors provide a regret bound that uses the DKW inequality to determine $f$, which in turn is used to fix the two auxiliary functions that allows for the potential trick to be applied. As the authors explain, one cannot just count the number of bad-arm-pulls especially when $c^t$ is not constant.  For the setting of the censored feedback setting, the authors provide a new estimator based on the Kaplan-Meier estimator, then go on to find $f$ (error bound for the estimator). The regret bounds should now proceed as in the previous sections.   Finally, they illustrate that the ordering of the threshold values affect the cost of learning. This is to be expected. The cost also seems to depend on how many values the threshold can take.
It would be interesting if the authors can justify why are the new rewards are 0-1 and not $X^t_j$ or in general some different distribution provided the threshold conditions are met.  It seems that the likeness of UCB and DKWUCB is due to the uniform convergence of the empirical distributions to their "means". Does this mean that we can let $c^t$ take values over a continuous range provided such uniform-convergence-property hold.   There should be a way to get around assuming that argmax_i F_i(j) is unique. This would certainly make it more applicable. Or is it that the $c^t$ may be perturbed to justify the assumption.  It would be interesting if the authors can provide a detailed setting in a practical application wherein the $c^t$'s are non-deterministic/uniformly distributed over a finite set.  The potential trick seems to capture the scenario in sequential decision settings wherein the "good arm"/right action changes with time. Can there be more insight into where the potential trick will prove helpful?


A Sparse Interactive Model for Matrix Completion with Side Information
Matrix completion methods can benefit from side information besides the partially observed matrix. The use of side features that describe the row and column entities of a matrix has been shown to reduce the sample complexity for completing the matrix. We propose a novel sparse formulation that explicitly models the interaction between the row and column side features to approximate the matrix entries. Unlike early methods, this model does not require the low rank condition on the model parameter matrix. We prove that when the side features span the latent feature space of the matrix to be recovered, the number of observed entries needed for an exact recovery is O(log N ) where N is the size of the matrix. If the side features are corrupted latent features of the matrix with a small perturbation, our method can achieve an -recovery with O(log N ) sample complexity. If side information is useless, our method maintains a O(N 3/2 ) sampling rate similar to classic methods. An efficient linearized Lagrangian algorithm is developed with a convergence guarantee. Empirical results show that our approach outperforms three state-of-the-art methods both in simulations and on real world datasets.

This paper addresses the problem of matrix completion when the side information about the features is available in form of features vector matrices X and Y. In such problems the matrix to be completed can be written as F = X^T G Y. The paper considers a model in which the matrix F is low rank and G is sparse. They propose a recovery method that minimizes the nuclear norm of X^T G Y and \ell_1 norm of G. This seems to be a fairly intuitive approach to complete the matrices that follow this model. The paper also proposes a solution along the lines of regularized maximum likelihood formulation for the matrix completion under Gaussian noise assumptions. Finally, the authors propose a LADMM algorithm for solving the problem and provide a proof of its convergence. Overall, the paper looks fairly exhaustive in the theoretical analysis of the proposed solution.
 The main comments are as follows:   1) The problem formulation under noise given in equation (2) of the paper seems to be problematic. It is not clear where the noise enters in the overall setup. Generally, the observations are corrupted by noise, i.e., R_{\Omega}(F) =  R_{\Omega} (X^T G Y + N). Under such a scenario, the likelihood assuming gaussian noise is given by || R_{\Omega}(F) -  R_{\Omega} (X^T G Y )  ||_F^2. The paper however does something confusing. It introduces a matrix E, and introduces a constraint  R_{\Omega}(E) = R_{\Omega}(F) and uses the likelihood term ||E - X^T G Y||_F^2.  This is problematic because the estimated low rank matrix E is constrained to have same value as the noisy observation. This basically will eliminate the possibility of de-noising the observed part of the matrix. As per my knowledge of existing literature for noisy matrix completion problem, the observed entries of the estimated matrix are never constrained to be equal to their noisy observations.    2) The main recovery result given in Theorem 2 introduces model assumptions ||E||_* <= \alpha and ||G||_1 <= \gamma. These assumptions raise some questions whose answers are not well argued in the paper. Firstly, even though bounds on any norm of the model parameters makes it inherently less complex, bounded nuclear norm and \ell_1 norm does not necessarily imply low rank E and sparse G. The whole paper is built upon the notion that G is sparse and E is low rank but the assumption in theorem does not necessary impose these on the model. In this regard looking at the Theorem statement it is not clear why authors minimize weighed sum nuclear norm and \ell_1 norm. Under the model assumptions stated in the Theorem a straight forward alternative is to consider constrained maximum likelihood formulation, i.e., put the constraints ||E||_* <= \alpha, ||G||_1 <= \gamma explicitly in the recovery problem formulation. A discussion based on these points will significantly improve the clarity of the paper from motivational point of view.     3) It is not clear how do the recovery results in the Theorem statement depend on the regularization parameters \lambda_E and \lambda_G.  As the value of regularization parameters increases the error should increase for very large values of these parameters. But this is not evident from the recovery bounds.    4) In the final LADMM formulation it is not clear why authors introduce an auxiliary variable C. Even without C, the G update step and E update are similar. For clarity authors need to explain why C is needed.   Minor Comments:  1) In abstract, it is not clear what is the size of the matrix. Mentioning that it is N = max{n_1, n_2} where n_1 is the number of rows and n_2 is number of columns.  2) Typo ’impaire’ on line 125.  3) In statement of Theorem 2, \Chi^2 is not defined.  4) After Theorem 3, a comment on how to calculate \tau_a and \tau_b would help the reader to use the proposed algorithm.


High-Rank Matrix Completion and Clustering under Self-Expressive Models
We propose efficient algorithms for simultaneous clustering and completion of incomplete high-dimensional data that lie in a union of low-dimensional subspaces. We cast the problem as finding a completion of the data matrix so that each point can be reconstructed as a linear or affine combination of a few data points. Since the problem is NP-hard, we propose a lifting framework and reformulate the problem as a group-sparse recovery of each incomplete data point in a dictionary built using incomplete data, subject to rank-one constraints. To solve the problem efficiently, we propose a rank pursuit algorithm and a convex relaxation. The solution of our algorithms recover missing entries and provides a similarity matrix for clustering. Our algorithms can deal with both low-rank and high-rank matrices, does not suffer from initialization, does not need to know dimensions of subspaces and can work with a small number of data points. By extensive experiments on synthetic data and real problems of video motion segmentation and completion of motion capture data, we show that when the data matrix is low-rank, our algorithm performs on par with or better than low-rank matrix completion methods, while for high-rank data matrices, our method significantly outperforms existing algorithms.

The paper presents an extension to the sparse subspace clustering (SSC) algorithm that allows to complete missing values while also recovering the sparse representation of data. With the sparse representation at hand, the clustering is performed in exactly the same way as in the original SSC algorithm. The authors show that the joint matrix completion and sparse learning may yield better clustering results than performing the two steps independently.
Major comments: The paper describes an interesting extension of the SSC algorithm to allow joint matrix completion and sparse representation. The mathematical formulation is straightforward and the main contribution seems to be in the way the problem is made computationally tractable. I did not find any issues with the technical approach and I think the contribution in that respect is sound.  On the downside, I was not convinced of the practical importance of this framework. Yes, joint matrix completion and subspace clustering seems like an interesting theoretical problem to solve, but do we have a practical real-life example where the benefits of this framework would become evident, particularly compared to the approach where one performs matrix completion and clustering in sequence (especially since LRMC+SSC performed quite decently)?   - The paper shows that the proposed approach outperforms competition in the matrix completion task, but the specific scenario under which one would be interested in matrix completion accuracy feels contrived.  - Additionally, I think the "real" experiments presented in the paper are in fact simulated. The "real" experiments were performed on data that was generated by simulating particular distributions using realistic datasets as the basis. These are not experiments on realistic datasets that would justify the applicability of the approach in real life.   Minor comments:  1. I found the notations paragraph(lines 82-98) somewhat confusing, primarily because it felt unnecessarily generic. For example: - why not state simply that \bar{y_i} is the original datapoint with missing values replaced by zeros.  - why not describe U_{\Omega_j} in simple words. Also, matrices U and P are denoted as matrices of real values, while in reality they are matrices over {0,1}.  2. The font of the references section seems very small.


Correlated-PCA: Principal Components' Analysis when Data and Noise are Correlated
Given a matrix of observed data, Principal Components Analysis (PCA) computes a small number of orthogonal directions that contain most of its variability. Provably accurate solutions for PCA have been in use for a long time. However, to the best of our knowledge, all existing theoretical guarantees for it assume that the data and the corrupting noise are mutually independent, or at least uncorrelated. This is valid in practice often, but not always. In this paper, we study the PCA problem in the setting where the data and noise can be correlated. Such noise is often also referred to as "data-dependent noise". We obtain a correctness result for the standard eigenvalue decomposition (EVD) based solution to PCA under simple assumptions on the data-noise correlation. We also develop and analyze a generalization of EVD, cluster-EVD, that improves upon EVD in certain regimes.

The paper analyzes the PCA (eigen value decomposition -EVD) for the cases where underlying data and noise are correlated. The authors prove the correctness of this approach, providing an upper bound on the subspace recovery error, and then provide a generalization of PCA - cluster EVD, which overcomes limitations on the sample complexity of PCA, which is bound by the condition number of the covariance matrix.   The paper provides various models of correlation between data and noise, and provides background literature on the recovery error and sample complexity for existing approaches that solve this problem.   The main contribution is the derivation of cluster-EVD, and proving the upperbound on the sample complexity when assumptions on the nature of clustering of eigen values across scales holds.
The paper analyzes the PCA (eigen value decomposition -EVD) for the cases where underlying data and noise are correlated. The authors prove the correctness of this approach, providing an upper bound on the subspace recovery error, and then provide a generalization of PCA - cluster EVD, which overcomes limitations on the sample complexity of PCA, which is bound by the condition number of the covariance matrix.   The adaptation of clustering of eigen vlaues and the strategy used are novel.   The proofs are reasonably well structured and clear. The authors could possibly position theoram 3.3 earlier in the text, and add the proof outline later.   The authors study some cases of videos where the assumption of clustered eigen values is observed. Some more practical scenarios will be useful to understand the scope of applicability of the algorithm.  The experiment section is validates the method in their chosen metrics, in comparison to other methods that solve similar problems in PCA. The authors clarify that there are certain generalized cases (where the clustered eigenvalues assumption doesn't hold, assumption on missing/corrupted values doesn't hold), where cluster-EVD is not optimal. A possible experiment would be a comparison with other methods when these assumptions are not satisfied. A satisfactory performance would be show that cluster-EVD can be used in less restrictive cases as well.   Overall, the paper provides a key observation that solves a specific problem. The paper is well written.


Multi-view Anomaly Detection via Robust Probabilistic Latent Variable Models
We propose probabilistic latent variable models for multi-view anomaly detection, which is the task of finding instances that have inconsistent views given multi-view data. With the proposed model, all views of a non-anomalous instance are assumed to be generated from a single latent vector. On the other hand, an anomalous instance is assumed to have multiple latent vectors, and its different views are generated from different latent vectors. By inferring the number of latent vectors used for each instance with Dirichlet process priors, we obtain multiview anomaly scores. The proposed model can be seen as a robust extension of probabilistic canonical correlation analysis for noisy multi-view data. We present Bayesian inference procedures for the proposed model based on a stochastic EM algorithm. The effectiveness of the proposed model is demonstrated in terms of performance when detecting multi-view anomalies.

This paper provides an approach, based on PPCA and PCCA, to finding outlier observations in multi-view data sets. The model embeds observations in a latent space shared across views and finds projections from that latent space to each view. Observations that do not agree with the shared latent space and projection (common to all observations in a particular view) are recognized as outliers. The approach uses stochastic EM to approximate the random variables of the model conditioned on data.
The paper presents an intriguing approach to outlier detection. Using the probability of the number of clusters being greater than 1 is an interesting innovation that I have not seen before. One limitation of that approach I see is whether to interpret instances with more than one latent vector as "outliers" or just multimodal. The description at the end of Section 5 bears this out: does the fact that a movie has more than one genre make it an outlier? In general, this means that the proposed method will work best when each instance can be represented with a single vector across multiple views.  Presentation was rated "sub-standard" mostly due to the very dense blocks of text in some places (e.g., the related work section). Also, it would be better to have subsections in the paper to structure the text more. The derivations starting line 99 needed more detail for how the marginal distributions were arrived at (perhaps a large chunk of this part can be pushed to the appendix?).  The experiments covered a large number of data sets but the only meaningful benchmark was PCCA (and SVM for one experiment). I thought the method of creating anomalous instances was clever and could be useful to others. The paragraph starting line 219 describes results for multi-view anomaly detection but it would be helpful in pointing toward specific data sets in the evaluation when doing this. The purpose of the paragraph starting line 238 was not clear to me, I could not see why it was needed.   I noticed a typo on line 89 (missing space).


SEBOOST -Boosting Stochastic Learning Using Subspace Optimization Techniques
We present SEBOOST, a technique for boosting the performance of existing stochastic optimization methods. SEBOOST applies a secondary optimization process in the subspace spanned by the last steps and descent directions. The method was inspired by the SESOP optimization method, and has been adapted for the stochastic learning. It can be applied on top of any existing optimization method with no need to tweak the internal algorithm. We show that the method is able to boost the performance of different algorithms, and make them more robust to changes in their hyper-parameters. As the boosting steps of SEBOOST are applied between large sets of descent steps, the additional subspace optimization hardly increases the overall computational burden. We introduce hyper-parameters that control the balance between the baseline method and the secondary optimization process. The method was evaluated on several deep learning tasks, demonstrating significant improvement in performance. Video presentation is given in [15]

The paper proposes to improve stochastic gradient algorithms by periodically applying an additional optimization in a subspace spanned by a set of previously stored then-locally-optimal directions. It is an adaptation of the Sequential Subspace Optimization (SESOP) framework of Nakiss & Zibulevsky to the minibatch setting. Experiments show that it can improve the performance compared to the baselie stochastic gradient descent algorithms that it is used in conjunction with.
The paper is overall clearly written, but one important aspect of the algorithm remains not sufficiently expounded: how precisely the subspace optimization is carried over. The paper only mentions in passing that it uses conjugate gradient (CG), but a number of points would deserve further clarification: a) is CG done over a *single* larger minibatch? And how precisely is this minibatch chosen. b) specify how you compute/provide the gradients needed for that CG subspace optimization, and what is the associated computational cost c) that optimization is still not convex, so you must be meaning nonlinear-CG? Which version/implementation do you use?   The computational cost *and* additional memory requirement (as this can constitute a practical limitation for large nets) for the subspace optimization would need to be disclosed and made precise.   Since in effect multiple CG optimizations are performed on different minibatches, it would have been welcome to contrast this approach, at least in discussion, with attempts to adapt CG to the stochastic/minibatch case.  Experimental results convincingly show that the approach can give a boost to minibatch-SGD-with-momentum, Nesterov accelerated gradient, and adagrad. But for vanilla SGD at least I would have appreciated to see the use of a learning rate decay rather than a fixed learning rate, and the assurance that all optimization hyperparameters (starting learning rate, momentum, learning rate decay) had been properly hyper-optimized to yield the lowest training error after the number of considered and displayed epochs. Captions in the figure stating that "baseline parameters was set .... which provided good convergence" is not precise enough (what was your criterion for "good convergence" and how extensively were these parameters tuned).  Also why, for an optimization method, do you report only test error in your graphs (all graphs are labeled with "test error")? Wouldn't the average error over the whole *training* set have been more appropriate, as it wouldn't risk conflating optimization and generalization ability? The best would be to show both.  In line 156 you comment Figure 3 saying "SEBOOST is able to significantly improve SGD and NAG and shows improvement over AdaGrad although not as noticeable", but I see exactly the contrary happening in Figure 3 (unnoticeable difference for SGD and NAG)!?! Your statement regarding SGD qualitatively supported by Figure 2 but then this is contradicted by Figure 3...   Further detailed remarks: l 82:"as few parameters as required" plase clarify l 103: "current gradient": specify what is meant precisely by current gradient l 107 "useful to normalize": how precisely, and why is it useful to normalize a *direction*?


Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
Effective convolutional neural networks are trained on large sets of labeled data. However, creating large labeled datasets is a very costly and time-consuming task. Semi-supervised learning uses unlabeled data to train a model with higher accuracy when there is a limited set of labeled data available. In this paper, we consider the problem of semi-supervised learning with convolutional neural networks. Techniques such as randomized data augmentation, dropout and random max-pooling provide better generalization and stability for classifiers that are trained using gradient descent. Multiple passes of an individual sample through the network might lead to different predictions due to the non-deterministic behavior of these techniques. We propose an unsupervised loss function that takes advantage of the stochastic nature of these methods and minimizes the difference between the predictions of multiple passes of a training sample through the network. We evaluate the proposed method on several benchmark datasets.

The paper explores the potential of using unlabeled data (transformed versions of the training set) for increasing the generalization properties of convolutional neural network. It formulates an unsupervised loss function, which can be used in conjunction with standard supervised training, for minimizing the variability of multiple passes of the same point through the network due to various stochastic regularization aspects of training (dropout, stochastic pooling etc.) or various augmentation transformations. The loss depends on minimizing the prediction difference among all combinations of transformed or perturbed points (without using label information), along with enforcing mutual exclusivity in the values of the prediction output vector. The large body of this work is of experimental nature, with multiple benchmarks using 2 different network architectures. The unsupervised loss improves performance consistently, especially in the low sample regime, and achieves sate-of-the-art performance in CIFAR10 and CIFAR100.
This work proposes to use semi-supervised learning, in the form of an unsupervised loss term, for improving the regularization capacity of CNNs. The idea (and the proposed loss) is conceptually simple and enforces stability explicitly by minimizing the difference between predictions corresponding to the same input data point. The paper focuses mainly on the experimental side, devoting the largest part in presenting results when adding the new loss on standard supervised CNNs. This is the stronger aspect of this work, with the weaker being the lack (or the definition) of baselines and the lack of some form of theoretical justification, derivation or discussion.    Novelty/originality: The main contribution is the application of the unsupervised loss term for controlling the stability of the predictions under transformations or stochastic variability. The mutual exclusivity term is from previous work. There is merit in the experimental results promoting the case of semi-supervised learning in training of large neural networks.     Technical quality: Extensive experimental evaluations are given on well known vision datasets with two different models and two different perturbation types (transforming the input and randomization during training), focusing on comparing performance with and without the new loss. Comparisons with one other relevant method is included (but not reproduced or controlled for). Baselines could be further improved (e.g. by including supervised training on augmented labeled data, using the same transformations like in the unsupervised case).  Potential impact: The experimental study makes the case for using unlabeled data to complement supervised training (like in [30]) and enforcing stability, through a simple loss, on the predictions. In addition, results that achieve state-of-the-art are reported on CIFAR10 and CIFAR100.   Q (analysis): The proposed loss is not studied theoretically or discussed in the context of the underlying perturbations. As a matter of fact many of these techniques play the role of additional regularization, while data augmentation acts as an expansion of the training set. For the latter, there is obvious purpose in enforcing the mapping to be the same (same model, perturbed input). The former, however, seems like enforcing different models to give the same prediction, thus canceling out the effects of randomization in the process). Some discussion would be useful here (e.g.see the equivalence of dropout to model averaging).    Q (efficiency): How does the proposed loss scale up with the number of transformations in the augmented set. Since (from (1)), there is a quadratic dependency on the number of augmented versions of the data n, which values of n are practical for computing the loss and being useful for the type of transformations used? The authors use n = 4, 5 here. These values may be small for the case augmentation via geometric transformations (e.g. affine). An empirical study on the role of n (and the trade-off between performance and efficiency) might be insightful.   Q (implementation): How is the loss computed during training? The paper mentions creating min-batches with replicates (or transformations) of the same data points. Will this create gradient bias or affect convergence?     Q (baselines): See below on experiments.   Q: The authors make a comment (Sec. 3, line 102-103) that the loss could be potentially used on labeled samples also to enforce stability. This could potentially be another interesting baseline to explore (in connection to unsupervised vs. supervised training).  Clarity and presentation:  - Paragraph 1 in Sec. 2 (Related work) is very generic and could be revised (or omitted all-together) to focus more on SSL method sin CNNs (like par. 2). Similar, a part of the related work should be devoted to stochastic regularization techniques (pooling, dropout etc.) and data augmentation.   - The network parameter description (line 155) can be confusing (especially if this is typeset in math mode). Is there a way to make the description more compact/descriptive? - Sec 4.2 and 4.3 could be merged (or shortened) as they describe the exact same experimental settings on two different datasets (SVHN and NORB).   Comments on experiments: - Baselines (e.g. line 151/152): The comparisons involve training without augmentation (supervised) and with augmentation for the unsupervised loss (semi-supervised). There should be anther baseline in my opinion -- training on the entire augmented labeled set (supervised with data augmentation). This would be equivalent to adding a loss like \sum_j |y -f(T^j x)| for each x in the training set, thus letting the CNN encode the transformations in the weights.   - Comparisons with Ladder networks [30] (Table 1): The comparison might not be exactly fair since the results are directly taken from Table 2 in [30] (i.e. not reproduced, for example, for the 100 sample training sets or with dropout/fractional max pooling etc.). In addition, the baseline CNN (supervised) for ladder net is different (supervised error is larger). Thus the absolute error comparison here might be misleading --a relative error decrease (with respect to the supervised network used as baseline for each model) might be better.  Similar, the comparison in Table 4 is without data augmentation for the ladder networks.   Secondary/minor comments and typos: - Many references have missing entries or only dates. Please revise. Update ImageNet ref [5].   - line 69: "Xavier's method"?  - line 35: "more stable generalization"?: stability is generalization


Dense Associative Memory for Pattern Recognition
A model of associative memory is studied, which stores and reliably retrieves many more patterns than the number of neurons in the network. We propose a simple duality between this dense associative memory and neural networks commonly used in deep learning. On the associative memory side of this duality, a family of models that smoothly interpolates between two limiting cases can be constructed. One limit is referred to as the feature-matching mode of pattern recognition, and the other one as the prototype regime. On the deep learning side of the duality, this family corresponds to feedforward neural networks with one hidden layer and various activation functions, which transmit the activities of the visible neurons to the hidden layer. This family of activation functions includes logistics, rectified linear units, and rectified polynomials of higher degrees. The proposed duality makes it possible to apply energy-based intuition from associative memory to analyze computational properties of neural networks with unusual activation functions -the higher rectified polynomials which until now have not been used in deep learning. The utility of the dense memories is illustrated for two test cases: the logical gate XOR and the recognition of handwritten digits from the MNIST data set.

The paper studies an associative memory model with polynomial and rectified polynomial Hamiltonians. The results focus on the one-step retrieval case, for which the model is closely related to a standard feedforward neural network. The authors motivate the use of rectified polynomial activation functions experimentally and invoking an increase in capacity argument.
The theoretical contribution presented in 291—310 is a welcome insight on the computational power of ReLUs. The experimental results for rectified polynomial units reported in figures 2 and 3 are interesting and apparently novel, even in the context of standard feedforward multi-layer networks.  Being 291—297 a central point of the paper it should be expanded and better justified. Furthermore, the simple capacity analysis developed in p. 3 for the polynomial energy function is invoked here for the rectified polynomial energy function. This has to be justified.  The paper starts from and mostly focuses on the associative memory (Hamiltonian) formulation, but then the findings are restricted to one-step retrieval. Can the authors provide at least some discussion on how to extrapolate the current findings to the case of recurrent network dynamics? Could a toy example relevant to learning be designed to illustrate the model under iterative retrieval?  The reference list should be extended and the novel aspects of the work more clearly delineated. There is previous published research on variants of the Hopfield model where the energy function is extended to include higher order terms. In particular, energy functions of the form of Eq. 2 with polynomial F have been studied before, for instance by Abbott and Arian (Phys Rev A, 1987; see also Ref. 4 therein). Such works should be cited. For example, the reported nonlinear increase (98–104) in capacity with polynomial degree has been found before.  Regarding the introduction, it should be noted that the reformulation of a classification problem as a partial input completion one is not novel. It is actually a common approach to use similar energy-based models to perform classification.  I couldn’t understand the remark on 105–108 on the structuring of the higher order interactions. In fact, how was the number of ‘memories’ K set for the MNIST experiments, and where is it reported?  Regarding section 5: can a comment be made on how a network with more than one hidden layer could be obtained starting from an associative memory problem? How would the picture change beyond one-step retrieval?  The Appendix would become clearer by including the weight update obtained after differentiating C.