Fixed-Rank Approximation of a Positive-Semidefinite Matrix from Streaming Data Volkan Cevher EPFL
Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates. Because of storage limitations, it may only be possible to retain a sketch of the psd matrix. This paper develops a new algorithm for fixed-rank psd approximation from a sketch. The approach combines the Nyström approximation with a novel mechanism for rank truncation. Theoretical analysis establishes that the proposed method can achieve any prescribed relative error in the Schatten 1-norm and that it exploits the spectral decay of the input matrix. Computer experiments show that the proposed method dominates alternative techniques for fixed-rank psd matrix approximation across a wide range of examples.

The authors address the problem of fixed-rank approximation of a positive
semidefinite (psd) matrix from streaming data. The authors propose a simple
method based on the Nystrom approximation, in which a sketch is updated as
streaming data comes in, and the rank-r approximation is obtained as the best
rank-r approximation to the full Nystrom approximation. For Gaussian or
uniformly distributed (Haar) orthogonal sketch matrices, the authors prove a
standard  upper bound on the Schatten-1 (aka Nuclear) norm of the approximation error.
Detailed empirical evidence are discussed, which indicate that the proposed
method improves on previously proposed methods for fixed-rank approximation of
psd matrices.
The paper is very well written and easy to follow. The result appears to be
novel. The proposed method admits a theoretical analysis and seems to perform
well in practice. Implementation details are discussed. This paper whould be a
welcome addition to the steaming PCA literature and is a clear accept in my
opinion.
Minor comment:
- The "experimental results" section does not mention which test
matrix ensemble was used, and whether the true underlying rank r was made
available to the algorithms.
- Some readers will be more familiar with the term "nuclear norm" than with the
  term "Schatten-1 norm"
- It would be nice to discuss the increase  in approximation error
due to streaming, compared with a classical sketch algorithm that is given all
the data in advance.


Natural Value Approximators: Learning when to Trust Past Estimates
Neural networks have a smooth initial inductive bias, such that small changes in input do not lead to large changes in output. However, in reinforcement learning domains with sparse rewards, value functions have non-smooth structure with a characteristic asymmetric discontinuity whenever rewards arrive. We propose a mechanism that learns an interpolation between a direct value estimate and a projected value estimate computed from the encountered reward and the previous estimate. This reduces the need to learn about discontinuities, and thus improves the value function approximation. Furthermore, as the interpolation is learned and state-dependent, our method can deal with heterogeneous observability. We demonstrate that this one change leads to significant improvements on multiple Atari games, when applied to the state-of-the-art A3C algorithm.

This paper proposes a novel way to estimate the value function of a game state, by treating its previous reward and value estimations as additional input besides the current state. These additional input is the direct reason why value function depicts a non-smoothness structure (e.g., a sparse immediate rewards lead to a bump in the value function). By taking them into consideration explicitly, the value function can be more easily estimated.
Although the proposed idea is quite interesting, there are a lot of baselines that the proposed method might need to compare. As mentioned in Section 7, eligible trace estimates the cumulative return as a linear combination of k-step *future* returns with geometric weights. Also Generalized Advantage Estimation, as a novel way to estimate the overall return using future estimation, is another example. Comparing these previous approaches with the proposed method that uses past estimation is both interesting and important.
In addition, there exist many simple approaches that also captures the history (and sparse reward that happened recently), e.g., frame stacking. A comparison will also be interesting.
The paper starts with the story that the value function is smooth, if the non-smooth part is explained away by previous rewards and values. There are also motivating examples (e.g., Fig. 1 and Fig. 2). However, there is no experiments in Atari games showing that the estimated value function using proposed approach is indeed smoother than the baseline.
Table. 1 shows that while median score shows a strong boost, mean score of the proposed approach is comparable to the baseline. This might suggests that the proposed approach does a good job in reducing the variance of the trained models, rather than giving higher performance. In the paper, there seems to be no relevant analysis.
Some detailed questions:
1. According to the paper, the modified A3C algorithm uses N = 20 steps, rather than N = 5 as in the vanilla A3C. Did the baseline also use N = 20? Note that N could be an important factor for performance, since with long horizon the algorithm will see more rewards in one gradient update given the same number of game frames.
2. Line 154 says "all agents are run with one seed". Does that mean the agents are initialized by the same random seed among different games, or the game environment starts from the same seed? Please clarify.


Implicit Regularization in Matrix Factorization
We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix X with gradient descent on a factorization of X. We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a full dimensional factorization converges to the minimum nuclear norm solution.

This paper studies an interesting observation:
with small enough step sizes and initialization close enough to the origin,
even for an underdetermined / unregularized problem of the form (1),
gradient descent seems to find the solution with minimum nuclear norm.
In other words, starting from an initialization with low nuclear norm,
gradient descent seems to increase the norm of the iterates "just as needed".
The authors state that conjecture and then proceed to prove it for the
restricted case when the operator A is based on commutative matrices.
The paper is pretty interesting and is a first step towards understanding
the implicit regularization imposed by the dynamics of an optimization algorithm
such as gradient descent.
There seems to be many small errors in Figure 1 and 2 (see below).
Detailed comments
-----------------
Line 46: I think there are way earlier references than [16] for multi-task
learning. I think it is a good idea to give credit to earlier references.
Another example of model that would fall within this framework is
factorization machines and their convex formulations.
Line 63: I guess X = U U^T? X is not defined...
Figure 1:
- the figure does not indicate (a) and (b) (I assume they are the left and right plots)
- what does the "training error" blue line refer to?
- X_gd is mentioned in the caption but does not appear anywhere in the plot
Figure 2:
- I assume this is the nuclear norm of U U^T
- X_gd (dotted black line) does not appear anywhere
- It is a bit hard to judge how close the nuclear norms are to the magenta line
without a comparison point
- How big is the nuclear norm when d < 40? (not dislpayed)
Line 218: I assume the ODE solver is used to solve Eq. (3)?
Please clarify.


Z-Forcing: Training Stochastic Recurrent Networks
Many efforts have been devoted to training generative latent variable models with autoregressive decoders, such as recurrent neural networks (RNN). Stochastic recurrent models have been successful in capturing the variability observed in natural sequential data such as speech. We unify successful ideas from recently proposed architectures into a stochastic recurrent model: each step in the sequence is associated with a latent variable that is used to condition the recurrent dynamics for future steps. Training is performed with amortised variational inference where the approximate posterior is augmented with a RNN that runs backward through the sequence. In addition to maximizing the variational lower bound, we ease training of the latent variables by adding an auxiliary cost which forces them to reconstruct the state of the backward recurrent network. This provides the latent variables with a task-independent objective that enhances the performance of the overall model. We found this strategy to perform better than alternative approaches such as KL annealing. Although being conceptually simple, our model achieves state-of-the-art results on standard speech benchmarks such as TIMIT and Blizzard and competitive performance on sequential MNIST. Finally, we apply our model to language modeling on the IMDB dataset where the auxiliary cost helps in learning interpretable latent variables.

The paper introduces a training technique that encourages autoregressive models based on RNNs to utilize latent variables. More specifically, the training loss of a recurrent VAE with a bidirectional inference network is augmented with an auxiliary loss.
The paper is well structured and written, and it has an adequate review of previous work.
The technique introduced is heuristic and justified intuitively. The heuristic is backed by empirical results that are good but not ground-breaking.
The results for speech modelling are very good, and for sequential MNIST good. The auxiliary loss brings no quantitative improvement to text modelling , but an interesting use for the latent variables is shown in section 5.4.
Some concerns:
Line 157 seems to introduce a second auxiliary loss that is not shown in L_{aux} (line 153). Is this another term in L_{aux}? Was the backward RNN pretrained on this objective? More details should be included about this. Otherwise, this paper should be rejected as there is plenty of space left to include the details.
Line 186: How was the \beta term introduced? What scheduled? Did detail this prove essential? No answers to these questions are provided
Line 191: What type of features were used? Raw signal? Spectral? Mel-spectral? Cepstral? Please specify.
Line 209: Why the change from LSTM to GRU? Was this necessary for obtaining good results? Were experiments run with LSTM?
Line 163: Gradient disconnection here and on paragraph starting on line 127. Can they be shown graphically on Figure 1? Maybe an x on the relevant lines?
Figure 2 has very low quality on a print out. Should be improved.
Line 212: "architecturaly flat" do the refer to number of layers? Do any of the baseline models include information about the 2D structure of the MNIST examples? specify if any does as this would be advantageous for those models.
The section that refers to IWAE: Were the models trained with IWAE or just evaluated? I think it was just evaluated, but it should be more clearly specified. Would it be possible to train using IWAE?


Nonparametric Online Regression while Learning the Metric
We study algorithms for online nonparametric regression that learn the directions along which the regression function is smoother. Our algorithm learns the Mahalanobis metric based on the gradient outer product matrix G of the regression function (automatically adapting to the effective rank of this matrix), while simultaneously bounding the regret -on the same data sequence-in terms of the spectrum of G. As a preliminary step in our analysis, we extend a nonparametric online learning algorithm by Hazan and Megiddo enabling it to compete against functions whose Lipschitzness is measured with respect to an arbitrary Mahalanobis metric.

This paper describes a novel algorithm for online nonparametric regression problem. It employs Mahalanobis metric to obtain a better distance measurement in the traditional online nonparametric regression framework. In terms of theoretical analysis, the proposed algorithm improves the regret bound and achieves a competitive result to the state-of-the-art. The theoretical proof is well organized and correct to the reviewer.
However, the novelty of the proposed algorithm may be limited. The overall algorithm consists two separable components: online nonparametric regression with Mahalanobis metric, the estimation of gradient outer-product.
The first component is a straightforward extension from [7], which proposed a general framework including all kinds of distance metric spaces. In Algorithm 1, it is obvious that the proposed work specializes the metric space as Mahalanobis distance and keep other steps unchanged. Therefore, the statement that the proposed algorithm is a generalized version of the algorithm from [7] in line 244 is improper. This extension is valuable only when the motivation of incorporating Mahalanobis distance is clearly introduced and convincing. Otherwise, this work mainly discusses a special case in the framework of [7]. On the other hand, the significance of the proposed method and the related regret bound should be discussed with more details. Because compared with the original method in [7], the improvement of regret bound depends on the specific problem.
The second component is an application from [14] to estimate the gradient outer-product. The authors should discuss the increased computational complexity brought by this phase.
Some minor issues:
1.	How does one compute the active center efficiently in line 8 of Algorithm 1
2.	$\tilde{\rho}$ in Theorem 2 is not defined.
3.	What's the relationship between smoother f_0 and Mahalanobis metric?
4.	There is no experimental evaluation to demonstrate the improvement of the proposed method in comparison with the original framework in terms of accuracy or convergence rate.


Dykstra's Algorithm, ADMM, and Coordinate Descent: Connections, Insights, and Extensions
We study connections between Dykstra's algorithm for projecting onto an intersection of convex sets, the augmented Lagrangian method of multipliers or ADMM, and block coordinate descent. We prove that coordinate descent for a regularized regression problem, in which the penalty is a separable sum of support functions, is exactly equivalent to Dykstra's algorithm applied to the dual problem. ADMM on the dual problem is also seen to be equivalent, in the special case of two sets, with one being a linear subspace. These connections, aside from being interesting in their own right, suggest new ways of analyzing and extending coordinate descent. For example, from existing convergence theory on Dykstra's algorithm over polyhedra, we discern that coordinate descent for the lasso problem converges at an (asymptotically) linear rate. We also develop two parallel versions of coordinate descent, based on the Dykstra and ADMM connections.

This paper studies connections/equivalences between Dykstra's algorithm, coordinate descent (à la Gauss-Seidel block alternating minimization). Many of these equivalences were already known
      (indeed essentially in the optimization literature as stated by the author). The authors investigates this for the soft feasibility (best approximation) problem and block-seperable regularized regression,
      where the regularizers are positively homogeneous (i.e. supports of closed convex sets containing the origin).
      The author claims that this is the first work to investigate this for the casen where the design is not unitary.
      Actually, the extension to arbitrary design is very straightforward (almost trivial) through Fenchel-Rockafellar duality.
      In fact, positive homogeneity is not even needed as Dykstra's algorithm has been extended to tje proximal setting (beyond indicator functions of closed convex sets).
      Having said this, I have several concerns on this paper.
      + My first concern pertains to the actual novelty of the work, which I beleive is quite limited. The manuscript has  a flavour of a review paper with some incremental extensions.
      + The paper contains some important inaccuracies that should be addressed.
      Other detailed comments are as follows:
      + (1): intersection of sets should be non-empty. Otherwise many of the statements made do not rigorously hold.
      + What the author is calling seminorms are actually support functions (symmetry is NOT needed), or equivalently gauges of polar sets.
      + Extension to the non-euclidean case: several key details are missing for all this to make sense. In particular the notion of a Legendre function. Qualification conditions for strong duality to hold are not stated either.


Differentially Private Empirical Risk Minimization Revisited: Faster and More General *
In this paper we study the differentially private Empirical Risk Minimization (ERM) problem in different settings. For smooth (strongly) convex loss function with or without (non)-smooth regularization, we give algorithms that achieve either optimal or near optimal utility bounds with less gradient complexity compared with previous work. For ERM with smooth convex loss function in high-dimensional (p n) setting, we give an algorithm which achieves the upper bound with less gradient complexity than previous ones. At last, we generalize the expected excess empirical risk from convex loss functions to non-convex ones satisfying the PolyakLojasiewicz condition and give a tighter upper bound on the utility than the one in [34].

Summary:
A large number of machine learning models are trained on potentially sensitive
data, and it is often import to guarantee privacy of the training data.
Chaudhuri and Monteleoni formulated the differentially private ERM problem and
started a line of work on designing differentially private optimization
algorithms for variants of ERM problems. Recent works have gotten nearly optimal
tradeoffs between the additional error introduced by the DP algorithm (the
privacy risk) and the privacy parameter, for a large class of settings. In this
work, these results are improved in the additional axis of computational
efficiency. For smooth and strongly convex losses, this work gets  privacy risk
bounds that are essentially the best known, but do so at a computational cost
that is essentially (n+ \kappa) gradient computaitons, instead of n\kappa, where
\kappa is the condition number. Similar improvements are presented for other
settings of interest, when the loss function is not strongly convex, or when the
constraint set has small complexity.
A different viewpoint on the results is that the authors show that DP noise
addition techniques and modern optimization methods can be made to work well
together. Speficially, one can use SVRG with noise addition at each step and the
authors show that this noisy SVRG also gets near optimal privacy risk. Similarly
for the case of constraint sets with small Gaussian width (such as l_1), where
previous work used noisy mirror descent, the authors show that one can use an
accelerated noisy mirror descent and get faster runtimes without paying in the
privacy cost.
I think the problem is very important and interesting. While the tools are somewhat
standard, I think this paper advances the state of the art sufficiently that I
am compelled to recommend acceptance.


Structured Embedding Models for Grouped Data
Word embeddings are a powerful approach for analyzing language, and exponential family embeddings (EFE) extend them to other types of data. Here we develop structured exponential family embeddings (S-EFE), a method for discovering embeddings that vary across related groups of data. We study how the word usage of U.S. Congressional speeches varies across states and party affiliation, how words are used differently across sections of the ArXiv, and how the copurchase patterns of groceries can vary across seasons. Key to the success of our method is that the groups share statistical information. We develop two sharing strategies: hierarchical modeling and amortization. We demonstrate the benefits of this approach in empirical studies of speeches, abstracts, and shopping baskets. We show how S-EFE enables group-specific interpretation of word usage, and outperforms EFE in predicting held-out data.

This paper presents a word embedding model for grouped data. It extends EFE to learn group-specific embedding vectors, while sharing the same context vector. To handle groups limited data, the authors propose two methods (hierarchical and amortization) to derive group-specific embedding vectors from a shared one. The paper is clearly written, but the novelty is a bit limited since it is an incremental work beyond EFE.
1. From Table 2, there is no clear winner among the three proposed models (hierarchical, amortiz+feedf, and amortiz+resnet), and the performance differences are subtle especially on the shopping data. If one would like to use S-EFE, do you have any practical guidance on choosing the right model? I guess we should prefer amortiz+resnet to amortiz+feedf, since amortiz+resnet always outperforms amortiz+feedf. Line 276 mentions that hierarchical S-EFE works better when there are more groups. Why?
2. Why Separate EFE performs worse than Global EFE?
3. The authors proposed hierarchical and amortization methods for the reasons in lines 128-131. It is interesting to see how S-EFE performs w.r.t. various data sizes. From this experiment, we might understand if S-EFE is going to surpass hierarchical/amortiz S-EFE, provided more and more data. If yes, would it be better to promote S-EFE as the leading method? If not, why?
4. I understand that pseudo log likelihood is a standard metric for evaluating embedding methods. However, it would be great to see how embedding helps in standard language understanding tasks, like text classification, machine translation, etc.
5. lines 141 and 143: Eq. 9 to Eq. 4


Learned D-AMP: Principled Neural Network Based Compressive Image Recovery
Compressive image recovery is a challenging problem that requires fast and accurate algorithms. Recently, neural networks have been applied to this problem with promising results. By exploiting massively parallel GPU processing architectures and oodles of training data, they can run orders of magnitude faster than existing techniques. However, these methods are largely unprincipled black boxes that are difficult to train and often-times specific to a single measurement matrix.
It was recently demonstrated that iterative sparse-signal-recovery algorithms can be "unrolled" to form interpretable deep networks. Taking inspiration from this work, we develop a novel neural network architecture that mimics the behavior of the denoising-based approximate message passing (D-AMP) algorithm. We call this new network Learned D-AMP (LDAMP).
The LDAMP network is easy to train, can be applied to a variety of different measurement matrices, and comes with a state-evolution heuristic that accurately predicts its performance. Most importantly, it outperforms the state-of-the-art BM3D-AMP and NLR-CS algorithms in terms of both accuracy and run time. At high resolutions, and when used with sensing matrices that have fast implementations, LDAMP runs over 50× faster than BM3D-AMP and hundreds of times faster than NLR-CS.

Summary of the work:
The authors present a theoretically motivated NN architecture for solving inverse problems arising in compressed sensing. The network architecture arises from ‘unrolling’ the denoiser-based approximate message passing (D-AMP) algorithm with convolutional denoiser networks inside. The work is nicely presented, and to my knowledge, original. The authors present competitive results on known benchmark data.
Overall evaluation:
I enjoyed reading this paper, well presented, theoretically motivated work. I was pleased to see that the method achieves very competitive performance in the end both in terms of accuracy and in terms of speed.
Comments and questions:
1. Why is the Onsager correction so important?: The Onsager correction is needed because the error distribution differs from the white Gaussian noise assumption typically used to train/derive denoising methods. However, if the whole D-IT unrolled network is trained (or only fine-tuned after initial training) end-to-end, then surely, subsequent denoisers can adopt to the non-Gaussian noise distribution and learn to compensate for the bias themselves. This would mean that the denoisers would no longer be good denoisers when taken out of context, but as part of the network they could co-adapt to compensate for each other’s biases. If this is not the case, why?
2. Validation experiments for layer-wise-training: My understanding is that all experiments use the layer-wise training procedure, and there is no explicit comparison between layer-wise or end-to-end training. Is this correct?
3. Connections to the theory denoising autoencoders: This paper is presented from the perspective of compressive sensing, with solid theory, and unsurprisingly most of the references are to work from this community/domain. However, denoisers are applied more generally in unsupervised learning, and there is a relevant body that the NIPS community will be familiar with which might be worth discussing in this paper. Specifically, (Alain and Bengio, 2012) https://arxiv.org/abs/1211.4246 show that neural networks trained for denoising learn approximate the gradients of the log data generating density. Therefore, one could implement iterative denoising as “taking a gradient step” along the image prior. This is exploited in e.g. (Sonderby et al, 2017) https://arxiv.org/abs/1610.04490 where the connection to AMP is noted - although not discussed in detail.
4. Lipschitz continuity remark: the authors say ConvNets are clearly Lipschitz, otherwise their gradients would explode during training. While this is true, and I would assume doesn’t cause any problems in practice, technically, the Lipschitz constant can change with training, and is unbounded. A similar Lipschitz-continuity requirement arises in - for example - the Wasserstein GAN’s discriminator network, where Lipschitz continuity with a certain constant is ensured by weight clipping. Can the authors comment on the effect of the Lipschitz constant’s magnitude on the findings?
Minor comments:
Table 1. would benefit from highlighting the fastest/best methods (and those that come within one confidence interval from the winner)


Protein Interface Prediction using Graph Convolutional Networks
We consider the prediction of interfaces between proteins, a challenging problem with important applications in drug discovery and design, and examine the performance of existing and newly proposed spatial graph convolution operators for this task. By performing convolution over a local neighborhood of a node of interest, we are able to stack multiple layers of convolution and learn effective latent representations that integrate information across the graph that represent the three dimensional structure of a protein of interest. An architecture that combines the learned features across pairs of proteins is then used to classify pairs of amino acid residues as part of an interface or not. In our experiments, several graph convolution operators yielded accuracy that is better than the state-of-the-art SVM method in this task.

The authors present a novel method for performing convolutions over graphs, which they apply to predict protein interfaces, showing clear improvements over existing methods. The methodological contribution is strong, and the paper mostly clearly written. However, the authors should evaluate compute costs and more alternative methods for performing graph convolutions.
Major comments
=============
The authors highlight that their method differs from related methods by being designed for pairs (or collections) of graphs (l104). However, the two protein graphs a convolved separately with shared weights, and the resulting node features merged afterward. Any other method can be used to convolve proteins before the resulting features are merged. The authors should clarify or not highlight this difference between their and existing methods.
The authors should compare their method to a second graph convolutional network apart from DCNN, e.g. Schlichtkrull et al or Duvenaud et al.
The authors should clarify if they used the features described in section 3.1.1 as input to all methods. For a fair comparison, all methods should be trained with the same input features.
The authors should compare the memory usage and runtime of their method to other convolutional methods. Does the method scale to large proteins (e.g. >800 residues each) with over 800^2 possible residue-residue contacts? The authors should also briefly describe if computations can be parallelized on GPUs and their method be implemented as a user friendly ‘graph convolutional layer’.
The authors should describe more formally (using equations) how the resulting feature vectors are merged (section 2.3). The authors should also clarify how they are dealing with variable-length proteins which result in a variable number of feature vectors. Are the merged feature vectors processed independently by a fully connected layer with shared weights? Or are feature vectors concatenated and flattened, such that the fully connected layer can model interactions between feature vectors as suggested by figure 2? The authors should also clarify if the same output layer is applied independently to feature vectors or jointly.
Section 3.2: The authors should describe more clearly which hyper-parameters were optimized, both for GCN, PAIRpred, and DCNN. For a fair comparison, the most important hyper-parameters of all methods must be optimized.
l221-225: The authors used the AUC for evaluation. Since labels are highly unbalanced, the authors should also compare and present the area under precision-recall curve. The authors should also describe if performance metrics were computed per protein complex as suggested by figure 3, or across complexes.
Minor comments
=============
l28-32: This section should be toned down since the authors use some of the mentioned ‘hand-crafted’ features as input to their own model.
l90: The authors should clarify what they mean by ‘populations’.
l92-99: The authors should mention that a model without downsampling results in higher memory and compute costs.
l95: typo ‘classiy’
Table 1: The authors should clarify that ‘positive examples’ and ‘negative examples’ are residues that are or are not in contact, respectively.
l155-162: The authors should mention the average protein length, which is important to assess compute costs (see comment above).
l180: The authors should clarify if the ‘Gaussian function’ corresponds to the PDF or CDF of a normal distribution.
Table 2: The authors should more clearly describe in the main text how the ‘No Convolutional’ model works. Does it consist of 1-4 fully connected layers that are applied to each node independently, and are the resulting feature vectors merged in the same way as in GCN? If so, which activation function was used and how many hidden units? Is it equivalent to GCN with a receptive field of 0?
Since the difference between the mean and median AUC is not clear by looking at figure 3, the authors should plot the mean and median as vertical lines. Since the figure is not very informative, I suggest to move it to the appendix and to show instead more protein complexes as in figure 4.
l192: Did the authors both downsample negative pairs (caption table 1) and give 10x higher weight to positive pairs? If so, it should be pointed out in the text that two techniques were used to account for class-imbalance.
l229: What are ‘trials’? Did the authors use different train/test split, or did they train models multiple times to account for the randomness during training?


Process-constrained batch Bayesian Optimisation
Prevailing batch Bayesian optimisation methods allow all control variables to be freely altered at each iteration. Real-world experiments, however, often have physical limitations making it time-consuming to alter all settings for each recommendation in a batch. This gives rise to a unique problem in BO: in a recommended batch, a set of variables that are expensive to experimentally change need to be fixed, while the remaining control variables can be varied. We formulate this as a process-constrained batch Bayesian optimisation problem. We propose two algorithms, pc-BO(basic) and pc-BO(nested). pc-BO(basic) is simpler but lacks convergence guarantee. In contrast pc-BO(nested) is slightly more complex, but admits convergence analysis. We show that the regret of pc-BO(nested) is sublinear. We demonstrate the performance of both pc-BO(basic) and pc-BO(nested) by optimising benchmark test functions, tuning hyper-parameters of the SVM classifier, optimising the heat-treatment process for an Al-Sc alloy to achieve target hardness, and optimising the short polymer fibre production process.

The paper presents a bayesian optimization framework in which samples can be collected in similarly constrained batches. In short, rather than selecting a single point to sample a label from (as in usual bayesian optimization) or selecting multiple points simultaneously (as in batch bayesian optimization), multiple samples in a batch can be collected in which one or more variables need to remain constant throughout the batch. This is motivated from experiments in which some parameter (e.g., temperature in an oven, geometry of an instrument) can be designed prior to sampling, and be reused in multiple experiments in a batch, but cannot be altered modified during a batch experiment.
The authors (a) propose this problem for the first time (b) propose two algorithms for solving it, (b) provide bounds on the regret for the most sophisticated of the two algorithms, and (c) present a comparison of the two algorithms on a variety of datasets. One experiment actually involves a metallurgical setup, in which the authors collaborated with researchers in their institution to design a metal alloy, an interdisciplinary effort that is commendable and should be encouraged.
My only concern is that the paper is somewhat dense. It is unlikely that someone unfamiliar with Bayesian optimization would be able to parse the paper. Even knowing about bayesian optimization, notation is not intoduced clearly and the algorithms are hard to follow. In particular:
-what is sigma in Algorithms 1, and 2. Is it the variance of the GP used in the UCB?
-The activation fuction alpha^GB-UCB should be clearly and explicitly defined somewhere.
-The description of the inner loop and the outer loop in Algorithm 1 are not consistent with the exposition in the text: h and g are nowhere to be found in Algorithm 1. This seems to be because your replace the optimization with the GP-UCB-PE. It would be better if the link is made explicit.
-There is absolutely no description of how any optimization involving an activation function is performed. Do you use sampling? Gradient descent? Some combination of both?   These should be described, especially in how they where instantiated in the experiments.
-Is alpha different from alpha^GP-UCB in Alg. 1? how?
-In both algorithms 1 and 2, it is not clear how previously selected samples in a batch (k'\< k) are used to select the k-th sample in a batch, other than that they share a constrained variable.

All of these are central to understanding the algorithm. This is a pity, as the exposition is otherwise quite lucid.


Learning with Bandit Feedback in Potential Games
This paper examines the equilibrium convergence properties of no-regret learning with exponential weights in potential games. To establish convergence with minimal information requirements on the players' side, we focus on two frameworks: the semi-bandit case (where players have access to a noisy estimate of their payoff vectors, including strategies they did not play), and the bandit case (where players are only able to observe their in-game, realized payoffs). In the semi-bandit case, we show that the induced sequence of play converges almost surely to a Nash equilibrium at a quasi-exponential rate. In the bandit case, the same result holds for ε-approximations of Nash equilibria if we introduce an exploration factor ε > 0 that guarantees that action choice probabilities never fall below ε. In particular, if the algorithm is run with a suitably decreasing exploration factor, the sequence of play converges to a bona fide Nash equilibrium with probability 1.

The authors show that exponential weight updates and its generalization FTRL, converges pointwise to a Nash equilibrium in potential games. The latter holds even when players receive bandit feedback of the game.
Unfortunately, the main result, for the case of full feedback is already known in the economics literature, where Hedge is known as smooth fictitious play. It is presented in the paper: Hofbauer, Sandholm, "On the global convergence of smooth fictitious play", Econometrica 2002. This work is not even cited in the paper and it shows that smooth fictitious play and its extensions of FTRL (though they do not call it FTRL) does converge to Nash in potential games and in three other classes of games.
Moreover, the result of Kleinberg, Piliouras, Tardos:"Multiplicative Updates Outperform Generic No-Regret Learning in Congestion Games", STOC'09, is presented in the paper, for the actual sequence of play (unlike what the authors claim). So that paper shows that for the class of congestion games (which is equivalent to the class of all potential games (c.f. Monderer-Shapley)), the actual play of MWU converges to Nash equilibria and in fact almost surely to Pure Nash equilibria (except for measure zero game instances), which is an even stronger result.
In light of the above two omissions, the paper needs to be re-positioned to argue that the extension to bandit feedback is not an easy extension of the above results. It is an interesting extension and I can potentially see that extension be presented at a NIPS quality conference, but it needs to be positioned correctly with the literature and the paper needs to argue as to why it does not easily follow from existing results. In particular, given that most of these results go through stochastic approximations, such stochastic approximations are most probably robust to unbiased estimates of the payoffs, which is all that is needed for the bandit extension.


Robust Conditional Probabilities
Conditional probabilities are a core concept in machine learning. For example, optimal prediction of a label Y given an input X corresponds to maximizing the conditional probability of Y given X. A common approach to inference tasks is learning a model of conditional probabilities. However, these models are often based on strong assumptions (e.g., log-linear models), and hence their estimate of conditional probabilities is not robust and is highly dependent on the validity of their assumptions. Here we propose a framework for reasoning about conditional probabilities without assuming anything about the underlying distributions, except knowledge of their second order marginals, which can be estimated from data. We show how this setting leads to guaranteed bounds on conditional probabilities, which can be calculated efficiently in a variety of settings, including structured-prediction. Finally, we apply them to semi-supervised deep learning, obtaining results competitive with variational autoencoders.

This paper studies the problem of computing probability bounds, more specifically bounds over probability of atoms of the joint space and conditional probabilities of the class, under the assumption that only some pairwise marginal as well as some univariate marginal values are known. The idea is that such marginals may be easier to obtain than fully specified probabilities, and that cautious inferences can then be used to produce predictions.
It is shown that when the marginals follow a tree structure (results are extended to a few other structures), then the problem can actually be solved in closed, analytical form, relating it to cover set and maximum flow problems. Some experiments performed on neural networks  show that this simple method is actually competitive with other more complex approaches (Ladder, VAE), while outperforming methods of comparable complexity.
The paper is elegantly written, with quite understandable and significant results. I see no reason to not accept it. Authors may be interested in looking at the following papers coming from the imprecise probability literature (since they deal with partially specified probabilities, this may be related):
* Benavoli, A., Facchini, A., Piga, D., & Zaffalon, M. (2017). SOS for bounded rationality. arXiv preprint arXiv:1705.02663.
* Miranda, E., De Cooman, G., & Quaeghebeur, E. (2007). The Hausdorff moment problem under finite additivity. Journal of Theoretical Probability, 20(3), 663-693.
Typos:
* L112:  solved Our —> solved. Our
* L156: assume are —> assume we are
* References: 28/29 are redundant


Online Convex Optimization with Stochastic Constraints
This paper considers online convex optimization (OCO) with stochastic constraints, which generalizes Zinkevich's OCO over a known simple fixed set by introducing multiple stochastic functional constraints that are i.i.d. generated at each round and are disclosed to the decision maker only after the decision is made. This formulation arises naturally when decisions are restricted by stochastic environments or deterministic environments with noisy observations. It also includes many important problems as special case, such as OCO with long term constraints, stochastic constrained convex optimization, and deterministic constrained convex optimization. To solve this problem, this paper proposes a new algorithm that achieves O( p T ) expected regret and constraint violations and O( p T log(T )) high probability regret and constraint violations. Experiments on a real-world data center scheduling problem further verify the performance of the new algorithm.

***Summary***
The paper proposes an algorithm to address a mixed online/stochastic setting where the objective function is a sequence of arbitrary convex functions (with bounded subgradients) under stochastic convex constraints (also with bounded subgradients).
A static regret analysis (both in expectation and high-probability) is carried out. A real-world experiment about the allocation of jobs across servers (so as to minimize electricity cost) illustrates the benefit of the proposed method.
Assessment:
+ Overall good & clear structure and presentation
+ Technically sound and appears as novel
+ Good regret guarantees
- Confusing connection with "Deterministic constrained convex optimization"
- Some confusion with respect to unreferenced previous work [A]
- More details would be needed in the experimental section
More details can be found in the next paragraph.
***Further comments***
-In Sec. 2.2, the rationale of the algorithm is described. In particular, x(t+1) is chosen to minimize the "drift-plus-penalty" expression. Could some intuition be given to explain how the importance of the drift and penalty are traded off (e.g., why is a simple sum, without a tuning weight, sufficient?).
-Lemma 5: I do not really have the expertise to assess whether Lemma 5 corresponds to "a new drift analysis lemma for stochastic process". In particular, I am not sufficiently versed in the stochastic process literature.
-Could some improvements be obtained by having \alpha and V dependent on t?
-For Lemma 9, it seems to me that the authors could reuse the results from Proposition 34 in [B]
-Comparison with "OCO with long term constraints": It appears that [A] (not referenced) already provide O(sqrt(T)) and O(sqrt(T)) guarantees, using similar algorithmic technique. This related work should be discussed.
-Comparison with "Stochastic constrained convex optimization": Is [16] the only relevant reference here?
-Confusion comparison with "Deterministic constrained convex optimization": In the deterministic constrained setting, we would expected the optimization algorithm to output a feasible solution; why is it acceptable (and why does it make sense) to have a non-feasible solution here? (i.e., constraint violation in O(1/sqrt(T)))
-Experiments:
   - More details about the baselines and the implementation (to make things reproducible, e.g., starting points) should appear in the core article
   - Is the set \mathcal{X_0} = [x_1^min, x_1^max] \times ... \times [x_100^min, x_100^max]? If this is the case, it means that problem (2) has to be solved with box constraints. More details would be in order.
   - Log-scale for unserved jobs (Figure d) may be clearer
   - Bigger figures would improve the clarity as well
   - An assessment of the variability of the results is missing to decide on the significance of the conclusions (e.g., repetitions to display standard errors and means).
   - To gain some space for the experiments, Sec. 4 could be reduced and further relegated to the supplementary material.
-Could the analysis extended to the dynamic regret setting?
***Minor***
-line 52: typo, min -> argmin. I may have missed them in the paper, but under which assumptions does the argmin of the problem reduce to the single x^*?
-line 204 (a): isn't it an inequality instead of an equality?
[A] Yu, H. & Neely, M. J. A Low Complexity Algorithm with O(sqrt(T)) Regret and Constraint Violations for Online Convex Optimization with Long Term Constraints preprint arXiv:1604.02218, 2016
[B] Tao, T.; Vu, V. Random matrices: universality of local spectral statistics of non-Hermitian matrices The Annals of Probability, 2015, 43, 782-874
==================
post-rebuttal comments
==================
I thank the authors for their rebuttal (discussion about [A] was ignored). I have gone through the other reviews and I maintain my score. I would like to emphasize though, that the authors should clarify the discussion about the stochastic/deterministic constrained convex opt. case (as answered in the rebuttal).


Safe Model-based Reinforcement Learning with Stability Guarantees
Reinforcement learning is a powerful paradigm for learning optimal policies from experimental data. However, to find optimal policies, most reinforcement learning algorithms explore all possible actions, which may be harmful for real-world systems. As a consequence, learning algorithms are rarely applied on safety-critical systems in the real world. In this paper, we present a learning algorithm that explicitly considers safety, defined in terms of stability guarantees. Specifically, we extend control-theoretic results on Lyapunov stability verification and show how to use statistical models of the dynamics to obtain high-performance control policies with provable stability certificates. Moreover, under additional regularity assumptions in terms of a Gaussian process prior, we prove that one can effectively and safely collect data in order to learn about the dynamics and thus both improve control performance and expand the safe region of the state space. In our experiments, we show how the resulting algorithm can safely optimize a neural network policy on a simulated inverted pendulum, without the pendulum ever falling down.

This paper addresses the problem of safe exploration for reinforcement learning. Safety is defined as the learner never entering a state where it can't return to a low cost part of the state space. The proposed method learns a Gaussian Process model of the system dynamics and uses Lyapunov functions to determine if a state-action pair is recoverable from. Theoretical results are given for safety and exploration quality under idealized conditions. A practical method is introduced and an experiment shows it explores much of the recoverable state-space without entering an unrecoverable state.
Overall, this paper addresses an important problem for real world RL problems. Being able to explore without risking entering a hazardous state would greatly enhance the applicability of RL. This problem is challenging and this work makes a nice step towards a solution. I have several comments for improvement but overall I think the work will have a good impact on the RL community and is a good fit for the NIPS conference.
One weakness of the paper is that empirical validation is confined to a single domain. It is nice to see the method works well on this domain but it would have been good to have tried it a more challenging domain. I suspect there are scalability issues and I think a clear discussion of what is preventing application to larger problems would benefit people wishing to build on this work. For example, the authors somewhat brush the issue of the curse of dimensionality under the rug by saying the policy can always be computed offline. But the runtime could be exponential in dimension and it would be preferable to just acknowledge the difficulty.
The authors should also give a clear definition of safety and the goal of the work early on in the text. The definition is somewhat buried in lines 91-92 and it wasn't until I looked for this definition that I found it. Emphasizing this would make the goal of the work much clearer.
Similarly, this paper could really benefit from paragraphs that clearly define notation. As is, it takes a lot of searching backwards in the text to make sense of theoretical results. New notation is often buried in paragraphs which makes it hard to access.
Updated post author feedback:
While I still think this is a good paper and recommend acceptance, after reflecting on the lack of clarity in the theoretical results, I've revised my recommendation to just accept. Theorem 4 seems of critical importance but its hard to see that the result maps to the description given after it. Clarifying the theoretical results would make this a much better paper than it already is.
Minor comments:
How is the full region of attraction computed in Figure 2A?
Line 89: What do you mean by state divergence?
Line 148: This section title could have a name that describes what the theoretical results will be, e.g., theoretical results on exploration and policy improvement
Line 308: Could footnote this statement.
Line 320: "as to" -> to
Theorem 4: How is n* used?


Dynamic Importance Sampling for Anytime Bounds of the Partition Function
Computing the partition function is a key inference task in many graphical models. In this paper, we propose a dynamic importance sampling scheme that provides anytime finite-sample bounds for the partition function. Our algorithm balances the advantages of the three major inference strategies, heuristic search, variational bounds, and Monte Carlo methods, blending sampling with search to refine a variationally defined proposal. Our algorithm combines and generalizes recent work on anytime search [16] and probabilistic bounds [15] of the partition function. By using an intelligently chosen weighted average over the samples, we construct an unbiased estimator of the partition function with strong finite-sample confidence intervals that inherit both the rapid early improvement rate of sampling and the long-term benefits of an improved proposal from search. This gives significantly improved anytime behavior, and more flexible trade-offs between memory, time, and solution quality. We demonstrate the effectiveness of our approach empirically on real-world problem instances taken from recent UAI competitions.

The authors present a method for estimating the partition function that alternates between performing heuristic search and importance sampling.  The estimated value of the partition function is confidence bounded and improves with additional computation time.  Experimental evaluation is performed on problems from 2006 and 2008 UAI competitions by comparing confidence bounds of the proposed method against previous work that uses only sampling [15] or search [16].  The proposed method significantly outperforms sampling on certain problems and search on others, while maintaining performance roughly comparable to or better than either sampling or search across all problems.
The originality of the work is a bit limited, as it is a fairly straightforward (but novel as far as I know) combination of two recent papers, references [15] and [16].
To review past work, [15] proposes a method that combines importance sampling with variational optimization.  The authors chose a proposal distribution that is the solution to a variational optimization problem and show this choice can be interpreted as picking the proposal distribution that minimizes an upper bound on the value of importance weights.  The importance weight bound is then used to construct a (probabilistic, but with confidence intervals) bound on the estimate of the partition function.
[16] describes a search algorithm for computing deterministic bounds on the partition function.  Unlike deterministic bounds given by variational methods, the search algorithm proposed in [16] can continue to improve its estimate in an anytime manner, even when given limited memory resources.
This paper combines the methods of approaches of [15] and [16] by alternating between importance sampling and search.  Search is performed using a weighted mini-bucket variational heuristic.  The same mini-buckets are used to improve the proposal distribution over time, generating better samples.  The authors provide a method for weighting later samples more heavily as the proposal distribution improves.
The authors demonstrate that the proposed method improves estimation of the partition function over state of the art sampling and search based methods ([15] and [16]).  There is room for researchers to build on this paper in the future, particularly in the areas of weighting samples generated by a set of proposal distributions that improve with time and exploring when computation should be focused more on sampling or search.
The paper is clear and well organized.  Previous work the paper builds on is well cited.
Minor:
- Line 243 states that DIS remains nearly as fast as AOBFS in figures 3(d) and (f), but it appears that the lower bound of DIS converges roughly half an order of magnitude slower.  Line 233 states, "we let AOBFS access a lower bound heuristic for no cost."  Does the lower bound converge more slowly for DIS than AOBFS in these instances or is the lower bound shown for AOBFS unfair?
- Line 190, properites is a typo.
- Line 146 states that, "In [15], this property was used to give finite-sample bounds on Z which depended on the WMB bound, h_{\emptyset}^{+}."  It appears that in [15] the bound was only explicitly written in terms of Z_{trw}.  Mentioning Z_{trw} may make it slightly easier to follow the derivation from [15].


Graph Matching via Multiplicative Update Algorithm
As a fundamental problem in computer vision, graph matching problem can usually be formulated as a Quadratic Programming (QP) problem with doubly stochastic and discrete (integer) constraints. Since it is NP-hard, approximate algorithms are required. In this paper, we present a new algorithm, called Multiplicative Update Graph Matching (MPGM), that develops a multiplicative update technique to solve the QP matching problem. MPGM has three main benefits: (1) theoretically, MPGM solves the general QP problem with doubly stochastic constraint naturally whose convergence and KKT optimality are guaranteed. (2) Empirically, MPGM generally returns a sparse solution and thus can also incorporate the discrete constraint approximately. (3) It is efficient and simple to implement. Experimental results show the benefits of MPGM algorithm.

This paper presents a novel method (MPMG) to solve QP graph matching problem. The graph matching problem is formulated as argmax_x(x^TWx) where W encodes the pair-to-pair and node-to-node affinities and x is the desired permutation matrix (in vector form). The desired solution is a permutation encoding the optimal matching, also expressed as doubly stochastic (entries >= 0, rows/columns sum to 1) and discrete (entries in {0,1}). The standard approaches to solving the QP is to relax the NP-hard problem (relaxation can be for either the doubly-stochastic constraint or the discrete constraint). This proposed MPMG solution follows the Lagrange multiplier technique which moves the doubly-stochastic constraint into the error term. This proposed approach, along with proofs for convergence and KKT-optimality are a a novel contribution for this graph matching formulation. Experimentally, evidence is provided that shows the approach converges near a sparse/discrete solution even though the constraint is not explicitly modeled in the solution, which is the most promising feature of the method. However, there are some concerns regarding the experimental details which might jeopardize the strength of the empirical claims.
Other state of the art techniques do rely on iterative solutions, so the paper should mention the convergence criterion especially for experiments like figure 1 where the MPMG is initialized with RRWM, it would be useful to make sure that the convergence criterion for RRWM is appropriately selected for these problems.
Following up on the previous comment, it seems initialization with RRWM is used for FIgure 1 but initialization with SM for the subsequent experiments. Furthermore, regarding the experiments, how is performance time computed? Is this including time for the initialization procedure, or just the subsequent MPGM iterations after initialization?
Also, RRWM also requires an initialization. In [3] the basic outline of the algorithm uses uniform, so is it a fair comparison at all for MPMG that uses a established algorithm as initialization vs RRWM which uses uniform? If the details have been handled properly perhaps more explanation should be given in the text to clarify these points. In light of these points it is hard to take anything meaningful away from the experiments provided.
In the experiments, how are the node-to-node affinities determined (diagonals of W)? I think this is not described in the paper.
Image feature point matching would not actually be solved as a graph matching problem in any practical scenario (this is true for a number of reasons not limited to number of features, pair-to-pair affinities, robustness to large # of outliers). As the paper suggests, there are many other practical labeling problems which are typically solved using this QP formulation and perhaps the paper would have more value stronger


Discovering Potential Correlations via Hypercontractivity
Discovering a correlation from one variable to another variable is of fundamental scientific and practical interest. While existing correlation measures are suitable for discovering average correlation, they fail to discover hidden or potential correlations. To bridge this gap, (i) we postulate a set of natural axioms that we expect a measure of potential correlation to satisfy; (ii) we show that the rate of information bottleneck, i.e., the hypercontractivity coefficient, satisfies all the proposed axioms; (iii) we provide a novel estimator to estimate the hypercontractivity coefficient from samples; and (iv) we provide numerical experiments demonstrating that this proposed estimator discovers potential correlations among various indicators of WHO datasets, is robust in discovering gene interactions from gene expression time series data, and is statistically more powerful than the estimators for other correlation measures in binary hypothesis testing of canonical examples of potential correlations.

Summary:
   Finding measures of correlation among data covariates is an important problem. There have been several measures that have been proposed in the past - maximum correlation, distance correlation and maximum information coefficient. This paper introduced a new correlation measure by connecting dots among some recent work in information theory literature. The candidate in this paper is called hyper-contractivity coefficient or recently also is known as information bottleneck.
   Information bottleneck principle is this: Whats the best summary of variable X (most compressed) such that it also retains a lot of information about another variable Y. So for all summaries of X, U the maximum ratio of I(U;Y)/I(U;X) is the hyper contractivity coefficient.

  This quantity has another interpretation due to recent work. Consider the conditional p(y|x) and marginal p(x) associate with joint distribution of (X,Y). Consider a different distribution r(x) and the marginal r(y) due to the joint r(x) p(y|x). The hypercontractivity coefficient is the maximum ratio of Divergence between r(y) (induced distribution on y by the change distribution r(x)) and p(y) to the divergence between r(x) and p(x). In other words "whats the maximum change at output y that can be caused by a new distribution that looks similar to x in KL distance"
 Renyi has come up with several axioms for characterizing correlation measures. The authors show that if you make one of the axioms strict in a sense, and then drop the need for symmetry, then hypercontractivity satisfies all the axioms.
 To be less abstract, the goal of authors is that they like to have a measure which has high correlation value even in for most values of X there seems to be no influence on Y while in some rare region of the X space, correlation with Y is almost deterministic. This has practical applications as the authors demonstrate in the experiments. It turns out the hypercontractivity coefficient  has this property while all other previous measures do not have this property provably. This is also related to their modification of the axioms of Renyi by a stronger one (axiom 6 instead of 6').
the authors propose an asymptotically consistent estimator using the second definition of hyper contractivity. They define an optimization problem in terms of the ratio of the change distribution r(x) and original distribution p(x) which they solve by gradient descent.To come to this formulation they use existing results in kernel density estimation + clever tricks in importance sampling and the defn of hyper contractivity.
The authors justify the use of the measure by a very exhanustive set of experiments.
Strengths:
 a) The paper connects hyper contractivity ( which has recently acquired attention in information theory community) to a correlation measure that can actually pick out "hidden strong correlations under rare events" much better than the existing ones. Theoretically justifying it with appropriate formulation of axioms based on Renyi's axioms and clarifying relationships amongs various other correlation measures is pretty interesting and a strong aspect of this paper.
b) Authors apply this to discover existing known gene pathways from data and also discover potential correlation among WHO indicators which are difficult to detect. There is an exhaustive simulations with synthetic functions too.
I have the following concerns too:
a) Detecting influence from X to Y is sort of well studied in causality literature. In fact counterfactual scores called probability of sufficiency and probability of necessity have been proposed (Please refer "Causality" book by Judea Pearl 2009. Chapter 7 talks about scores for quantifying influence). So authors must refer to and discuss relationships to such counterfactual measures. Since these are counterfactual measures, they also do not measure just factual influence. Further , under some assumptions the above counterfactual measures can be computed from observational data too. Such key concepts must be referred to and discussed if not compared. I however admit that the current works focus is on correlation measures and not on influence in the strictest causal sense.
b) This comment is related to my previous comment : The authors seem to suggest that hypercontractivity suggest a direction of influence from X to Y ( this suggestion is sort of vague in the paper). I would like to point out two things : a) In experiments in the WHO dataset, some of the influences are not causal (for example case E) and some of them may be due to confounders (like F between arms exports and health expenditure). So this measure is about finding hidden potential correlations or influences if the causal direction if already known b) For gaussians, S(Y,X) and S(X,Y) are same - therefore directional differences depend on the functional relationships between X and Y. So the title of discovering potential influence may not be a suitable title given prior work on counterfactual influence scores? I agree however that it is discovering rare hidden correlations.
c) Does it make sense to talk about a conditional version ? If so where would those be useful ?
d) Section 4.3 - Authors say "X_t will cause ... Y_t to change at a later point in time and so on". However, the authors for the same t_i check for the measure s(X_{t_i},Y_{t_i}). So are we not measure instantaneous influence here. I agree that the instantaneous influences can come one after the other in a sequence of pairs of genes. However, this discrepancy needs to be clarified.
e) Why did the authors not use UMI or CMI estimators for experiments in Section 4.2 and 4.1 It seems like shannon capacity related estimators proposed in ref 26 does loosely have properties like that of axiom 6 ( roughly speaking). Actually I wonder why the authors did not include shannon capacity in the theoretical treatment with other measures (I understand that certain normalization criterions like being between 0 and 1 and 5 are not satisfied)?
f)Slightly serious concern: Does Axiom 6, for a given {\cal X,\cal Y} domain need to hold for "some" non constant function f or "every" non constant function f ?? Axiom 6' is for any function f ?? The reason I am asking is because the last expression in page 15 in the appendix has H(f(X) in the numerator and denominator. The authors say "H(f(X)) =infinity for a continuous random variable X and a nonconstant function f" - So clearly if this is the proof Axiom 6 is not for any f. Because f being a linear function and X being a uniform distribution on bounded support clearly does not satisfy that statement in the proof if it means any f.
g) Follow up of concerns in (f) : In page 14, proof of Proposition 1, point 6' - If Y=f(X) s(X;f(X)) =1 from Theorem 1" . That seems to rely on the above concern in (f). It seems like axiom 6 is not valid for any f. In that case, how is proposition 1 point 6' correct? I think that needs a separate proof for alpha=1 right ??
h) Regarding directionality: Can you characterize functions f, noise such that Y= f(X, noise) and S (Y,X) << S(X,Y). If this can be done, this could be used potentially for casual inference between pairs from just observational data on which there have been several works in the past few years. Have the authors considered applying this to that case?
i) With respect to the material on page 27 in the appendix: It seems like definition in (5) has an interventional interpretation. You are trying different soft interventions r(x) at x and you are checking for what soft intervention that is not too different from p(x) the channel output changes dramatically. This seems like a measure that sort of simulates an intervention assuming a causal direction to check the strength of causation. In this sense, mildly it is a counterfactual measure. Can this be use as a substitute for interventions in certain structural equation models (under some suitable assumptions on non-linearity noise etc). If authors have answers, I would like to hear it.


Optimal Sample Complexity of M -wise Data for Top-K Ranking
We explore the top-K rank aggregation problem in which one aims to recover a consistent ordering that focuses on top-K ranked items based on partially revealed preference information. We examine an M -wise comparison model that builds on the Plackett-Luce (PL) model where for each sample, M items are ranked according to their perceived utilities modeled as noisy observations of their underlying true utilities. As our result, we characterize the minimax optimality on the sample size for top-K ranking. The optimal sample size turns out to be inversely proportional to M . We devise an algorithm that effectively converts M -wise samples into pairwise ones and employs a spectral method using the refined data. In demonstrating its optimality, we develop a novel technique for deriving tight ∞ estimation error bounds, which is key to accurately analyzing the performance of top-K ranking algorithms, but has been challenging. Recent work relied on an additional maximum-likelihood estimation (MLE) stage merged with a spectral method to attain good estimates in ∞ error to achieve the limit for the pairwise model. In contrast, although it is valid in slightly restricted regimes, our result demonstrates a spectral method alone to be sufficient for the general M -wise model. We run numerical experiments using synthetic data and confirm that the optimal sample size decreases at the rate of 1/M . Moreover, running our algorithm on real-world data, we find that its applicability extends to settings that may not fit the PL model.

Summary:
    The authors consider the problem of optimal top-K rank aggregation from many samples each containing M-wise comparisons. The setup is as follows: There are n items and each has a hidden utility w_i such that w_i >= w_{i+1}. w_K-w_{K+1} has a large separation Delta_K. The task is to identify the top K items. There is a series of samples that are given. Each samples is a preference ordering among M items out of the n chosen. Given the M items in the sample, the preference ordering follows the PL model. Now there is a random hyper graph where each set of M vertices is connected independently with probability p. Further, every hyperedge gives L M-wise i.i.d preference order where M items come from the hyperedge.

     The authors show that under some density criterion on p, the necessary and sufficient number of samples nis  O( n log n/M Delta_K^2 ) . The notable point is that it is inversely proportional to M.
     The algorithm has two parts: For every M-wise ordered sample, one creates M pariwise orderings that respect the M-wise ordering such that these pairwise orderings are not correlated. This is called sample breaking. Then the problem uses standard algorithmic techniques like Rank Centrality that use the generated pairwise ordered samples to actually generate a top K ranking.
    The proofs involve justifying the sample breaking step and new analysis is needed to prove the sample complexity results.
 Strengths:
  a) For the dense hypergraph case, the work provides sample optimal strategy for aggregation which is the strength of the paper.
  b) Analyzing rank centrality along with path breaking seems to require a strong l_infinity bound on the entries of the stationary distribution of a Markov Chain whose transition matrix entries have been estimated upto to some additive error. Previous works have provided only l_2 error bounds. This key step helps them separate the top K items from the rest due to the separation assumed.
I did not have the time to go through the whole appendix. However I only quickly checked the proof of Theorem 3 which seems to be quite crucial. It seems to be correct.
Weaknesses:
      a) Spectral MLE could be used once sample breaking is employed and new pairwise ordered samples generated. Why has that not been compared with the proposed approach for the experiments in section 4.2 ? Infact that could have been compared even for synthetic experiments in Section 4.1 for M > 2 ?
      b) The current method's density requirements are not optimal (at least w.r.t the case for M=2). I was wondering if experimentally, sample breaking + some other algorithmic approach can be shown to be better ?


Robust Estimation of Neural Signals in Calcium Imaging
Calcium imaging is a prominent technology in neuroscience research which allows for simultaneous recording of large numbers of neurons in awake animals. Automated extraction of neurons and their temporal activity from imaging datasets is an important step in the path to producing neuroscience results. However, nearly all imaging datasets contain gross contaminating sources which could originate from the technology used, or the underlying biological tissue. Although past work has considered the effects of contamination under limited circumstances, there has not been a general framework treating contamination and its effects on the statistical estimation of calcium signals. In this work, we proceed in a new direction and propose to extract cells and their activity using robust statistical estimation. Using the theory of M-estimation, we derive a minimax optimal robust loss, and also find a simple and practical optimization routine for this loss with provably fast convergence. We use our proposed robust loss in a matrix factorization framework to extract the neurons and their temporal activity in calcium imaging datasets. We demonstrate the superiority of our robust estimation approach over existing methods on both simulated and real datasets.

This is an important and timely paper in the automated signal detection of calcium imaging of neurons.   The authors have developed a new methodology based on careful noise model and robust statistics.   The prevalence of calcium imaging experimental studies has increased interest in powerful analysis methods for understanding correlation of neuronal firing patterns, and this paper represents a strong advance in this direction.
The use of robust location estimator appears be a good approach, particularly given the magnitude of signal and noise variability and strong presence of ouliers in neuropil.  It is somewhat surprising that previous methods have not considered this.
The author's noise model as a superposition of positive sources and lower amplitude normal component is interesting and gives more statistical power in the analysis.  In equations (2) these components should be more explicitly labelled for readability.
The authors present a rigorously argument for the algorithms convergence rate which is fast.
The authors present a nice comparison of their EXTRACT algorithm with two other approaches in actual manually labelled microendoscopic single-photon imaging data and show superior performance.  This section of the paper could be developed a little more carefully with more explicitly stated performance statistics,  although Figure 4 is well done.
In summary this is a very strong paper on a timely and important topic.  The availability of a code and how it might be deployed would be helpful.


Deep Recurrent Neural Network-Based Identification of Precursor microRNAs
MicroRNAs (miRNAs) are small non-coding ribonucleic acids (RNAs) which play key roles in post-transcriptional gene regulation. Direct identification of mature miRNAs is infeasible due to their short lengths, and researchers instead aim at identifying precursor miRNAs (pre-miRNAs). Many of the known pre-miRNAs have distinctive stem-loop secondary structure, and structure-based filtering is usually the first step to predict the possibility of a given sequence being a pre-miRNA. To identify new pre-miRNAs that often have non-canonical structure, however, we need to consider additional features other than structure. To obtain such additional characteristics, existing computational methods rely on manual feature extraction, which inevitably limits the efficiency, robustness, and generalization of computational identification. To address the limitations of existing approaches, we propose a pre-miRNA identification method that incorporates (1) a deep recurrent neural network (RNN) for automated feature learning and classification, (2) multimodal architecture for seamless integration of prior knowledge (secondary structure), (3) an attention mechanism for improving long-term dependence modeling, and (4) an RNN-based class activation mapping for highlighting the learned representations that can contrast pre-miRNAs and non-pre-miRNAs. In our experiments with recent benchmarks, the proposed approach outperformed the compared state-of-the-art alternatives in terms of various performance metrics.

The paper presents an LSTM model with an attention mechanism for classifying whether an RNA molecule is a pre-microRNA from its sequence and secondary structure. Class weights are incorporated into log-loss to account for class imbalance in the datasets used. The proposed method is extensively evaluated against 5 other existing methods on 3 datasets, and is shown to outperform the existing methods in most cases. The paper then attempts to give some insight into the features that are important for achieving good performance. First, by showing that secondary structures are largely responsible, but sequence features give a small boost, and second, by interpreting the attention weights using an adapted version of class activation mapping (proposed in an earlier CVPR paper). Results using different architectures and hyperparameters are also shown.
The paper presents an interesting application of LSTMs to biological data, and has some novel elements for model interpretation. The main issue I have with this paper is in how the method is evaluated. If the goal is to find new pre-microRNAs, especially those that have noncanonical structures (as stated in the abstract) special care must be taken in the construction of the training and test sets to make sure the reported performance will be reflective of performance on this discovery task. This is why in ref [10] describing miRBoost, methods are evaluated based on ability to predict completely new pre-microRNAs that were not previously found in previous versions of miRBase (included as 'test' dataset in this paper). While holding out 10% of the dataset as a test set may seem to also be sufficient, it results in a far smaller evaluation set of positives (1/5 - 1/10) than using the 'test' dataset, as well as possibly allowing the models to learn structural features of newly discovered RNAs, some of which may have been randomly placed in the training set.
An even stricter evaluation would use "structurally nonhomologous training and test data sets", as proposed in Rivas et al. (PMID:22194308), that discusses the importance of doing so for properly evaluating methods on the related RNA secondary structure prediction task. If a model is able to perform well on this stricter evaluation, where test RNA sequences are structurally different from training sequences, one can then be more confident in the ability to discover structurally novel pre-microRNAs.
A related comment on the evaluation is that the AUROC metric is unsuitable for tasks where only performance on the positive class is important (as in this case) and the area under the precision-recall curve (AUPR), or even better, the area under the precision-recall-gain curve (AUPRG; Flach and Kull NIPS 2015) should be used.
The adaptation of the class activation mapping method to the LSTM model is interesting and the paper hints that it could indeed be useful in uncovering new biological insight. It would strengthen the section if features common to the negatively predicted examples could be discussed as well.
The attention mechanism seemed to make a big difference to the performance - it would be interesting if it was possible to dissect how/why this is occurring in the model (and on this task). Does the attention mechanism make a bigger difference for longer sequences? And is it possible to show some of the long range dependencies that the attention mechanism picks up?
Other questions for the authors:
- In section 4.3 were the chosen examples actually correctly classified as positives and negatives respectively?
- Line 175 - how were the dropout rates selected empirically? Using cross validation?
- In the discussion it is mentioned that additional data modes can be added as separate network branches. Why was this approach not taken for the structure data and instead an explicit encoding that simultaneously considers structure and sequence was used?
Typos:
Line 91: "long-term dependency" -> "long-term dependencies"
Table 1: "Positive exmaples" -> "Positive examples"
Table 2 caption: "under curve of receiver operation characteristic" -> "area under receiver operating characteristic curve"
Table 3: microPred (test), cross-species, 0.985 should be highlighted instead
Figure 4 caption: "RNN ouputs" -> "RNN outputs"
Line 237: "technqiue" -> "technique"
Reference 10 should be Tran et al. not Tempel et al.


Collaborative PAC Learning
We consider a collaborative PAC learning model, in which k players attempt to learn the same underlying concept. We ask how much more information is required to learn an accurate classifier for all players simultaneously. We refer to the ratio between the sample complexity of collaborative PAC learning and its non-collaborative (single-player) counterpart as the overhead. We design learning algorithms with O(ln(k)) and O(ln 2 (k)) overhead in the personalized and centralized variants our model. This gives an exponential improvement upon the naïve algorithm that does not share information among players. We complement our upper bounds with an Ω(ln(k)) overhead lower bound, showing that our results are tight up to a logarithmic factor.
A natural but naïve algorithm that forgoes collaboration between players can achieve our objective by taking, from each player distribution, a number of samples that is sufficient for learning the individual task, and then training a classifier over all samples. Such an algorithm uses k times as many samples as needed for learning an individual task -we say that this algorithm incurs O(k) overhead in sample complexity. By contrast, we are interested in algorithms that take advantage of the collaborative environment, learn k tasks by sharing information, and incur o(k) overhead in sample complexity.
We study two variants of the aforementioned model: personalized and centralized. In the personalized setting (as in the department store example), we allow the learning algorithm to return different functions for different players. That is, our goal is to return classifiers f 1 , . . . , f k that have error of at most on player distributions D 1 , . . . , D k , respectively. In the centralized setting (as in the hospital example), the learning algorithm is required to return a single classifier f that has an error of at most on all player distributions D 1 , . . . , D k . Our results provide upper and lower bounds on the sample complexity overhead required for learning in both settings.

**Edit** I have read the other reviews and rebuttal. I thank the authors for their feedback and clarification and my positive evaluation of this paper remains unchanged.
The authors give theoretical guarantees for collaborative learning with shared information under the PAC regime. They show that the sample complexity for (\epsilon,\delta)-PAC learning k classifiers for a shared problem (or a size k majority-voting ensemble) only needs to grow by a factor of approximately 1+log(k) (or 1+log^2(k)) rather than a factor of k as naive theory might predict.
The authors provide pseudocode for two algorithms treated by the theory. These were correct as far as I could see, though I didn't implement either.
The paper is mostly clearly written, and strikes a sensible balance between what is included here and what is in the supplementary material, and I enjoyed reading it. I checked the paper and sections A and B of the supplementary material quite carefully, and skimmed C and D. There were no obvious major errors I could find.
It would, of course, have been ideal to see experimental results corroborating the theory, but space is limited and the theory is the main contribution. Hopefully those will appear in a later extended version.
I have mostly minor comments:
Somewhere it should be highlighted explicitly that t is a placeholder for log_{8/7}(k) thus although t is indeed logarithmic in k it is also not *very* small compared to k unless k is large (few hundred say) and in this setting a very large k strikes me as unlikely. On the other hand some central steps of the proof involve union bounding over 2t terms, which is usually disastrous, but here the individual sample complexities for each classifier still only grow like log(2t/\delta), i.e. loglog(k) and you may like to highlight that as well.
I think it is also worth mentioning that the guarantees for the second algorithm depend on a large enough quorum of individual voters being right at the same time (w.h.p) thus the guarantee I think is for the whole ensemble and not for parts of it i.e. since we don't know which individual classifiers are right and which are not. In particular, in practice could early stopping or communications errors potentially invalidate the theoretical guarantees here?
Line 202 and elsewhere use the notation of line 200, i.e. n_{r-1,c} + \frac{1}{8}n_{r-1,c-1}, since it is much clearer.
Line 251 can you please clarify the notation? \mathcal{D}_{\orth} is a distribution over - what? - with relation to the \mathcal{D}_i? It seems like \mathcal{F}_k and \mathcal{D}_k have some dependency on one another?
Line 393 in supplementary should be \forall r < r' \in [t].
A comment - the theory for the second algorithm looks like it could be adapted to, say, a bagged ensemble with only very minor modifications. There is little in the way of good theory for ensemble classification, in particular when is an ensemble of "weak" classifiers better than a single "strong" classifier is not well understood.


Triple Generative Adversarial Nets
Generative Adversarial Nets (GANs) have shown promise in image generation and semi-supervised learning (SSL). However, existing GANs in SSL have two problems: (1) the generator and the discriminator (i.e. the classifier) may not be optimal at the same time; and (2) the generator cannot control the semantics of the generated samples. The problems essentially arise from the two-player formulation, where a single discriminator shares incompatible roles of identifying fake samples and predicting labels and it only estimates the data without considering the labels. To address the problems, we present triple generative adversarial net (Triple-GAN), which consists of three players-a generator, a discriminator and a classifier. The generator and the classifier characterize the conditional distributions between images and labels, and the discriminator solely focuses on identifying fake image-label pairs. We design compatible utilities to ensure that the distributions characterized by the classifier and the generator both converge to the data distribution. Our results on various datasets demonstrate that Triple-GAN as a unified model can simultaneously (1) achieve the state-of-the-art classification results among deep generative models, and (2) disentangle the classes and styles of the input and transfer smoothly in the data space via interpolation in the latent space class-conditionally.

Summary:
The paper presents a GAN-like architecture called Triple-GAN that, given partially labeled data, is designed to achieve simultaneously the following two goals: (1) Get a good generator that generates realistically-looking samples conditioned on class labels; (2) Get a good classifier, with smallest possible prediction error. The paper shows that other similar GAN-based approaches always implicitly privileged either (1) or (2), and or needed much more labeled data to train the classifier. By separating the discrimination task between true and fake data from the classification task, the paper outperforms the state-of-the-art, both in (1) and (2). In particular, the classifier achieves high accuracy with only very few labeled dataset, while the generator produces state-of-the-art images, even when conditioned on y labels.
Quality & Clarity:
The paper indeed identifies plausible reasons of failure/inefficiency of the other similar GAN-type methods (such as Improved-GAN).  The underlying idea behind Triple-GAN is very elegant and comes with both nice theoretical justifications and impressive experimental results.
However the overall clarity of exposition, the text-flow and the syntax can still be much improved. Concerning syntax: too many typos and too many grammatically incorrect sentences. Concerning the exposition: there are too many vague or even unclear sentences/paragraphs (see below); even the statements of some Corollaries are vague (Corollary 3.2.1 & 3.3.1). Some proofs are dismissed as evident or refer to results without precise citation. There are some long and unnecessary repetitions (f.ex. l.58-73 and l.120-129 which are essentially the same), while other points that would need more explanations are only mentioned with a few unclear sentences (l.215-220). This is in striking contrast with the content's quality of this paper. So before publication, there will be  some serious work to do on the overall presentation, exposition, and syntax!
Detailed comments:
l.13-14: the expression 'concentrating to a data distribution', which is used in several places in this paper, does not exist. Please use f.ex. something like 'converges to'.
l. 37-38: formulation of points (1) and (2) is too vague and not understandable at the first read. Please reformulate.
l. 46-49: good point, but formulation is cumbersome. Reformulate it more neatly. You could f.ex. use a 'on the one hand ... on the other ...' type of construction.
l. 52-56: Unclear. ('the marginal distribution of data' , 'the generator cannot leverage the missing labels inferred by the discriminator': what do you mean?). Please reformulate.
l.58-73: too many unnecessary overlap with l.120-129. (And text is clearer in l.120-129, I find...). Please mind repetitions. One only needs repetitions if one messed up the explanation in the first place...
l. 140&157: please format your equations more nicely.
l .171: proof of Lemma 3.1: last sentence needs an explanation or a reference.
l.174: Corollary 3.2.1: I find it clearer to say p(x) = p_g(x) = p_d(x) and p(y) = p_g(y) = p_d(y).
l.185: Corollary 3.3.1: What is a 'regularization on the distances btw ...' ? Define it. Without this definition, I can't say whether the proof is obvious or not. And in any way, please say one or two sentences why the statement is true, instead of dismissing it as obvious.
l.215-220: I don't see what the REINFORCE alrogrithm, which is used in reinforcement learning, has to do with your algorithm. And the whole paragraph is quite obscure to me...Please reformulate. (Maybe it needs to be a bit longer?).
Esperiments:
- please state clearly when you are comparing to numbers reported in other papers, and when you really re-run the algorithms yourself.
l.228: '10 000 samples' should be '10 000 test samples'
l.232: '(Results are averaged over 10 runs)' : don't put it in parenthesis: it's important! Also, remind this in the caption of Table 1.
l.254-263: how many labeled data do you use for each of the experiments reported here? Please specify!
Also, you did not mention that triple-GAN simply also has more parameters than an improved GAN, so it already has an advantage. Please mention it somewhere.
table 1: what are the numbers in parenthesis? 1 std? Calculated using the 10 runs?
l.271: 'the baseline repeats to generate strange samples' : reformulate.
Generally: there are quite a few articles ('the') and other small words missing in the text. A careful proof reading will be necessary before publication.
Answer to rebuttal:
We thank the reviewers for their answers and hope that they will incorporate all necessary clarifications asked by the reviewers. And please be precise: in your rebuttal, you speak of a distance over joint probability distributions, and then cite the KL-divergence as an example. But mathematically speaking, the KL-divergence is not a distance! So please, be precise in your mathematical statements!


Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix Multiplication
We consider a large-scale matrix multiplication problem where the computation is carried out using a distributed system with a master node and multiple worker nodes, where each worker can store parts of the input matrices. We propose a computation strategy that leverages ideas from coding theory to design intermediate computations at the worker nodes, in order to optimally deal with straggling workers. The proposed strategy, named as polynomial codes, achieves the optimum recovery threshold, defined as the minimum number of workers that the master needs to wait for in order to compute the output. This is the first code that achieves the optimal utilization of redundancy for tolerating stragglers or failures in distributed matrix multiplication. Furthermore, by leveraging the algebraic structure of polynomial codes, we can map the reconstruction problem of the final output to a polynomial interpolation problem, which can be solved efficiently. Polynomial codes provide order-wise improvement over the state of the art in terms of recovery threshold, and are also optimal in terms of several other metrics including computation latency and communication load. Moreover, we extend this code to distributed convolution and show its order-wise optimality.

This paper extends the coded distributed computing of matrix multiplication proposed in Lee et al, 2016. The idea is to distribute the matrices by pre-multiplying them with the generator matrices of Reed Solomon code. Since Reed-Solomon codes are MDS codes, any K out of N distributed components are sufficient to recover the desired matrix multiplication.
I am somewhat confused by this paper. Reed-Solomon codes ARE defined to be evaluations of polynomials over finite field (or any fields). So why the authors say that they define something new called polynomial code and then follow up by saying the decoding is just like Reed-Solomon code? Reed Solomon codes are MDS code (in fact, Lee et al also used Reed Solomon codes, they just called it MDS codes).
The key idea of using codes for both matrices in matrix-matrix multiplication has appeared recently in Lee et al, 2017. I think there is a major problem from an application point of view in this paper that was not there for a simple matrix-vector multiplication of Lee et al of 2016. In case of matrix-vector multiplication A times x of Lee et al, 2016, the matrix A was fixed and x was variable. The matrix A was coded and distributed to the workers. x was uncoded. So the encoding is done only one time. It does indeed speed up the distributed computing. On the other hand, in this paper both the matrices A and B that are to be multiplied are being coded up. As a result when two matrices are given to multiply, a central node has to do a lot of polynomial evaluations and then distribute the evaluations among workers. The overhead for the central node seem to be way more than the actual computing task. Notably, this has to be done every time; as opposed to the one-time-encoding of Lee et al, 2016, or Dutta et al, 2016.
 Also surprising is the lack of experimental results. Most of the related literature is interesting because of the experimental superior performance they exhibit in real systems. This paper does not make any attempt in that direction.  However since theoretically it is not that novel, and given the overhead in evaluating the polynomials, one really needs experimental validation of the methods.
========================
After the rebuttal:
I did not initially see the making of the product MDS as a big deal (i.e. the condition that no two terms in the expression (16) have same exponent of x is not that difficult to satisfy). But on second thought, the simplicity should not be deterrent, given that the previous papers kept this problem open.
There is one other point. What I understand is that each matrix is encoded using polynomials such that the product becomes codewords of Reed_Solomon codes. I do not necessarily agree with the term Polynomial code and the claim that a new class of code has been discovered.
Finally, it is not easy to approximate matrix multiplications over reals with finite field multiplication. I don't know why that is being seen as straight-forward quantization.