Multi-view Matrix Factorization for Linear Dynamical System Estimation
We consider maximum likelihood estimation of linear dynamical systems with generalized-linear observation models. Maximum likelihood is typically considered to be hard in this setting since latent states and transition parameters must be inferred jointly. Given that expectation-maximization does not scale and is prone to local minima, moment-matching approaches from the subspace identification literature have become standard, despite known statistical efficiency issues. In this paper, we instead reconsider likelihood maximization and develop an optimization based strategy for recovering the latent states and transition parameters. Key to the approach is a two-view reformulation of maximum likelihood estimation for linear dynamical systems that enables the use of global optimization algorithms for matrix factorization. We show that the proposed estimation strategy outperforms widely-used identification algorithms such as subspace identification methods, both in terms of accuracy and runtime.

This paper proposes an efficient maximum likelihood algorithm for parameter estimation in linear dynamical systems. The problem is reformulated as a two-view generative model with a shared latent factor, and approximated as a matrix factorization problem. The paper then proposes a novel proximal update. Experiments validate the effectiveness of the proposed method.
The paper realizes that maximum likelihood style algorithms have some merit over classical moment-matching algorithms in LDS, and wants to solve the efficiency problem of existing maximum likelihood algorithms. Then the paper proposes a theoretical guaranteed proximal update to solve the optimization problem.
However, I do not understand why the paper tell the story from a two-view aspect. In LDS, we can construct x_t from two ways. One is from \phi_t and the other is from \phi_{t-1}. Eq.4 is minimizing the reconstruction error for x_t constructed in both ways, with regularization on \Phi, C and A. This is a general framework, which is a reconstruction loss plus regularization, widely used in many classical machine learning algorithms. I do not see much novelty in the proposed learning objective Eq.4, and I cannot tell the merit of stating the old story from a two-view direction.
Lemma 1 proposes a method to transform the nonconvex problem into a convex one. However, I cannot see any benefit of transforming Eq.4 into a convex problem. The new learning objective Z in Lemma 1 is just a de-noised version of x_t. The original low-dimensional latent space estimation problem now becomes the problem directly estimating the de-noised high-rank observed state. Thus all merits from low-dimensional assumption are not available now. Since the proposed model is still non-convex, what is its merit compared to classical non-convex matrix factorization style algorithms in LDS?


Parallel Streaming Wasserstein Barycenters
Efficiently aggregating data from different sources is a challenging problem, particularly when samples from each source are distributed differently. These differences can be inherent to the inference task or present for other reasons: sensors in a sensor network may be placed far apart, affecting their individual measurements. Conversely, it is computationally advantageous to split Bayesian inference tasks across subsets of data, but data need not be identically distributed across subsets. One principled way to fuse probability distributions is via the lens of optimal transport: the Wasserstein barycenter is a single distribution that summarizes a collection of input measures while respecting their geometry. However, computing the barycenter scales poorly and requires discretization of all input distributions and the barycenter itself. Improving on this situation, we present a scalable, communication-efficient, parallel algorithm for computing the Wasserstein barycenter of arbitrary distributions. Our algorithm can operate directly on continuous input distributions and is optimized for streaming data. Our method is even robust to nonstationary input distributions and produces a barycenter estimate that tracks the input measures over time. The algorithm is semi-discrete, needing to discretize only the barycenter estimate. To the best of our knowledge, we also provide the first bounds on the quality of the approximate barycenter as the discretization becomes finer. Finally, we demonstrate the practical effectiveness of our method, both in tracking moving distributions on a sphere, as well as in a large-scale Bayesian inference task.
barycenter of the input measures [1], and should be thought of as an aggregation of the input measures which preserves their geometry. This particular aggregation enjoys many nice properties: in the earlier Bayesian inference example, aggregating subset posterior distributions via their Wasserstein barycenter yields guarantees on the original inference task [47].
If the measures µ j are discrete, their barycenter can be computed relatively efficiently via either a sparse linear program [2], or regularized projection-based methods [16,7,51,17]. However, 1. these techniques scale poorly with the support of the measures, and quickly become impractical as the support becomes large. 2. When the input measures are continuous, to the best of our knowledge the only option is to discretize them via sampling, but the rate of convergence to the true (continuous) barycenter is not well-understood. These two confounding factors make it difficult to utilize barycenters in scenarios like parallel Bayesian inference where the measures are continuous and a fine approximation is needed. These are the primary issues we work to address in this paper.
Given sample access to J potentially continuous distributions µ j , we propose a communicationefficient, parallel algorithm to estimate their barycenter. Our method can be parallelized to J worker machines, and the messages sent between machines are merely single integers. We require a discrete approximation only of the barycenter itself, making our algorithm semi-discrete, and our algorithm scales well to fine approximations (e.g. n ⇡ 10 6 ). In contrast to previous work, we provide guarantees on the quality of the approximation as n increases. These rates apply to the general setting in which the µ j 's are defined on manifolds, with applications to directional statistics [46]. Our algorithm is based on stochastic gradient descent as in [22] and hence is robust to gradual changes in the distributions: as the µ j 's change over time, we maintain a moving estimate of their barycenter, a task which is not possible using current methods without solving a large linear program in each iteration.
We emphasize that we aggregate the input distributions into a summary, the barycenter, which is itself a distribution. Instead of performing any single domain-specific task such as clustering or estimating an expectation, we can simply compute the barycenter of the inputs and process it later any arbitrary way. This generality coupled with the efficiency and parallelism of our algorithm yields immediate applications in fields from large scale Bayesian inference to e.g. streaming sensor fusion.
Contributions. 1. We give a communication-efficient and fully parallel algorithm for computing the barycenter of a collection of distributions. Although our algorithm is semi-discrete, we stress that the input measures can be continuous, and even nonstationary. 2. We give bounds on the quality of the recovered barycenter as our discretization becomes finer. These are the first such bounds we are aware of, and they apply to measures on arbitrary compact and connected manifolds. 3. We demonstrate the practical effectiveness of our method, both in tracking moving distributions on a sphere, as well as in a real large-scale Bayesian inference task.

Title: Parallel Streaming Wasserstein Barycenters
Comments:
- This paper presents a new method for performing low-communication parallel inference via computing the Wasserstein barycenter of a set of distributions. Unlike previous work, this method aims to reduce certain approximations incurred by discretization. Theoretically, this paper gives results involving the rate of the convergence of the barycenter distance. Empirically, this paper shows results on a synthetic task involving a Von Mises distribution and on a logistic regression task.
- I feel that the clarity of writing in this paper is not great. It would be better to clearly (and near the beginning of the paper) give an intuition behind the methodology improvements that this paper aims to provide, relative to previous work on computing the Wasserstein barycenter. This paper quickly dives into the algorithm and theory details (“Background”, “Mathematical preliminaries”, “Deriving the optimization problem”), without giving a clear treatment of previous work, and how this methods of this paper differ from this work. It is therefore is hard to see where the material developed in previous work ends and the new methodology of this paper begins. A simple description (early on) of this method, how it differs from existing methods, and why it solves the problem inherent in these existing methods, would greatly increase the clarity of this paper.
- Furthermore, it would be nice to include more motivation behind the main theoretical results that are proved in this paper. It is hard to get a grasp on the usefulness and contribution of these results without some discussion or reasoning on why one would like to prove these results (e.g. the benefits of proving this theory).
- Finally, I do not think that the empirical results in this paper are particularly thorough. The results in Table 1 are straightforward, but these seem to be the only empirical argument of the paper, and they are quite minimal.


Parametric Simplex Method for Sparse Learning
High dimensional sparse learning has imposed a great computational challenge to large scale data analysis. In this paper, we are interested in a broad class of sparse learning approaches formulated as linear programs parametrized by a regularization factor, and solve them by the parametric simplex method (PSM). Our parametric simplex method offers significant advantages over other competing methods: (1) PSM naturally obtains the complete solution path for all values of the regularization parameter; (2) PSM provides a high precision dual certificate stopping criterion; (3) PSM yields sparse solutions through very few iterations, and the solution sparsity significantly reduces the computational cost per iteration. Particularly, we demonstrate the superiority of PSM over various sparse learning approaches, including Dantzig selector for sparse linear regression, LAD-Lasso for sparse robust linear regression, CLIME for sparse precision matrix estimation, sparse differential network estimation, and sparse Linear Programming Discriminant (LPD) analysis. We then provide sufficient conditions under which PSM always outputs sparse solutions such that its computational performance can be significantly boosted. Thorough numerical experiments are provided to demonstrate the outstanding performance of the PSM method.

This paper extends simplex algorithm to several sparse learning problem with regularization parameter. The proposed method can collect all the solutions (corresponding to different values of the regularization parameter) in the process of simplex algorithm. It is an efficient way to get the sparse solution path and avoid tuning the regularization parameter. The connection between path Dantzig selector formulation and sensitivity analysis looks interesting to me.
Major comments:
- The method used in this paper seems closely related to the sensitivity analysis of LP. What is the key difference? It looks like just an application of sensitivity analysis.
- The paper mentioned that the number of iterations is linear in the number of nonzero variables empirically. Is there any guarantee for such linear dependency?
Experiment:
- Table 1 is not necessary since PSM never violate the constraints.
- Simplex method is not very efficient for large scale LP. I did not see comparison with interior based approaches or primal dual approaches. Authors are expected to make such comparison to experiments more convincing.
Typo/question:
line 122: z^+, z^- => t^+, t^-
line 291: solutio => solution
line 195: what is the relation between the feasibility condition and the sign of b and c?
line 201: \bar{x}_B and \bar{z}_N are always positive? (otherwise how to always guarantee the feasibility when \lambda is large)
line 207: here we see an upper-bound for \lambda, it contradicts the claim that the feasibility will be guaranteed when \lambda is large enough. In addition, is it possible that \lambda_min > \lambda_max?
line 236: can we guarantee that there is no cycle in such searching?


Accelerated Stochastic Greedy Coordinate Descent by Soft Thresholding Projection onto Simplex
In this paper we study the well-known greedy coordinate descent (GCD) algorithm to solve 1 -regularized problems and improve GCD by the two popular strategies: Nesterov's acceleration and stochastic optimization. Firstly, based on an 1 -norm square approximation, we propose a new rule for greedy selection which is nontrivial to solve but convex; then an efficient algorithm called "SOft ThreshOlding PrOjection (SOTOPO)" is proposed to exactly solve an 1 -regularized 1 -norm square approximation problem, which is induced by the new rule. Based on the new rule and the SOTOPO algorithm, the Nesterov's acceleration and stochastic optimization strategies are then successfully applied to the GCD algorithm. The resulted algorithm called accelerated stochastic greedy coordinate descent (ASGCD) has the optimal convergence rate O( 1/ ); meanwhile, it reduces the iteration complexity of greedy selection up to a factor of sample size. Both theoretically and empirically, we show that ASGCD has better performance for high-dimensional and dense problems with sparse solutions.

Paper Summary:
The main idea is that Nesterov's acceleration method's and Stochastic Gradient Descent's (SGD) advantages are used to solve sparse and dense optimization problems with high-dimensions by using an improved GCD (Greedy Coordinate Descent) algorithm. First, by using a greedy rule, an $l_1$-square-regularized approximate optimization problem (find a solution close to $x^*$ within a neighborhood $\epsilon$) can be reformulated as a convex but non-trivial to solve problem. Then, the same problem is solved as an exact problem by using the SOTOPO algorithm. Finally, the solution is improved by using both the convergence rate advantage of Nesterov's method and the "reduced-by-one-sample" complexity of SGD. The resulted algorithm is an improved GCD (ASGCD=Accelerated Stochastic Greedy Coordinate Descent) with a convergence rate of $O(\sqrt{1/\epsilon})$ and complexity reduced-by-one-sample compared to the vanilla GCD.
Originality of the paper:
The SOTOPO algorithm proposed, takes advantage of the l1 regularization term to investigate the potential values of the sub-gradient directions and sorts them to find the optimal direction without having to calculate the full gradient beforehand. The combination of Nesterov's advantage with SGC advantage and the GCD advantage is less impressive. Bonus for making an efficient and rigorous algorithm despite the many pieces that had to be put together
Contribution:
-Reduces complexity and increases convergence rate for large-scale, dense, convex optimization problems with sparse solutions (+),
-Uses existing results known to improve performance and combines them to generate a more efficient algorithm (+),
-Proposes a criterion to reduce the complexity by identifying the non-zero directions of descent and sorting them to find the optimal direction faster (+),
-Full computation of the gradient beforehand is not necessary in the proposed algorithm (+),
-There is no theoretical way proposed for the choice of the regularization parameter $\lambda$ as a function of the batch size. The choice of $\lambda$ seems to affect the performance of the ASGCD in both batch choice cases (-).
Technical Soundness:
-All proofs to Lemmas, Corollaries, Theorems and Propositions used are provided in the supplementary material (+),
-Derivations are rigorous enough and solid. In some derivations further reference to basic optimization theorems or Lemmas could be more en-lighting to non-optimization related researchers (-).
Implementation of Idea:
The algorithm is complicated to implement (especially the SOTOPO part).
Clarity of presentation:
-Overall presentation of the paper is detailed but the reader is not helped to keep in mind the bigger picture (might be lost in the details). Perhaps reminders of the goal/purpose of each step throughout the paper would help the reader understand why each step is necessary(-),
-Regarding the order of application of different known algorithms or parts of them to the problem: it is explained but could be more clear with a diagram or pseudo-code (-),
-Notation:  in equation 3, $g$ is not clearly explained and in Algorithm 1 there are two typos in referencing equations (-),
-For the difficulty of writing such a mathematically incremental paper, the clarity is at descent (+).
Theoretical basis:
-All Lemmas and transformations are proved thoroughly in the supplementary material (+),
-Some literature results related to convergence rate or complexity of known algorithms are not referenced (lines 24,25,60,143 and 73 was not explained until equation 16 which brings some confusion initially). Remark 1 could have been referenced/justified so that it does not look completely arbitrary (-),
-A comparison of the theoretical solution accuracy with the other pre-existing methods would be interesting to the readers (-),
-In the supplementary material in line 344, a $d \theta_t$ is missing from one of the integrals (-).
Empirical/Experimental basis:
-The experimental results verify the performance of the proposed algorithm with respect to the ones chosen for comparison. Consistency in the data sets used between the different algorithms, supports a valid experimental analysis (+),
-A choice of better smoothing constant $T_1$ is provided in line 208 (+) but please make it more clear to the reader why this is a better option in the case of $b=n$ batch size (-),
-The proposed method is under-performing (when the batch size is 1) compared to Katyusha for small regularization $10^{-6}$ and for the test case Mnist while for Gisette it is comparable to Katyusha. There might be room for improvement in these cases or if not it would be interesting to show which regularization value is the threshold and why. The latter means that the algorithm proposed is more efficient for large-scale problems with potentially a threshold in sparsity (minimum regularization parameter) that the authors have not theoretically explored. Moreover, there seems to be a connection between the batch size (1 or n, in other words stochastic or deterministic case) and the choice of regularization value that makes the ASGCD outperform other methods which is not discussed (-).
Interest to NIPS audience [YES]: This paper compares the proposed algorithm with well-established algorithms or performance improvement schemes and therefore would be interesting to the NIPS audience. Interesting discussion might arise related to whether or not the algorithm can be simplified without compromising it's performance.


Working hard to know your neighbor's margins: Local descriptor learning loss
We introduce a loss for metric learning, which is inspired by the Lowe's matching criterion for SIFT. We show that the proposed loss, that maximizes the distance between the closest positive and closest negative example in the batch, is better than complex regularization methods; it works well for both shallow and deep convolution network architectures. Applying the novel loss to the L2Net CNN architecture results in a compact descriptor named HardNet. It has the same dimensionality as SIFT (128) and shows state-of-art performance in wide baseline stereo, patch verification and instance retrieval benchmarks.
best of our knowledge, no work in local descriptor learning fully mimics such strategy as the learning objective.
Simonyan and Zisserman [20] proposed a simple filter plus pooling scheme learned with convex optimization to replace the hand-crafted filters and poolings in SIFT. Han et al. [14] proposed a twostage siamese architecture -for embedding and for two-patch similarity. The latter network improved matching performance, but prevented the use of fast approximate nearest neighbor algorithms like kd-tree [21]. Zagoruyko and Komodakis [15] have independently presented similar siamese-based method which explored different convolutional architectures. Simo-Serra et al [22] harnessed hardnegative mining with a relative shallow architecture that exploited pair-based similarity.
The three following papers have most closedly followed the classical SIFT matching scheme. Balntas et al [23] used a triplet margin loss and a triplet distance loss, with random sampling of the patch triplets. They show the superiority of the triplet-based architecture over a pair based. Although, unlike SIFT matching or our work, they sampled negatives randomly. Choy et al [7] calculate the distance matrix for mining positive as well as negative examples, followed by pairwise contrastive loss.
Tian et al [24] use n matching pairs in batch for generating n 2 − n negative samples and require that the distance to the ground truth matchings is minimum in each row and column. No other constraint on the distance or distance ratio is enforced. Instead, they propose a penalty for the correlation of the descriptor dimensions and adopt deep supervision [25] by using intermediate feature maps for matching. Given the state-of-art performance, we have adopted the L2Net [24] architecture as base for our descriptor. We show that it is possible to learn even more powerful descriptor with significantly simpler learning objective without need of the two auxiliary loss terms.

The paper presents a variant of patch descriptor learning using neural networks and a triplet loss. While many similar approaches exist, the particular variant proposed here appears to have better results in a large number of benchmarks.
Still, I am really unsure about the technical contribution as the paper is not very clear on this point. The approach appears to be very similar to others such as Balntas 16 that also used triplet losses and deep nets to learn patch descriptors. It seems that the main difference is to consider in the loss hard negative examples. However, Balntas 16 *also* uses hard negatives in the triplets (not merely random samples as argued here on line 46). So what is the difference that changes the empirical results so much?
The paper generally needs far more polish. Experiments should carefully describe the difference between different approaches (e.g. descriptor dimensionally, neural network architecture, training set, and anything else that could make a difference). Then, the main reason for the observed empirical boost should be unequivocally identified through empirical assessment.  For example, if the claim is that the key is to pool hard negatives, a carefully ablation study comparing this to alternative tripled-formation strategies should be included. Part of such a study may be included in the HardTFeat vs TFeat experiment of Fig. 7, but I would liket to check with the authors that they ran both methods using the same code base and only changing the feature sampling strategy. If more than one thing changes at a time, is difficult to reach a conclusion.
If the authors could clearly identify an interesting reason explaining their empirical boost (e.g. they may conclude that the key is a new and better way of doing hard negative mining compared to what has been done so far), then the paper could be interesting enough for acceptance. If, however, the boost is due to other differences such as more tuning or a tweaked architecture, then there would be much less of a reason to accept this into NIPS.
Rebuttal: the authors provided an informative rebuttal.
One of my key question was about the difference with Blantas 16. The authors argue that Blantas 16 does *not* do hard negative mining. On re-reading Blantas 16, this is partially true: their point is that a simplified form of hard negative mining (called in-triplet hard negatives) is just as effective and in fact superior to hard negative mining, tested in e.g. "Discriminative learning of deep convolutional feature point descriptors".
Given the new experiments in the rebuttal, it seems that the main idea here is a new variant of negative mining, within each batch rather than in the whole datasets as done in "Discriminative learning of deep convolutional feature point descriptors", which seems to be similar to the HNM method described in the rebuttal.
Hence to me the main message of the paper is that, while HB and HNM are very simple and very similar approaches, HB is in practice far better than HNM.  The authors should modify the paper to include a careful experimental analysis of this point, extending and consolidating the new experiments in the rebuttal.
With this and other promised improvements, the paper would be good enough for acceptance in my opinion. However, the modifications required from the submitted version are fairly large.


Practical Hash Functions for Similarity Estimation and Dimensionality Reduction
Hashing is a basic tool for dimensionality reduction employed in several aspects of machine learning. However, the perfomance analysis is often carried out under the abstract assumption that a truly random unit cost hash function is used, without concern for which concrete hash function is employed. The concrete hash function may work fine on sufficiently random input. The question is if they can be trusted in the real world where they may be faced with more structured input. In this paper we focus on two prominent applications of hashing, namely similarity estimation with the one permutation hashing (OPH) scheme of Li et al. [NIPS'12]   and feature hashing (FH) of Weinberger et al. [ICML'09], both of which have found numerous applications, i.e. in approximate near-neighbour search with LSH and large-scale classification with SVM. We consider the recent mixed tabulation hash function of Dahlgaard et al. [FOCS'15]  which was proved theoretically to perform like a truly random hash function in many applications, including the above OPH. Here we first show improved concentration bounds for FH with truly random hashing and then argue that mixed tabulation performs similar when the input vectors are not too dense. Our main contribution, however, is an experimental comparison of different hashing schemes when used inside FH, OPH, and LSH. We find that mixed tabulation hashing is almost as fast as the classic multiply-modprime scheme (ax + b) mod p. Mutiply-mod-prime is guaranteed to work well on sufficiently random data, but here we demonstrate that in the above applications, it can lead to bias and poor concentration on both real-world and synthetic data. We also compare with the very popular MurmurHash3, which has no proven guarantees. Mixed tabulation and MurmurHash3 both perform similar to truly random hashing in our experiments. However, mixed tabulation was 40% faster than MurmurHash3, and it has the proven guarantee of good performance (like fully random) on all possible input making it more reliable.

The main contribution of the paper is the set of empirical comparisons of various hashing functions, mutiply, 2-universal (2U), Murmur, and mixed tabulation, for (1) similarity estimation and (2) LSH. This type of empical evaluations is very important and will benefit practioners. Overall, this is a good paper. For improving the paper, I would suggest the authors to take into consideration of the following comments (some of which might be crucial):
---1. There is already a study of using 2U and 4U hashing for similarity estimation and classification tasks. See
[1] b-Bit Minwise Hashing in Practice. Proceedings of the 5th Asia-Pacific Symposium on Internetware, 2013.
[2] https://arxiv.org/abs/1205.2958, which is a more detailed version of [1], as far as I can tell.
[1,2] used 2U and 4U for evalating both b-bit minwise hashing and FH　(although they used VW to refer to FH). For example, [1,2] showed that for dense sets, 2U and 4U had an obvious bias and larger MSE.
I would suggest the authors to cite one of [1,2] and comment on the additional contributions beyond [1,2].
---2. [1,2] already showed that for classification tasks, 2U and 4U seems to be sufficiently accurate compared to using fully random hash functions. This paper ommited classification experiments. It is not at all surprising that mixed tabulation will work as well as fully random hash for classification, which the NIPS community would probably care more (than LSH).
---3. This paper used 2U instead of 4U. It would be more coninvincing to present the results using 4U (or both 2U and 4U) as it is well-understood that in some cases 4U is indeed better.
---4. The LSH experiments can have been presented more convincingly.
a) In practice, we typically must guaranttee a good recall, even at the cost of retrieving more points. therefore, while the presentation of using the ratio # retrieved / recall is useful, it may cover important details.
b) only mulply hash and mix tabulation results are presented. It is important to also present fully random results and 2U/4U hashing results.
c) If the results from different hash functions as shown  in Figure 5 really differ that much, then it is not that appropriate to compare the results at a fixed K and L, because each hash function may work better with a particular parameter.
d) The LSH parameters are probably chosen not so appropriately. The paper says it followed [30] for the parameters K,L by using K ={8,10,12} and L={8,10,12}. However, [30] actually used  for K = {6, 8, 10, 12} and L = {4, 8, 16, 32, 64, 128}
e)  LSH-OPH with K = {8,10,12} is likely too large. L =[8,10,12} is likely too small, according to prior experience. If the parameters are not (close to) optimal, then it is difficult to judge usefulness of the experiment results.
-----------
In Summary, overall, the topic of this paper is very important. The limitation of the current submission includes i) a lack of citation of the prior work [1,2] and explanation on the additional contribution beyond [1,2]. ii) the classification experiments are missing. It is very possible that no essential difference will be observed for any hash function, as concluded in the prior work [1,2]. iii) the LSH experiments may have issues and the conclusions drawn from the LSH experiments are hard to judge.


On the Model Shrinkage Effect of Gamma Process Edge Partition Models Iku Ohama
The edge partition model (EPM) is a fundamental Bayesian nonparametric model for extracting an overlapping structure from binary matrix. The EPM adopts a gamma process (ΓP) prior to automatically shrink the number of active atoms. However, we empirically found that the model shrinkage of the EPM does not typically work appropriately and leads to an overfitted solution. An analysis of the expectation of the EPM's intensity function suggested that the gamma priors for the EPM hyperparameters disturb the model shrinkage effect of the internal ΓP. In order to ensure that the model shrinkage effect of the EPM works in an appropriate manner, we proposed two novel generative constructions of the EPM: CEPM incorporating constrained gamma priors, and DEPM incorporating Dirichlet priors instead of the gamma priors. Furthermore, all DEPM's model parameters including the infinite atoms of the ΓP prior could be marginalized out, and thus it was possible to derive a truly infinite DEPM (IDEPM) that can be efficiently inferred using a collapsed Gibbs sampler. We experimentally confirmed that the model shrinkage of the proposed models works well and that the IDEPM indicated state-of-the-art performance in generalization ability, link prediction accuracy, mixing efficiency, and convergence speed.

This paper presents several variants of GP-EPM to solve the shrinkage problem of GP-EPM for modelling relational data. The proposed models are demonstrated to have better link prediction performance than GP-EPM by estimating the number of latent communities better.
Note I always point out this problem to my students as:  the model is
"non-identifiable".   Don't get me wrong, the whole body of work,
EPM, etc., is fabulous, but identifiability was always an important lesson in class.
The paper is well-written and easy to follow. The proposed models are well-motivated empirically with synthetic examples and then with theoretical analysis. To me, the main contribution of the paper is it points out that the obvious unidentifiability issue in GP-EPM can, interestingly, be a problem in some real cases. The proposed CEPM is just a naive model, while DEPM is a simple but intuitive solution and DEPM to IDEPM is like mapping the Dirichlet distribution to Dirichlet process. The solutions are not novel but straightforward to the problem. Finally the experimental results support the main claims, which makes it an interesting paper.
Note, in 4.3 (line 187), you say "one remarkable property".  This *should* be well known in the non-parametric community.  The result is perhaps the simplest case of the main theorem in "Poisson Latent Feature Calculus for Generalized Indian Buffet Processes", Lancelot James, 2014.  The terms in $K_+$ in (5) are the marginal for the gamma process.  The theorem can be pretty much written down without derivation using standard results.
This is a good paper cleaning up the obvious flaws in EPM and its implementation. It covers good though routine theory, and experimental work.  Section 4.3 should be rewritten.
However, I belief the important issue of identifiability should be better discussed and more experiments done.  For instance, a reviewer points out that the problem is partially overcome more recently using stronger priors.  Clearly, identifiability is something we require to make theory easier and priors more interpretable.  It is not strictly necessary, though it is routinely expected in the statistics community.
Anyway, would be good to see more discussion of the issues and further experimental investigation.


Learning to Inpaint for Image Compression
We study the design of deep architectures for lossy image compression. We present two architectural recipes in the context of multi-stage progressive encoders and empirically demonstrate their importance on compression performance. Specifically, we show that: (a) predicting the original image data from residuals in a multi-stage progressive architecture facilitates learning and leads to improved performance at approximating the original content and (b) learning to inpaint (from neighboring image pixels) before performing compression reduces the amount of information that must be stored to achieve a high-quality approximation. Incorporating these design choices in a baseline progressive encoder yields an average reduction of over 60% in file size with similar quality compared to the original residual encoder.

This paper proposes a progressive image compression method that's "hybrid". The authors use the framework of Toderici et al (2016) to setup a basic progressive encoder, and then they improve on it by studying how to better propagate information between iterations. The solution involves using "temporal" residual connections, without the explicit need to have an RNN per se (though this point is a bit debatable because in theory if the residual connections are transformed by some convolution, are they acting as an additive RNN or not?). However, the authors also employ a predictor (inpainter). this allows them to encode each patch after trying to predict an "inpainted" version first. It is important to note here that this will only work (in practice) if all the patches on which this patch depends on have been decoded. This introduces a linear dependency on patches, which may make the method too slow in practice, and it would be nice to see a bit more in the text about this issue (maybe some timing in formation vs. not using inpainting).
Overall, I think the paper was well written and an expert should be able to reproduce the work.
Given that the field of neural image compression is still in its infancy and that most of the recent papers have been focusing on non-progressive methods, and this paper proposes a *progressive* encoder/decoder, I think we should seriously consider accepting it.


The Expxorcist: Nonparametric Graphical Models Via Conditional Exponential Densities
Non-parametric multivariate density estimation faces strong statistical and computational bottlenecks, and the more practical approaches impose near-parametric assumptions on the form of the density functions. In this paper, we leverage recent developments to propose a class of non-parametric models which have very attractive computational and statistical properties. Our approach relies on the simple function space assumption that the conditional distribution of each variable conditioned on the other variables has a non-parametric exponential family form.

This paper proposes a method to estimate a joint multivariate density non-parametrically by estimating a product of marginal conditional exponential densities. The marginal distribution of each variable is conditioned on the neighbors of that variable in a graph. The authors first discuss how their work relate to the literature. They study the consistency  of their approach in terms of the estimated marginals with respect to a joint distribution, by relying on a theorem in ref. 28. They also develop an estimation algorithm, study statistical properties of their approach and discuss the relationships with copulas. The paper ends with experimental results. The paper is well written.
1) Are there guarantees if the real distribution does not belong to the family of distribution estimated?
2) I am not very familiar with nonparametric density estimation. How practical is the assumption that the base measures are known?
3) The experiments seem well conducted an the baselines selected in a sensible manner. The evaluation is performed both on synthetic and real data. On synthetic data, the approach is only evaluated on models matching the assumption that the factors are at most of size 2. Imho, testing the approach on more general model would help understand its usefulness.
4) l119: why doesn't nonparametric estimation benefit from estimating a joint distribution as a product of conditional marginal distribution?
5) How easy/hard would it be to extend your approach to cliques of size greater than 2?
6) Will you make your code available?
7) Based on the definition of the density, shouldn't the edges of the graph E mentioned in the introduction correspond to non-independence rather than independence?
Typos:
You use both "nonparametric" and "non-parametric".


Adaptive SVRG Methods under Error Bound Conditions with Unknown Growth Parameter
Error bound, an inherent property of an optimization problem, has recently revived in the development of algorithms with improved global convergence without strong convexity. The most studied error bound is the quadratic error bound, which generalizes strong convexity and is satisfied by a large family of machine learning problems. Quadratic error bound have been leveraged to achieve linear convergence in many first-order methods including the stochastic variance reduced gradient (SVRG) method, which is one of the most important stochastic optimization methods in machine learning. However, the studies along this direction face the critical issue that the algorithms must depend on an unknown growth parameter (a generalization of strong convexity modulus) in the error bound. This parameter is difficult to estimate exactly and the algorithms choosing this parameter heuristically do not have theoretical convergence guarantee. To address this issue, we propose novel SVRG methods that automatically search for this unknown parameter on the fly of optimization while still obtain almost the same convergence rate as when this parameter is known. We also analyze the convergence property of SVRG methods under Hölderian error bound, which generalizes the quadratic error bound.

This paper introduces a restarting scheme for SVRG specialised to the case where the underlying problem satisfies the quadratic error bound (QEB) condition, a weaker condition than strong convexity that still allows for linear convergence. The algorithm solves the problem of having to know the value c of the QEB constant before hand.
The restarting scheme applies full SVRG repeatedly in a loop. If after the application of SVRG the resulting iterate is less than a 25% relative improvement (according to a particular notion of solution quality they state), the value of c is increased by \sqrt{2}, so that the next run of SVRG uses double the number of iterations in it's inner loop (T propto c^2).
The use of doubling schemes for estimating constants in optimisation algorithms is a very standard technique. It's use with SVRG feels like only an incremental improvement. A particular problem with such techniques is setting the initial value of the constant. Too small a value will result in 5-10 wasted epochs, where as too large a value results in very slow convergence.
The experiments section is well-explained, with standard test problems used. I do have some issues with the number of steps shown on the plots. The x axis #grad/n goes to 1000 on each, which is unrealistically large. It masks the performance of the algorithm during the early iterations. In the SVRG and SAGA papers the x axis spans from 0 to 30-100 of #grad/n, since this represents the range where the loss bottoms out on held-out tests sets. It's hard to tell from the plots as they are at the moment if the algorithm works well where it matters. This would be much clearer if error on a test set was shown as well.
On the first plot, a comparison is made against SVRG with a fixed number of inner iterations T, for a variety of T. The largest T tried had the best performance, I would suggest adding larger values to the plot to make clear that using too large a value results in worse performance (i.e. there is a sweet spot). I would remove the T=100 from the plot to make room.
In terms of language, the paper is ok, although in a number of places the language is a little informal. There are a moderate number of grammatical issues. The paper does require additional proof reading to get it up to standard.


Improved Dynamic Regret for Non-degenerate Functions
Recently, there has been a growing research interest in the analysis of dynamic regret, which measures the performance of an online learner against a sequence of local minimizers. By exploiting the strong convexity, previous studies have shown that the dynamic regret can be upper bounded by the path-length of the comparator sequence. In this paper, we illustrate that the dynamic regret can be further improved by allowing the learner to query the gradient of the function multiple times, and meanwhile the strong convexity can be weakened to other non-degenerate conditions. Specifically, we introduce the squared path-length, which could be much smaller than the path-length, as a new regularity of the comparator sequence. When multiple gradients are accessible to the learner, we first demonstrate that the dynamic regret of strongly convex functions can be upper bounded by the minimum of the path-length and the squared path-length. We then extend our theoretical guarantee to functions that are semi-strongly convex or selfconcordant. To the best of our knowledge, this is the first time that semi-strong convexity and self-concordance are utilized to tighten the dynamic regret.

In this paper, the authors study the problem of minimizing the dynamic regret in online learning. First, they introduce squared path-length to measure the complexity of the comparator sequence. Then, they demonstrate that if multiple gradients are accessible to the learner, the dynamic regret of strongly convex functions can be upper bounded by the minimum of the path-length and the squared path-length. Finally, they extend their theoretical guarantees to functions that are semi-strongly convex or self-concordant.
This is a theoretical paper for analyzing the dynamic regret. The main difference from previous work is that the learner is able to query the gradient multiple times. The authors prove that under this feedback model, dynamic regret could be upper bounded by the minimum of the path-length and the squared path-length, which is a significant improvement when the squared path-length is small.
Strong Points:
1. A new performance measure is introduced to bound the dynamic regret.
2. When functions are strongly convex, the authors develop a new optimization algorithm and prove its dynamic regret is upper bounded by the minimum of the path-length and the squared path-length.
3. This is the first time that semi-strong convexity and self-concordance are utilized to tighten the dynamic regret.
Suggestions/Questions:
1. It is better to provide some empirical studies to support the theoretical results.
2. For self-concordant functions, why do we need an additional condition in (11)?
3. Due to the matrix inverse, the complexity of online multiple Newton update (OMNU) is much higher than online multiple gradient descent (OMGD), which should be mentioned explicitly.


Dual-Agent GANs for Photorealistic and Identity Preserving Profile Face Synthesis
Synthesizing realistic profile faces is promising for more efficiently training deep pose-invariant models for large-scale unconstrained face recognition, by populating samples with extreme poses and avoiding tedious annotations. However, learning from synthetic faces may not achieve the desired performance due to the discrepancy between distributions of the synthetic and real face images. To narrow this gap, we propose a Dual-Agent Generative Adversarial Network (DA-GAN) model, which can improve the realism of a face simulator's output using unlabeled real faces, while preserving the identity information during the realism refinement. The dual agents are specifically designed for distinguishing real v.s. fake and identities simultaneously. In particular, we employ an off-the-shelf 3D face model as a simulator to generate profile face images with varying poses. DA-GAN leverages a fully convolutional network as the generator to generate high-resolution images and an auto-encoder as the discriminator with the dual agents. Besides the novel architecture, we make several key modifications to the standard GAN to preserve pose and texture, preserve identity and stabilize training process: (i) a pose perception loss; (ii) an identity perception loss; (iii) an adversarial loss with a boundary equilibrium regularization term. Experimental results show that DA-GAN not only presents compelling perceptual results but also significantly outperforms state-of-the-arts on the large-scale and challenging NIST IJB-A unconstrained face recognition benchmark. In addition, the proposed DA-GAN is also promising as a new approach for solving generic transfer learning problems more effectively. DA-GAN is the foundation of our submissions to NIST IJB-A 2017 face recognition competitions, where we won the 1 st places on the tracks of verification and identification.

This work uses GANs to generate synthetic data to use for supervised training of facial recognition systems. More specifically, they use an image-to-image GAN to improve the  quality of faces generated by a face simulator. The simulator is able to produce a wider range of face poses for a given face, and the GAN is able to refine the simulators output such that it is more closely aligned with the true distribution of faces (i.e. improve the realism of the generated face) while maintaining the facial identity and pose the simulator outputted. They show that by fine tuning a facial recognition system on this additional synthetic data they are able to improve performance and outperform previous state of the art methods.
Pros:
- This method is simple, apparently effective and is a nice use of GANs for a practical task. The paper is clearly written
Cons:
- My main concern with this paper is regarding the way in which the method is presented. The authors term their approach "Dual Agent GANS" and seem to claim a novel GAN architecture. However, it is not clear to me what aspect of their GAN is particularly new. The "dual agent"aspect of their GAN comes from the fact that they have a standard adversarial term (in their case the BE-GAN formulation) plus a cross entropy  term to ensure the facial identity is preserved. But previous work (e.g. InfoGAN, Auxiliary classifier GAN) have both also utilized a combination of "heads". So it seems odd to me that the authors are pushing this work as a new GAN architecture/method. I realize it's very trendy these days to come up with a slightly new GAN architecture and give it a new cool name, but this obfuscates the contributions. I think this is an interesting paper from perspective of using GANs in a data augmentation pipeline (and certainly their particular formulation is tailored to the task at hand) but I do not like that the authors appear to be claiming a new GAN method.
- Since I think the main contribution of this paper is a data augmentation technique for facial recognition systems, it'd be good to see > 1 dataset explored.
Some additional comments/questions:
- In eq. 8, do you mean to have a minus sign in the L_G term?
- What was the performance of the network you trained before fine tuning? i.e. how much improvement comes from this technique vs. different/better architectures/hyper-parameters/etc. compared to other methods


Learning Affinity via Spatial Propagation Networks
In this paper, we propose spatial propagation networks for learning the affinity matrix for vision tasks. We show that by constructing a row/column linear propagation model, the spatially varying transformation matrix exactly constitutes an affinity matrix that models dense, global pairwise relationships of an image. Specifically, we develop a three-way connection for the linear propagation model, which (a) formulates a sparse transformation matrix, where all elements can be outputs from a deep CNN, but (b) results in a dense affinity matrix that effectively models any task-specific pairwise similarity matrix. Instead of designing the similarity kernels according to image features of two points, we can directly output all the similarities in a purely data-driven manner. The spatial propagation network is a generic framework that can be applied to many affinity-related tasks, such as image matting, segmentation and colorization, to name a few. Essentially, the model can learn semantically-aware affinity values for high-level vision tasks due to the powerful learning capability of deep CNNs. We validate the framework on the task of refinement of image segmentation boundaries. Experiments on the HELEN face parsing and PASCAL VOC-2012 semantic segmentation tasks show that the spatial propagation network provides a general, effective and efficient solution for generating high-quality segmentation results.

The paper describes a method for learning pairwise affinities for recurrent label refinement in deep networks. A typical application is as follows: a feature map is produced by a convolutional network and is then refined by additional layers that in effect pass messages between pixels. The weights for such message passing are often set using hand-defined feature spaces (although prior work on learning such weights exists, see below). The submission describes a formulation for learning such weights.
The paper has a number of issues that lead me to recommend rejection:
1. The general problem tackled in this paper -- refining poorly localized segmentation boundaries -- has been tackled in many publications. Two representative approaches are: (a) add layers that model mean field inference in a dense CRF and train them jointly with the initial segmentation network (as in [1,13,30]); (b) add a convolutional refinement module, such as the context module in [Multi-Scale Context Aggregation by Dilated Convolutions, ICLR 2016], also trained jointly with the segmentation network. The submission should provide controlled experiments that compare the presented approach to this prior work, but it doesn't. An attempt is made in Table 1, but it is deeply flawed. As far as I can tell, the dense CRF is not trained end-to-end with the segmentation network, as commonly done in the literature, such as [1,13,30]. And the context module (the ICLR 2016 work referred to above) is not compared to at all, even though it was developed for this specific purpose and is known to yield good results. (In fact, the ICLR 2016 paper reports refinement results on the VOC 2012 dataset with the VGG network that are better than the SPN results in Table 1 (IoU of 73.9 in the "Front + Large + RNN" condition in Table 3 of the ICLR 2016 paper). And that's a comparatively old work by now.)
2. There is other related work in the literature that specifically addresses learning affinities for label refinement. This work is closely related but is not cited, discussed, or compared to:
   -Semantic Segmentation with Boundary Neural Fields. Gedas Bertasius, Jianbo Shi and Lorenzo Torresani. CVPR 2016
   - Convolutional Random Walk Networks for Semantic Image Segmentation. Gedas Bertasius, Lorenzo Torresani, Stella X. Yu, Jianbo Shi. CVPR 2017
   - Learning Dense Convolutional Embeddings for Semantic Segmentation. Adam W. Harley, Konstantinos G. Derpanis, Iasonas Kokkinos. ICLR Workshop 2016
3. The results on VOC 2012 are well below the current state of the art, which stands at 86% IoU (compared to 79.8% in the submission). One could argue that the authors are more interested in evaluating the contribution of their refinement approach when added to some baseline segmentation networks, but (a) such controlled evaluation was not done properly (see point (1) above); and (b) the authors' combined approach is quite elaborate, so it would be hard to claim that it is somehow much simpler than state-of-the-art networks that dominate the leaderboard. With this level of complexity, it is reasonable to ask for state-of-the-art performance.
Minor comment:
- Since the dense CRF seems to play an important role in the submission, as a baseline that is repeatedly compared against, the submission should cite the dense CRF paper: [Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials, NIPS 2011].


Semi-supervised Learning with GANs: Manifold Invariance with Improved Inference
Semi-supervised learning methods using Generative adversarial networks (GANs) have shown promising empirical success recently. Most of these methods use a shared discriminator/classifier which discriminates real examples from fake while also predicting the class label. Motivated by the ability of the GANs generator to capture the data manifold well, we propose to estimate the tangent space to the data manifold using GANs and employ it to inject invariances into the classifier. In the process, we propose enhancements over existing methods for learning the inverse mapping (i.e., the encoder) which greatly improves in terms of semantic similarity of the reconstructed sample with the input sample. We observe considerable empirical gains in semi-supervised learning over baselines, particularly in the cases when the number of labeled examples is low. We also provide insights into how fake examples influence the semi-supervised learning procedure.

Chiefly theoretical work with some empirical results on SVHN and CIFAR10. This paper proposes using a trained GAN to estimate mappings from and to the true data distribution around a data point, and use a kind of neural PCA to estimate tangents to those estimates, then used for training a manifold-invariant classifier. Some additional work investigating regular GAN vs feature matching GAN is briefly presented, and an augmentation to BiGAN.
It feels a bit like this is two works squeezed into one. Maybe three.
There is an exploration of FM-GAN vs a regular GAN training objective, with some nice results shown where the classification entropy (confidence of the classification, as I read it) is much better for an FM-GAN than for a regular GAN.
There is an Augmented BiGAN which achieves nicely lower classification error than BiGAN on infer-latents-then-generate g(h(x)).
The most substantial work presented here is the manifold-invariance. The first thing I wrestle with is that the method is a bit complex, making it probably tricky for others to get right, and complex/fiddly to implement. In particular, 2.1.2 proposes to freeze f, g, and h, and introduce p and pbar as a nonlinear approximation to SVD. This introduces the second thing I wrestle with: numerous layered approximations. The method requires g and h to be good approximations to generate and infer to and from the data manifold. The results (e.g. figure 2) do not indicate that these approximations are always very good. The method requires that p and pbar reasonably capture singular directions in the latent space, but figure 3 shows this approximation only sort-of holds. This has me wondering about transferability, and wondering how to measure each of the approximation errors to evaluate whether the method is useful for a dataset. The CIFAR-10 results reinforce my concern.
The MNIST result in line 264 (0.86) is quite good for semi-supervised. What is the difference between the results in 264-265 vs the results in Table 1? Different numbers are given for SVHN in each location, yet line 251 suggests the results in both locations are semi-sup. Work would feel more complete with comparisons on semi-sup MNIST in Table 1, then you could show ladder, CAE, MTC, etc. I'm guessing you're up against space constraints here...
Table 1 missing some competitive results. A quick Google search for svhn semi supervised gives https://arxiv.org/abs/1610.02242 showing 7.05% for SVHN with 500 labels, 5.43% with 1000 labels; 16.55% CIFAR10@4k. https://papers.nips.cc/paper/6333-regularization-with-stochastic-transformations-and-perturbations-for-deep-semi-supervised-learning.pdf reports 6.03% on SVHN with 1% of labels (~700).
Missing conclusion/future directions.
Minor nitpicks:
grammar of lines 8-9 needs work
grammar of lines 42-43
lines 72/73 have two citations in a row
lines 88/89 unnecessary line break
line 152 wrong R symbol
line 204 comma before however
lne 295 'do not get as better' grammar needs work
All in all, the results are OK-to-good but not universally winning. The topic is of interest to the community, including a few novel ideas. (The paper perhaps should be multiple works.) It looks like a new SOTA is presented for SVHN semisup, which tips me toward accept.


Perturbative Black Box Variational Inference
Black box variational inference (BBVI) with reparameterization gradients triggered the exploration of divergence measures other than the Kullback-Leibler (KL) divergence, such as alpha divergences. In this paper, we view BBVI with generalized divergences as a form of estimating the marginal likelihood via biased importance sampling. The choice of divergence determines a bias-variance trade-off between the tightness of a bound on the marginal likelihood (low bias) and the variance of its gradient estimators. Drawing on variational perturbation theory of statistical physics, we use these insights to construct a family of new variational bounds. Enumerated by an odd integer order K, this family captures the standard KL bound for K = 1, and converges to the exact marginal likelihood as K → ∞. Compared to alpha-divergences, our reparameterization gradients have a lower variance. We show in experiments on Gaussian Processes and Variational Autoencoders that the new bounds are more mass covering, and that the resulting posterior covariances are closer to the true posterior and lead to higher likelihoods on held-out data.

Summary:
The authors present a new variational objective for approximate Bayesian inference.  The variational objective is nicely framed as an interpolation between classic importance sampling and the traditional ELBO-based variational inference.  Properties of the variance of importance sampling estimator and ELBO estimators are studied and leveraged to create a better marginal likelihood bound with tractable variance properties.  The new bound is based on a low-degree polynomial of the log-importance weight (termed the interaction energy).  The traditional ELBO estimator is expressed as a first order polynomial in their more general framework.
The authors then test out the idea on a Gaussian process regression problem and a Variational autoencoder.
Quality:  I enjoyed this paper --- I thought the idea was original, the presentation is clear, and the experiments were convincing.  The paper appears to be technically correct, and the method itself appears to be effective.
Clarity:  This paper was a pleasure to read.  Not only was this paper extremely clear and well written, the authors very nicely frame their work in the context of other current and previous research.
Originality:  While establishing new lower bounds on the marginal likelihood is a common subject at this point, the authors manage to approach this with originality.
Impact:  I think this paper has the potential to be highly impactful --- the spectrum drawn from importance sampling to KLVI is an effective way of framing these ideas for future research.
Questions/Comments:
  - Figure 1: should the legend text "KLVI: f(x) = 1 + log(x)" read "f(x) = log(x)" ?  I believe that would cohere with the bullet point on line 124.
  - How does the variance of the marginal likelihood bound estimators relate to the variance of the gradients of those estimators wrt variational params?  KLVI reparameterization gradients can have some unintuitive irreducibility (Roeder et al, https://arxiv.org/abs/1703.09194); is this the case for PVI?


Union of Intersections (UoI) for Interpretable Data Driven Discovery and Prediction
The increasing size and complexity of scientific data could dramatically enhance discovery and prediction for basic scientific applications. Realizing this potential, however, requires novel statistical analysis methods that are both interpretable and predictive. We introduce Union of Intersections (UoI), a flexible, modular, and scalable framework for enhanced model selection and estimation. Methods based on UoI perform model selection and model estimation through intersection and union operations, respectively. We show that UoI-based methods achieve low-variance and nearly unbiased estimation of a small number of interpretable features, while maintaining high-quality prediction accuracy. We perform extensive numerical investigation to evaluate a UoI algorithm (U oI Lasso ) on synthetic and real data. In doing so, we demonstrate the extraction of interpretable functional networks from human electrophysiology recordings as well as accurate prediction of phenotypes from genotype-phenotype data with reduced features. We also show (with the U oI L1Logistic and U oI CU R variants of the basic framework) improved prediction parsimony for classification and matrix factorization on several benchmark biomedical data sets. These results suggest that methods based on the UoI framework could improve interpretation and prediction in data-driven discovery across scientific fields.

This paper focuses on model selection and, to some extent, feature selection in large datasets with many features, of which only a small subset are assumed to be necessary for accurate prediction.  The authors propose a general method by which model selection is performed by way of feature compression performed by taking the intersection of a multiple regularization parameters in an ensemble method, and then model estimation by taking a union over multiple outputs.  (This is also the major contribution of the paper.)  A second contribution is found in the union operation in a model averaging step with a boosting/bagging flavor.  Overall, I found the paper's method section well written and the idea proposed to be complete.  The paper's experimental section was difficult to follow, but the results do seem to support the framework.  One major missing part of the paper is a reasonable discussion of using the framework beyond a Lasso base.   Are there reasons why this method would not work for classification?  Are there potential hitches to using this method with already-ensemble-based methods like random forests?  While there are many uses for the UoI with a Lasso base already, it was increase the general interest of the framework if UoI could be placed in the more general ML space.


Conic Scan-and-Cover algorithms for nonparametric topic modeling
We propose new algorithms for topic modeling when the number of topics is unknown. Our approach relies on an analysis of the concentration of mass and angular geometry of the topic simplex, a convex polytope constructed by taking the convex hull of vertices representing the latent topics. Our algorithms are shown in practice to have accuracy comparable to a Gibbs sampler in terms of topic estimation, which requires the number of topics be given. Moreover, they are one of the fastest among several state of the art parametric techniques.
1 Statistical consistency of our estimator is established under some conditions.

* Summary
This paper introduces a novel algorithm, Conic Scan Coverage, that is
based on convex geometry ideas and can perform non-parametric topic
modeling. The algorithm is intuitively based on the idea of covering
the topic simplex with cones. The papers presents the results of an
experimental evaluation and the supplementary contains detailed proofs
and example topics inferred by the algorithm.
* Evaluation
I have very mixed feelings about this paper.
On the one hand, I must say I had a lot of fun reading it. I am really
fond of the convex geometric approach to topic modeling, it is such an
original and potentially promising approach. Indeed, the results are
very good, and the topics presented in the supplementary material look
great.
On the other hand, I have a lot of uncertainty about this
paper. First, I must confess I do not understand the paper as well as
I would like. Of course, it could mean that I should have tried
harder, or maybe that the authors should explain more. Probably a bit
of both. Second, the experiments are weak. However, the paper is so
original that I don't think it should be rejected just because of
this.
To summarize:
+ Impressive results: excellent runtime and statistical performance, good looking topics
- Weak experimental evaluation
+ Very original approach to topic modeling
- Algorithm 1 seems very ad-hoc, and justification seems insufficient to me
Could you please show me a few example of inferred topic distribution?
* Discussion
- The experimental evaluation could be much better.
  The python implementation of Gibbs-LDA is pretty bad, it makes for a
  very poor baseline. You really need to say somewhere how many
  iterations were run.
  When comparing against algorithms like LDA Gibbs or HDP Gibbs, as in
  table 1, you can't report a single number. You should plot the
  evolution of perplexity by iteration. For large datasets, Gibbs LDA
  can reach a good perplexity and coherence in surprisingly few
  iterations. Also, there should be some error bars, a number in
  itself is not very convincing.
  You should also share information about hyperparameter settings. For
  instance, HDP and LDA can exhibit very different perplexity for
  different values of their alpha and beta hyperparameters.
  Finally, you should also vary the vocabulary size. I did note that V
  is very small, and it is worrysome. There are been several
  algorithms proposed in the past that seemed very effective for
  unrealistic small vocabulary sizes and didn't scale well to this
  parameter.
- The algorithm seems very ad-hoc
  I am surprised that you choose a topic as the furthest document and
  then remove all the documents within some cosine distance. Why makes
  such a farthest document a good topic representative? Also, if we
  remove the documents from the active sets based solely on one topic,
  are we really trying to explain the documents as mixture of topics?
  I would be really curious to see a few documents and their inferred
  topic distributions to see if it is interpretable.
- What good is Theorem 4?
  Theorem 4 is very interesting and definitely provides some good
  justification for the algorithm. However, it assumes that the number
  of topics K is known and doesn't say much about why such a procedure
  should find a good number of topics. Indeed, in the end, I don't
  have much intuition about what exactly drives the choice of number
  of topics.


Fisher GAN
Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce Fisher GAN which fits within the Integral Probability Metrics (IPM) framework for training GANs. Fisher GAN defines a critic with a data dependent constraint on its second order moments. We show in this paper that Fisher GAN allows for stable and time efficient training that does not compromise the capacity of the critic, and does not need data independent constraints such as weight clipping. We analyze our Fisher IPM theoretically and provide an algorithm based on Augmented Lagrangian for Fisher GAN. We validate our claims on both image sample generation and semi-supervised classification using Fisher GAN.

This paper proposes a new criterion for training a Generative Adversarial Network and shows that this new criterion yields stability benefits.
The criterion is related to Fisher discriminant analysis and is essentially a normalized IPM.
The authors show that this criterion is equivalent to the symmetric chi-squared divergence.
One reason for not being fully enthusiastic about this paper is that the fact that the proposed criterion is equivalent to chi-squared, which is
an f-divergence, reduces the novelty of the approach and raises several questions that are not addressed:
- is the proposed implementation significantly different than what one would obtain by applying f-GAN (with the appropriate f corresponding
to the chi-squared divergence)? if not, then the algorithm is really just f-GAN with chi-squared divergence and the novelty is drastically reduced. If yes,
then it would be great to pinpoint the differences and explain why they are important or would significantly address the issues that are inherent
to classical f-divergences based GANs in terms of training stability.
- how do the results compare if one uses other f-divergences (for example non-symmetric chi-squared but other f-divergences as well)?
- because the criterion is an f-divergence, it may suffer from the same issues that were pointed out in the WGAN paper: gradients would vanish
for distributions with disjoint support. Is there a reason why chi-squared would not have the same issues as Jensen-Shannon or KL ? Or is
it that the proposed implementation, not being exactly written as chi-squared and only equivalent at the optimum in lambda, doesn't suffer from
these issues?
Regarding the experimental results:
The DCGAN baseline for inceptions scores (Figure 4) seem lower than what was reported in earlier papers and the current state of the art is more around 8
(not sure what it was at the time of the writing of this paper though).
The SSL results (Table 2) are ok when compared to the weight clipping implementation of WGAN but this is not necessarily the right baseline to compare with,
and compared to other algorithms, the results are not competitive.
My overall feeling is that this paper presents an interesting connection between the chi-squared divergence and the normalized IPM coefficient which
deserves further investigation (especially a better understanding of how is this connected to f-GAN with chi-squared). But the comparison with the
known suboptimal implementation of WGAN (with weight clipping) is not so interesting and the results are not really convincing either.
So overall I think the paper is below the acceptance bar in its current form.


Simple Strategies for Recovering Inner Products from Coarsely Quantized Random Projections
Random projections have been increasingly adopted for a diverse set of tasks in machine learning involving dimensionality reduction. One specific line of research on this topic has investigated the use of quantization subsequent to projection with the aim of additional data compression. Motivated by applications in nearest neighbor search and linear learning, we revisit the problem of recovering inner products (respectively cosine similarities) in such setting. We show that even under coarse scalar quantization with 3 to 5 bits per projection, the loss in accuracy tends to range from "negligible" to "moderate". One implication is that in most scenarios of practical interest, there is no need for a sophisticated recovery approach like maximum likelihood estimation as considered in previous work on the subject. What we propose herein also yields considerable improvements in terms of accuracy over the Hamming distance-based approach in Li et al. (ICML 2014)  which is comparable in terms of simplicity.

********************************
* Summary 	               *
********************************
The paper investigates theoretically and empirically different strategies for recovery of inner products using quantized random projections of data instances. Random projections are often used in learning tasks involving dimensionality reduction. The goal of the additional quantization step is data compression that allows for a reduction in space complexity of learning algorithms and more efficient communication in distributed settings.
********************************
* Theoretical contributions    *
********************************
The main focus of the paper is on studying a linear strategy for recovery of inner products from quantized random projections of the data. The strategy approximates the inner product between two instances from the instance space with the inner product of the corresponding quantized random projections divided by the dimension of the projection space. The main theoretical contribution is a bound on the bias of such approximations (Theorem 1). In addition to this strategy, the paper considers recovery of inner products from random projections that are normalized (i.e., having unit norm) prior to quantization. For such approximations, the paper expresses the bias and variance in terms of the relevant terms of the linear strategy for recovery of inner products (Proposition 1). The paper also provides a bound on the variance of recovery of inner products with values close to one and strategies based on quantization with finitely many bits (Theorem 2).
********************************
* Quantization 		       *
********************************
The quantization of random projections is performed using the Lloyd-Max quantizer. The method resembles one dimensional K-means clustering where interval end-points determine the clusters and the centroid is given as the conditional expectation of the standard normal random variable given the interval, i.e., c_k = E [ z | z \in [t_k, t_{k+1}) ], where c_k is the cluster centroid or quantization value, t_k and t_{k + 1} are interval end-points of the kth interval, and the total number of intervals K is given with the number of quantization bits B = 1 + \log_2 K.
********************************
* Empirical study 	       *
********************************
# Figure 2
The goal of this experiment is to numerically verify the tightness of the bound on the bias of the linear strategy for recovery of inner products from quantized random projections. The figure shows that the bound is tight and already for the recovery with 5-bit quantization the bound almost exactly matches the bias. The paper also hypothesizes that the variance of the linear strategy with finite bit-quantization is upper bounded by the variance of the same strategy without quantization. The provided empirical result is in line with the hypothesis.
# Figure 3
The experiment evaluates the mean squared error of the recovery of inner products using the linear strategy as the number of quantization bits and dimension of the projection space change. The plot indicates that quantization with four bits and a few thousand of random projections might suffice for a satisfactory recovery of inner products.
# Figure 4
- In the first experiment, the normalized strategy for recovery of inner products from quantized random projections is compared to the linear one. The plot (left) indicates that a better bias can be obtained using the normalized strategy.
- In the second experiment, the variance of the quantized normalized strategy is compared to that without quantization. The plot (middle) indicates that already for quantization with 3 bits the variance is very close to the asymptotic case (i.e., infinitely many bits and no quantization).
- The third experiment compares the mean squared error of the normalized strategy for recovery of inner products from quantized random projections to that of the collision strategy. While the collision strategy performs better for recovery of inner products with values close to one, the normalized strategy is better globally.
# Figure 5
The experiment evaluates the strategies for recovery of inner products on classification tasks. In the first step the random projections are quantized and inner products are approximated giving rise to a kernel matrix. The kernel matrix is then passed to LIBSVM that trains a classifier. The provided empirical results show that the quantization with four bits is capable of generating an approximation to kernel matrix for which the classification accuracy matches that obtained using random projections without quantization. The plots depict the influence of the number of projections and the SVM hyperparameter on the accuracy for several high-dimensional datasets. The third column of plots in this figure also demonstrates that the normalized strategy for recovery of inner products from quantized random projections is better on classification tasks than the competing collision strategy.
********************************
* Theorem 1 		       *
********************************
Please correct me if I misunderstood parts of the proof.
# Appendix B: Eq. (4) --> Bound
Combining Eq. (6) with Eq. (4) it follows that
E[\rho_{lin}] - \rho = -2 \rho D_b + E[(Q(Z) - Z)(Q(Z') - Z')] >= -2 \rho D_b    ==>    2 \rho D_b >= \rho - E[\rho_{lin}] .
To be able to square the latter without changing the inequality one needs to establish that \rho - E[\rho_{lin}] >= 0. Otherwise, it is possible that | \rho - E[\rho_{lin}] | > 2 \rho D_b and \rho - E[\rho_{lin}] < 0.
# Appendix B: Eq. (6)
- A proof of the left-hand side inequality is incomplete, E[(Q(Z) - Z)(Q(Z') - Z')] >= 0. At the moment the term is just expanded and it is claimed that the expansion is a fact. If so, please provide a reference for this result. Otherwise, an explanation is needed for why it holds that E[ Z Q(Z') ] - E[ Q(Z) Q(Z') ] <= E[ Z Z'] - E[ Z Q(Z') ].
- For the proof of the right-hand side inequality, it is not clear why it holds that E[ Z Z' ] + E[ Q(Z) Q(Z') ] - 2 E[ Z Q(Z') ] <= E[ Z Z' ] - E[ Z Q(Z') ].
### My decision is conditional on these remarks being addressed properly during the rebuttal phase. ###
********************************
* Minor comments 	       *
********************************
- line 79: bracket is missing ( || z ||^2 + || z' ||^2 ) / k
- Appendix A, Eq. (2): the notation is not introduced properly


Differentiable Learning of Submodular Models
Can we incorporate discrete optimization algorithms within modern machine learning models? For example, is it possible to incorporate in deep architectures a layer whose output is the minimal cut of a parametrized graph? Given that these models are trained end-to-end by leveraging gradient information, the introduction of such layers seems very challenging due to their non-continuous output. In this paper we focus on the problem of submodular minimization, for which we show that such layers are indeed possible. The key idea is that we can continuously relax the output without sacrificing guarantees. We provide an easily computable approximation to the Jacobian complemented with a complete theoretical analysis. Finally, these contributions let us experimentally learn probabilistic log-supermodular models via a bi-level variational inference formulation.

This paper proposes a way to differentiate the process of submodular function minimization thus enabling to use these functionals as layers in neural networks. The key insight of the paper consists in the usage of the interpretation of discrete optimization of submodular functions as continuous optimization. As a concrete example the paper studies the CRF for image segmentation and creates and the graphcut layer. This layer is evaluated on the Weizmann dataset for horse segmentation and is reported to bring some improvements.
I generally like the paper very much, find the description of the method clear enough. In particular, I liked the short introduction into submodular functions and their connection to min-norm-point.
I have some comments that might allow to increase the impact of the paper. My comments mostly cover experimental evaluation and related works.
	1. The experimental evaluation presented in Section 6 is a bit disappointing to a practitioner. The results are clearly far below state-of-the-art in image segmentation. To demonstrate the full potential of the new layer, I would recommend to plug the new layer into one of the state-of-the-art systems for image segmentation such as DeepLab (Chen et al., DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs, 2016). I understand that the graphcut layer is applicable only to binary problems, but it would be very interesting to try it e.g. on one class of PASCAL VOC.
	2. In particular, there is an extension of DeepLab (Chandra and Kokkinos, Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFs, ECCV 2016 ) that puts a Gaussian CRF on top of the potentials learned by CNNs. They make the inference layer differentiable by connection using the fact that inference in Gaussian CRFs reduces to solving systems of linear equations. It would be very interesting to see how the graph layer compares against the Gaussian CRF layer.
	3. It would make sense to mention another line of works that incorporate discrete optimization into networks. In particular, if the final loss can be computed directly from the results of discrete optimization (for example, when the whole system is trained with a structured SVM objective: Jaderberg et al., Deep structured output learning for unconstrained text recognition, ICLR 2015; Vu et al., Context-aware CNNs for person head detection, ICCV 2015). Comparison to this approach can also strengthen the paper.
	4. As mentioned in line 65 a popular way of embedding algorithms into neural networks consists in unrolling a fixed number of steps of iterative algorithms into layers of neural network. This paper uses one of such iterative algorithms (a total variation solver) to do inference, so it would make sense to simply backprop through several iterations of it. Again, comparison to this approach would strengthen the paper.
	5. Are there any other examples (more rich than graphcuts) of the cases when minimization of submodular functions can be plugged into neural networks? Even naming such cases together with the appropriate submodular solvers would be very valuable.
	6. In terms of the potentials learned by the graphcut layer, it would be very interesting to visualize what the network has learned and, e.g., compare those with the standard potentials based on the gradient of the image.
Minor comments:
	- Line 130 says "the Lovasz extension is linear in O", but O is a set and it is clear what the phrase means.
	- Line 135. [27] looks like the wrong reference
	- Line 143. The definition of optimal partition is never given, so it remains unclear what it is.
	- Line 295 says that only a small fraction of labelled pixels was used for training. It is not clear why this is done.


Visual Reference Resolution using Attention Memory for Visual Dialog
Visual dialog is a task of answering a series of inter-dependent questions given an input image, and often requires to resolve visual references among the questions. This problem is different from visual question answering (VQA), which relies on spatial attention (a.k.a. visual grounding) estimated from an image and question pair. We propose a novel attention mechanism that exploits visual attentions in the past to resolve the current reference in the visual dialog scenario. The proposed model is equipped with an associative attention memory storing a sequence of previous (attention, key) pairs. From this memory, the model retrieves the previous attention, taking into account recency, which is most relevant for the current question, in order to resolve potentially ambiguous references. The model then merges the retrieved attention with a tentative one to obtain the final attention for the current question; specifically, we use dynamic parameter prediction to combine the two attentions conditioned on the question. Through extensive experiments on a new synthetic visual dialog dataset, we show that our model significantly outperforms the state-of-the-art (by ≈ 16 % points) in situations, where visual reference resolution plays an important role. Moreover, the proposed model achieves superior performance (≈ 2 % points improvement) in the Visual Dialog dataset [1], despite having significantly fewer parameters than the baselines.

This paper proposed a visual reference resolution model for visual dialog. The authors proposed to attentions, 1: tentative attention that only consider current question and history, and 2: relevant attention that retrieved from an associate attention memory. Two attentions are further combined with a dynamic parameter layer from [9] and predict the final attention on image. The authors create MNIST Dialog synthetic dataset which model the visual reference resolution of ambiguous expressions. The proposed method outperform baseline with large margin. The authors also perform experiments on visual dialog dataset, and show improvements over the previous methods.
[Paper Strengths]
1: Visual reference resolution is a nice and intuitive idea on visual dialog dataset.
2: MNIST Dialog synthetic dataset is a plus.
3: Paper is well written and easy to follow.
[Paper Weaknesses]
My major concerns about this paper is the experiment on visual dialog dataset. The authors only show the proposed model's performance on discriminative setting without any ablation studies.  There is not enough experiment result to show how the proposed model works on the real dataset. If possible, please answer my following questions in the rebuttal.
1: The authors claim their model can achieve superior performance having significantly fewer parameters than baseline [1]. This is mainly achieved by using a much smaller word embedding size and LSTM size. To me, it could be authors in [1] just test model with standard parameter setting. To backup this claim, is there any improvements when the proposed model use larger word embedding, and LSTM parameters?
2: There are two test settings in visual dialog, while the Table 1 only shows the result on discriminative setting. It's known that discriminative setting can not apply on real applications, what is the result on generative setting?
3: To further backup the proposed visual reference resolution model works in real dataset, please also conduct ablation study on visDial dataset. One experiment I'm really interested is the performance of ATT(+H) (in figure 4 left). What is the result if the proposed model didn't consider the relevant attention retrieval from the attention memory.


Attend and Predict: Understanding Gene Regulation by Selective Attention on Chromatin
The past decade has seen a revolution in genomic technologies that enabled a flood of genome-wide profiling of chromatin marks. Recent literature tried to understand gene regulation by predicting gene expression from large-scale chromatin measurements. Two fundamental challenges exist for such learning tasks: (1) genome-wide chromatin signals are spatially structured, high-dimensional and highly modular; and (2) the core aim is to understand what the relevant factors are and how they work together. Previous studies either failed to model complex dependencies among input signals or relied on separate feature analysis to explain the decisions. This paper presents an attention-based deep learning approach, AttentiveChrome, that uses a unified architecture to model and to interpret dependencies among chromatin factors for controlling gene regulation. AttentiveChrome uses a hierarchy of multiple Long Short-Term Memory (LSTM) modules to encode the input signals and to model how various chromatin marks cooperate automatically. AttentiveChrome trains two levels of attention jointly with the target prediction, enabling it to attend differentially to relevant marks and to locate important positions per mark. We evaluate the model across 56 different cell types (tasks) in humans. Not only is the proposed architecture more accurate, but its attention scores provide a better interpretation than state-of-the-art feature visualization methods such as saliency maps.
DNA. These spatial re-arrangements result in certain DNA regions becoming accessible or restricted and therefore affecting expressions of genes in the neighborhood region. Researchers have established the "Histone Code Hypothesis" that explores the role of histone modifications in controlling gene regulation. Unlike genetic mutations, chromatin changes such as histone modifications are potentially reversible ( [5]). This crucial difference makes the understanding of how chromatin factors determine gene regulation even more impactful because this knowledge can help developing drugs targeting genetic diseases.
At the whole genome level, researchers are trying to chart the locations and intensities of all the chemical modifications, referred to as marks, over the chromatin 4 . Recent advances in nextgeneration sequencing have allowed biologists to profile a significant amount of gene expression and chromatin patterns as signals (or read counts) across many cell types covering the full human genome. These datasets have been made available through large-scale repositories, the latest being the Roadmap Epigenome Project (REMC, publicly available) ( [18]). REMC recently released 2,804 genome-wide datasets, among which 166 datasets are gene expression reads (RNA-Seq datasets) and the rest are signal reads of various chromatin marks across 100 different "normal" human cells/tissues [18].
The fundamental aim of processing and understanding this repository of "big" data is to understand gene regulation. For each cell type, we want to know which chromatin marks are the most important and how they work together in controlling gene expression. However, previous machine learning studies on this task either failed to model spatial dependencies among marks or required additional feature analysis to explain the predictions (Section 4). Computational tools should consider two important properties when modeling such data.
• First, signal reads for each mark are spatially structured and high-dimensional. For instance, to quantify the influence of a histone modification mark, learning methods typically need to use as input features all of the signals covering a DNA region of length 10, 000 base pair (bp) 5 centered at the transcription start site (TSS) of each gene. These signals are sequentially ordered along the genome direction. To develop "epigenetic" drugs, it is important to recognize how a chromatin mark's effect on regulation varies over different genomic locations.
• Second, various types of marks exist in human chromatin that can influence gene regulation. For example, each of the five standard histone proteins can be simultaneously modified at multiple different sites with various kinds of chemical modifications, resulting in a large number of different histone modification marks. For each mark, we build a feature vector representing its signals surrounding a gene's TSS position. When modeling genome-wide signal reads from multiple marks, learning algorithms should take into account the modular nature of such feature inputs, where each mark functions as a module. We want to understand how the interactions among these modules influence the prediction (gene expression).
In this paper we propose an attention-based deep learning model, AttentiveChrome, that learns to predict the expression of a gene from an input of histone modification signals covering the gene's neighboring DNA region. By using a hierarchy of multiple LSTM modules, AttentiveChrome can discover interactions among signals of each chromatin mark, and simultaneously learn complex dependencies among different marks. Two levels of "soft" attention mechanisms are trained, (1) to attend to the most relevant regions of a chromatin mark, and (2) to recognize and attend to the important marks. Through predicting and attending in one unified architecture, AttentiveChrome allows users to understand how chromatin marks control gene regulation in a cell. In summary, this work makes the following contributions:
• AttentiveChrome provides more accurate predictions than state-of-the-art baselines. Using datasets from REMC, we evaluate AttentiveChrome on 56 different cell types (tasks).
• We validate and compare interpretation scores using correlation to a new mark signal from REMC (not used in modeling). AttentiveChrome's attention scores provide a better interpretation than state-of-the-art methods for visualizing deep learning models. • AttentiveChrome can model highly modular inputs where each module is highly structured.
AttentiveChrome can explain its decisions by providing "what" and "where" the model has focused 4 In biology this field is called epigenetics. "Epi" in Greek means over. The epigenome in a cell is the set of chemical modifications over the chromatin that alter gene expression. on. This flexibility and interpretability make this model an ideal approach for many real-world applications.
• To the authors' best knowledge, AttentiveChrome is the first attention-based deep learning method for modeling data from molecular biology.
In the following sections, we denote vectors with bold font and matrices using capital letters. To simplify notation, we use "HM" as a short form for the term "histone modification".

The paper presents a novel method for predicting gene regulation by LSTM with an attention mechanism. The model consists of two levels, where the first level is applied on bins for each histone modifications (HM) and the second level is applied to multiple HMs. Attention mechanism is used in each level to focus on the important parts of the bins and HMs. In the experiments, the proposed method improves AUC scores over baseline models including CNN, LSTM, and CNN with an attention mechanism. This is an interesting paper which shows that LSTM with an attention mechanism can predict gene regulation.
1. It is unclear to me why the second level is modeled using LSTM because there is no ordering between HMs. Would it be reasonable to use a fully connected layer to model dependencies between HMs? Based on Table 1 and 2, the one level model (LSTM-\alpha) outperforms the two level model ((LSTM-\alpha,\beta) in many cases. It would be interesting to investigate how HMs are coupled with each other in the learned model.
2. In Section 2, an algorithm box that includes the equations for the entire model would be very helpful.
3. It is unclear to me if DeepChrome was compared with the proposed method in the experiments. It would be helpful to indicate which model corresponds to DeepChrome.
4. As baselines, it would be helpful to include non-neural network models such as SVM or logistic regression.
5. To allow researchers in biology to use this method, it would be very helpful to have the source code publicly available with URLs in the paper.