Fast Bidirectional Probability Estimation in Markov Models
We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in steps after starting from a given source distribution. Given the target state t, we use a (reverse) local power iteration to construct an 'expanded target distribution', which has the same mean as the quantity we want to estimate, but a smaller variance -this can then be sampled efficiently by a Monte Carlo algorithm. Our method extends to any Markov chain on a discrete (finite or countable) state-space, and can be extended to compute functions of multi-step transition probabilities such as PageRank, graph diffusions, hitting/return times, etc. Our main result is that in 'sparse' Markov Chains -wherein the number of transitions between states is comparable to the number of statesthe running time of our algorithm for a uniform-random target node is order-wise smaller than Monte Carlo and power iteration based algorithms; in particular, our method can estimate a probability p using only O(1/ √ p) running time.

The paper introduces a new algorithm for estimating multi-step transition probabilities (MSTP) for (discrete) Markov chains, and also presents some analysis of the algorithm.The main theoretical results are (1) a bound on the relative error of the (unbiased) estimate -- this is based on concentration bounds, and (2) a bound on the runtime for sparse problems.For (2), the interesting result is that the runtime of O(sqrt(1/delta)), where delta is the threshold above which we'd like to estimate MSTPs. The dependence with respect to sparsity is on the average number of neighbors (rather than an upper bound on the number of neighbors). This work appears to build directly on the recent FAST-PPR algorithm (KDD 2014) -- which also achieves runtime O(sqrt(1/delta)) but was designed specifically for PageRank.            Section 3 devotes a page to a bulleted list of potential applications of MSTP estimation.It feels a bit slapped on, as if to flesh out the paper, but some readers may appreciate this section.It would be useful here to explicitly distinguish between applications where single MSTP values are of interest and those that call for many or all such values.This information would complement the experiments Section 4.            I wasn't a fan of the experiments section. The experiments focus exclusively on runtime, but this is not very interesting unless complemented by results for accuracy. It is great that you compare 3 algorithms on 4 real datasets, but you are missing critical experimental results here. I'm also on the fence about whether the comparisons are fair.As the authors note, the other algorithms "were designed for community detection, and return scores from the source to all targets, rather than just one target."So, it is indeed interesting that the proposed algorithm here beats the others on a single target (or more generally, a small number of targets).This suggests that to round out a fair comparison, you might also report how long the bidirectional algorithm would require on all targets.Alternatively or additionally, you might evaluate a parallel implementation of your algorithm (i.e., the dead-simple embarrassingly parallel thing).E.g., how many parallel cores would you need to beat the other algorithms on all targets?     - For reasons of interpretability, it would be useful to also report some basic stats for the various datasets in a table -- number of nodes, average degree.     - The bar graphs (Figure 2) on a log scale in units of ms are a bit difficult to interpret. I would consider a table in units of seconds.      Minor comments:     - "order-wise" might be more common     - Some hyphens unnecessary or incorrect:Monte-Carlo, transition-probabilities, bidirectional-MSTP     - No line break in the middle of "PageRank" (page 2)     - Some of the hyphens in Algorithms seem unnecessary:estimate-vectors, residual-vectors, sample-path(s)     - Define c as an input to Algorithm 2 (?)     - Avoid using red + green in figures, and it would be nice if figures worked in grayscale     - "we defer it to our full version" -- you mean supplement?     - Use ~ to connect a citation to the previous word      Section 3 looks hastily assembled:     - Could really use some white space (e.g., between bullets)     - For random walks on a (weighted) graph, such functions are called...     - Missing some periods under "Graph Diffusions"     - Footnote 2 (subscript following an integer) is very awkward            Section 4:     - state-of-the-art


Backpropagation for Energy-Efficient Neuromorphic Computing
Solving real world problems with embedded neural networks requires both training algorithms that achieve high performance and compatible hardware that runs in real time while remaining energy efficient. For the former, deep learning using backpropagation has recently achieved a string of successes across many domains and datasets. For the latter, neuromorphic chips that run spiking neural networks have recently achieved unprecedented energy efficiency. To bring these two advances together, we must first resolve the incompatibility between backpropagation, which uses continuous-output neurons and synaptic weights, and neuromorphic designs, which employ spiking neurons and discrete synapses. Our approach is to treat spikes and discrete synapses as continuous probabilities, which allows training the network using standard backpropagation. The trained network naturally maps to neuromorphic hardware by sampling the probabilities to create one or more networks, which are merged using ensemble averaging. To demonstrate, we trained a sparsely connected network that runs on the TrueNorth chip using the MNIST dataset. With a high performance network (ensemble of 64), we achieve 99.42% accuracy at 108 µJ per image, and with a high efficiency network (ensemble of 1) we achieve 92.7% accuracy at 0.268 µJ per image.

The authors present an interesting paper about a backpropagation training method using spike probabilities and takes into account the hardware constraints of a particular platform which has spiking neurons and discrete synapses. This is a topic of current interest in mapping deep networks to multi-neuron hardware platforms.Quality: The proposed training method is useful especially in consideration of new multi-neuron hardware platforms with constraints. Clarity: The paper is easy to read. The claims made at the end of Section 1 can be reworded because as it stands, if not read carefully, suggests that the paper proposes for the first time a training methodology that employs spiking neurons, synapses with reduced precision. I would not necessarily put the demonstration of running this network on a specific hardware platform, in this case TN, as a novel feature to be included in this case. Running the network on TN is a validation of the training method.In Section 2, what does it mean for 0.15 bits per synapse? The network topology is not clearly described in one place. How much longer is the training time because of the probabilistic synaptic connection update? Why is the result from a single ensemble of the 30 core network so low especially when compared to the 5 core network? Why are the results of the different ensembles of the 5 core network about the same? What are the spike rates for the inputs during testing? Is the input a spike train? Ref 12 on line 399 includes constraints (e.g. bias=0) during training so it is not just a train-then-constrain alone approach (typo on this line, "approach approach").Originality: Can the authors provide a discussion or comparison with other spiking backprop-type rules like SpikeProp? Significance: The development of new training methods that consider constraints of hardware platforms is of great interest, The constraints which are considered during training in this paper are based on the TN architecture, these constraints might not all apply to other platforms. If the results are based on a 64 ensemble, that means more hardware resources have to be dedicated to obtain the accuracy needed of the network.


Shepard Convolutional Neural Networks
Deep learning has recently been introduced to the field of low-level computer vision and image processing. Promising results have been obtained in a number of tasks including super-resolution, inpainting, deconvolution, filtering, etc. However, previously adopted neural network approaches such as convolutional neural networks and sparse auto-encoders are inherently with translation invariant operators. We found this property prevents the deep learning approaches from outperforming the state-of-the-art if the task itself requires translation variant interpolation (TVI). In this paper, we draw on Shepard interpolation and design Shepard Convolutional Neural Networks (ShCNN) which efficiently realizes endto-end trainable TVI operators in the network. We show that by adding only a few feature maps in the new Shepard layers, the network is able to achieve stronger results than a much deeper architecture. Superior performance on both image inpainting and super-resolution is obtained where our system outperforms previous ones while keeping the running time competitive.

PAPER SUMMARYA recent work [4] proposed using a deep CNN for super-resolution.Although it generated impressive results, it could not outperform a state-of-the-art super-resolution method based on sparse coding [10].The authors of this paper argue that this limitation is attributed to the inability of the CNN model in [4] to handle translation variant interpolation (TVI).To address this problem, they propose using an old interpolation method to develop a CNN variant called Shepard convolutional neural network (ShCNN).ShCNN is compared with other methods on inpainting and super-resolution tasks.CRITERION 1: QUALITYIt is widely perceived by many researchers that the translation-invariant property of conventional CNN, though promising for tasks such as robust object recognition, may not be very suitable for many other computer vision and image processing tasks.For example, in the context of visual tracking (not even a low-level image processing task), it was pointed out by Wang and Yeung in "Learning a deep compact image representation for visual tracking" (NIPS 2013) that a translation-variant CNN is needed for tracking applications.Likewise, it is reasonable to put forward the working hypothesis that introducing TVI to CNN would help to improve the results for inpainting and super-resolution tasks.Nevertheless, the authors have not provided justification on why Shepard interpolation is adopted here.Shepard interpolation is essentially a kernel that imposes an inverse distance weighting function.What is so special about Shepard interpolation that it is chosen here instead of many other kernel choices?Does it have computational advantages, or representational advantages, or both?Justification is missing in the paper.While it is reasonable to use masks for the inpainting task, is super-resolution with masks also a reasonable problem setting?The proposed method is evaluated empirically using a quantitative performance measure (PSNR).There is no discussion on the weaknesses of the proposed method or even analysis of some failure cases.Since ShCNN adds one or two interpolation layers on top of the original CNN, a more fair comparison is to add the same number of conventional convolutional layers.CRITERION 2: CLARITYAlthough the proposed extension to CNN is relatively simple, the paper presentation falls short of describing the proposed changes clearly and precisely.Since the paper has not reached the page limit of 9 pages, there should be plenty of space to provide more details about the model and algorithm.For example, exactly how the convolution form can be rewritten equivalently based on matrix multiplications using the unrolled matrices should beprovided in greater depth in Section 4.1.How can weight sharing be achieved using the equivalent scheme?The data preprocessing procedure in the experiments is also confusing.It seems that some pixels of the high-resolution images are kept unchanged when generating the low-resolution images and they are used to specify the masks for ShCNN.If this is the case, it may raise a fairness concern as the other methods compared (e.g., [4]) do not use any mask information.With the mask information, ShCNN somehow already knows part of the "ground truth" of the super-resolved images.Confusion is also caused by some problems with the paper writing and formatting.For example, on line #071: "... learn both translational variant operations ..." Since "both" refers to two things, what is the second thing besides "translational variant operations"?Or do you mean such operations for inpainting and super-resolution?Also, in the first paragraph on page 4, is there any difference between M and \mathbf{M}?Or is it simply a formatting error?It is very confusing to use both regular and bold fonts for matrices (e.g., I and \mathbf{K}).Moreover, sometimes it is switched between the two font styles (e.g., \mathbf{K} and K (line #240)).On the last line of page 4, what is UM?Also, what are the parameters needed for specifying the kernel \mathbf{K}?As said in the beginning of Section 5, experiments have been conducted on both inpainting and super-resolution tasks.However, the subsequent results presented seem to be for super-resolution only.To make it less cluttered, the unit "dB" can be left out in the table entries but only mentioned (once) in the figure caption.Besides, the writing has much room for improvement.Here are just a few of the many examples of language errors: #029: "outperfromed" should be written as "outperformed" #035: "In the past a few years" should be written as "In the past few years" #039: "Encouraging results has" should be written as "Encouraging results have" #044: "when applying to" should be written as "when applied to" and many more (particularly in later sections).There are also some formatting problems.For example, there should be a space before a citation, i.e., "in [8]" but not "in[8]".This problem happens a few times.CRITERION 3: ORIGINALITYShepard interpolation is an old method proposed in 1968.Since the optimization variables are in both the numerator and denominator of the fraction in Eq (1), a more sophisticated learning algorithm (as opposed to applying brute-force back-propagation) would help to handle this highly nonlinear optimization problem better.Unfortunately such an exploration is missing.In its current form, the originality of this paper appears minimal.On the problem of the previous methods discussed in the last paragraph of Section 3.1 and illustrated in Figure 2, actually it is discussed in [5] that K-SVD can already solve this problem.It should be mentioned in this paper.The difference is mainly between blind image inpainting and non-blind image inpainting.In the context of existing work, it weakens the claimed contribution of this paper.CRITERION 4: SIGNIFICANCEThe paper, especially its early parts, makes some claims that seem a bit too general.Is the proposed CNN variant for all TVI operations?Since experimental evaluation has only been done on super-resolution, it is probably more appropriate to make the claim for this particular task only.Super-resolution is always difficult to evaluate.It has been found and reported by researchers before that a method that gives higher PSNR may not give visually better result as judged by humans.One example is that if a method focuses only on the edges (primal sketch) by increasing their resolution, the super-resolved images may look better than those obtained by another method giving higher PSNR.For a new super-resolution method to bring about significant contribution, it is worth supplementing quantitative comparison with qualitative comparison involving humans.This possibility is now more feasible with the emergence of crowdsourcing platforms.


Learning From Small Samples: An Analysis of Simple Decision Heuristics
Simple decision heuristics are models of human and animal behavior that use few pieces of information-perhaps only a single piece of information-and integrate the pieces in simple ways, for example, by considering them sequentially, one at a time, or by giving them equal weight. We focus on three families of heuristics: single-cue decision making, lexicographic decision making, and tallying. It is unknown how quickly these heuristics can be learned from experience. We show, analytically and empirically, that substantial progress in learning can be made with just a few training samples. When training samples are very few, tallying performs substantially better than the alternative methods tested. Our empirical analysis is the most extensive to date, employing 63 natural data sets on diverse subjects.

The authors evaluate a number of decision rules from the ABC research group. They first analyze, through simulations, that the "cue direction" and "learning to order cue" heuristics can provide accurate performance with even just a few samples. Next, they evaluate a number of heuristics on 63 data sets and demonstrate that some of them achieve performance close to an additive logistic regression and CART, with default parameter settings.This is a well-written paper containing two substantial results that I believe will be of interest to a broad community at NIPS. I have a few main criticisms regarding the submission (in order of appearance). The first result they have (few samples suffice for some heuristics) is very similar in nature to a previous result by Vul, Goodman, Griffiths and Tenenbaum (2009; 2014). Although the results are not identical, I did not find the simulations terribly surprising or impressive given this previous work. Also, I am confused why the simulations discuss cue direction and informative sampling when neither are discussed nor used later in the paper. The other main criticism is concerned with the models used as comparisons to evaluate the performance of the heuristics on the 63 data sets. A simple logistic regression where each cue is an independent input is not that powerful of a statistical model. What if pairs and triplets are included as well? What about support vector machines/kernel methods on these data sets? I know this increases the computational complexity of the comparison models, but if the argument is that heuristics are close to the best models, than the best models should really be powerful. I was more impressed with CART, but the fact that the default settings were used with no exploration made me concerned that CART is underperforming because there is no tuning whatsoever to each data set. If the best parameters are chosen for CART on each data set, how well would it do? What are the best parameters for CART that do the best across all data sets?Minor concerns: Line 131: it might be worth mentioning that e1 is simply the CDF of a binomial distribution. Figure 1: The text is too small in many places and I found the subplots difficult to interpret. Also, the scale of the colormap for the bottom left subplot is different than the other subplots. Line 201-202: "..., but the in the majority" -> "..., but in the majority" Line 219: Wouldn't it be more appropriate to use ordinal logistic regression, (polyr in the MASS library of R), rather than turn an ordinal factor into a nominal factor? Line 258-259: "with upto 3000 repetitions." -> "with up to 3000 repetitions."Vul, E., Goodman, N., Griffiths, T., & Tenenbaum, J. (2014). One and Done? Optimal Decisions from Very Few Samples. Cognitive Science, 38, 599-637.Vul, E., Goodman, N., Griffiths, T., & Tenenbaum, J. (2009). One and done? Optimal decisions from few samples. In N. Taatgen, H. van Rijn, L. Schomaker, & J. Nerbonne (Eds.), Proceedings of the 31st annual conference of the cognitive science society. Amsterdam, the Netherlands.Edits based on author feedback: I'm glad to hear the initial review was helpful. The paper is a really great contribution and your feedback increases my confidence in that assessment. I apologize regarding the confusion with "pairs and triplets". What I mean is including interaction terms between your predictors in the logistic regression. Perhaps you did that and I misunderstood what you did in the manuscript? If so, it's probably worth revising the section on logistic regression a bit to make this clear.


Texture Synthesis Using Convolutional Neural Networks
Here we introduce a new model of natural textures based on the feature spaces of convolutional neural networks optimised for object recognition. Samples from the model are of high perceptual quality demonstrating the generative power of neural networks trained in a purely discriminative fashion. Within the model, textures are represented by the correlations between feature maps in several layers of the network. We show that across layers the texture representations increasingly capture the statistical properties of natural images while making object information more and more explicit. The model provides a new tool to generate stimuli for neuroscience and might offer insights into the deep representations learned by convolutional neural networks.

In this submission, the authors develop a new texture synthesis algorithm, that begins with the ideas and general methodology of the classical Portilla-Simoncelli texture synthesis algorithm, and augments it with the rich feature set provided by convolutional neural nets. This procedure overcomes a major limitation of Portilla-Simoncelli -- its dependence on a hand-crafted feature set. The convnets, while trained on some other supervised objective (eg classification on imagenet), provide a possible way forward. The authors demonstrate compelling texture synthesis, and some simple and intuitive observations of how generated textures depend on choices of architecture and layer.For clarity, originality, and significance, this paper gets a thumbs up. For quality, I've given this paper a 6, but I would certainly bump it up higher if the authors convincingly address and fix a few crucial points. In no particular order:(1) At the moment, the paper is framed in terms of its potential for studying the ventral stream. I can see why this is a tempting proposition, given how Portilla-Simoncelli has fed into to the Freeman et al V2 studies. However, the authors aren't really doing anything in this direction in the body of the paper. The closest they come to perception is in Section 6, wherein they tack a classifier onto the summary statistics collected at a particular layer of VGG. Yet the improvement here with depth seems more to me a statement about the representation that VGG is learning in order to perform well on imagenet, and less about textures and the brain. This could end up a useful tool for studying the internal representations in deep nets, but the connection to the ventral stream is too vague. I think it would be better off if this link was less central, and made into a discussion point, with the texture synthesis in its own right being the focus of the paper. If the authors wish to propose that the summary statistics collected at different layers of a convnet will be useful for studying hierarchical representations in the brain, then they must acknowledge that these two architectures engage different tradeoffs between scale and complexity. By scale here I mean the size of the receptive field; by complexity, I mean the degree of nonlinearity. In a convnet like VGG, these two factors are correlated: as one goes deeper, units are integrating over a larger spatial region, and have cumulatively computed a more complex nonlinearity. In the ventral stream, this is not the case: as early as V1, the representation is multiscale, including simple cells with lower spatial frequency tuning (i.e. larger RFs). Portilla-Simoncelli is useful as a tool for probing the V1->V2 transformation precisely because it parallels the multiscale feature of the physiology; a convnet is designed with different architectural constraints that may make it less analogous to physiology in this regard. (2) The authors make a point that the features in Portilla-Simoncelli are hand-crafted, but those proposed in this paper are not. This is mostly true, *except* the choice of the correlation summary statistic *is* hand-crafted. There is no explicit reason for them to choose this, other than it works. For the record, the term "correlation" is misleading here (perhaps in signal processing terms it's a correlation, but it's not normalised, so I would have described it as a covariance). (3) It's really important to tell us how many (cumulative) summary statistics are being enforced. If I get the numbers right (from inspecting VGG), by pool1 there are 3 * 64*63/2 = 6048 parameters collected, this goes up to 30k by pool2, 194k by pool3, 0.8M by pool4, and 1.5M by pool5. These numbers should go in Figure 2. That's a *lot* of parameters (especially when the number of pixels is ~50k?) and it makes it hard to interpret these as "summary" statistics. The authors should comment on what this means. (4) I'm at a loss as to why conv5 and/or pool5 are not shown in Figure 2. They're clearly part of VGG, and it's weird that they are omitted without mention. (5) I'm confused as to why the yellow boxes in the bottom left hand corner of each P-S source texture are showing up in the bottom left hand corner of each synthesised image. If the statistics are stationary, and the seed is white noise, then there's no reason for this feature to be generated precisely here. This looks like a boundary condition artefact. If boundaries are handled in some special way, or ignored, we need to know. (6) It's pretty topical, so the authors could relate their approach to the recent google inception/deepdream syntheses to make it a more attractive paper. Clearly this work was developed in parallel, and the overall objectives are different. But there is an overlap. For example, my understanding of the deepdream algorithm is that it places pressure on a summary statistic of the current image, namely the variance of one feature map. This is precisely one of the statistics being collected in the algorithm presented here.  (7) As one goes deeper in the net, the texture model is holding on to all the summary statistics computed in previous layers, so the representation is cumulative. This seems like an important point, and yet it is not clearly justified.  ** Following author rebuttal: with less emphasis on the biological link, and the authors' proposed amendments, this is an interesting paper. I don't agree with all the interpretations, but it's discussion-provoking and relevant to many communities.


Fast Distributed k-Center Clustering with Outliers on Massive Data
Clustering large data is a fundamental problem with a vast number of applications. Due to the increasing size of data, practitioners interested in clustering have turned to distributed computation methods. In this work, we consider the widely used kcenter clustering problem and its variant used to handle noisy data, k-center with outliers. In the noise-free setting we demonstrate how a previously-proposed distributed method is actually an O(1)-approximation algorithm, which accurately explains its strong empirical performance. Additionally, in the noisy setting, we develop a novel distributed algorithm that is also an O(1)-approximation. These algorithms are highly parallel and lend themselves to virtually any distributed computing framework. We compare each empirically against the best known sequential clustering methods and show that both distributed algorithms are consistently close to their sequential versions. The algorithms are all one can hope for in distributed settings: they are fast, memory efficient and they match their sequential counterparts.

The 4-approximation is surprisingly simple: simply compute the k-center (approximately) in each machine, collect all such centers in a single machine, and then compute the k-center for these centers. The proof is also very simple. I am not sure if this simple argument has been found (or implied) from previous work. But if not, it is definitely a simple, yet nice and cute new result.The algorithm for the outlier version is somewhat harder. Again, compute the k-center (approximately) for the points in each machine, and collect all such centers in a single machine. Now, each such center can be through as a representative of several original points (by assigning a proper weight). Then the machine with all such centers runs an algorithm similar to Charikar et al. The analysis of the approximation ratio requires some effort. Overall, the results of the paper are nice. The theoretical depth is ok, yet not terribly deep nor exciting. I would rate it as a weak accept (7).Some minor comments:The idea of creating weighted point is somewhat similar to the recent work for a generalization of the outlier k-center problem. Chen, Danny Z., et al. "Matroid and knapsack center problems."Integer Programming and Combinatorial Optimization. Springer Berlin Heidelberg, 2013. 110-122. The work is in the very same spirit of several recent work on distributed and composable coreset, in which the idea is to compute a coreset on each machine, and then compute the approximate optimal solution for the union of all coresets. These papers are definitely related.Indyk, Piotr, et al. "Composable core-sets for diversity and coverage maximization."Proceedings of the 33rd ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems. ACM, 2014. Barbosa, Rafael, et al. "The Power of Randomization: Distributed Submodular Maximization on Massive Datasets."arXiv preprint arXiv:1502.02606 (2015).


Fast Lifted MAP Inference via Partitioning
Recently, there has been growing interest in lifting MAP inference algorithms for Markov logic networks (MLNs). A key advantage of these lifted algorithms is that they have much smaller computational complexity than propositional algorithms when symmetries are present in the MLN and these symmetries can be detected using lifted inference rules. Unfortunately, lifted inference rules are sound but not complete and can often miss many symmetries. This is problematic because when symmetries cannot be exploited, lifted inference algorithms ground the MLN, and search for solutions in the much larger propositional space. In this paper, we present a novel approach, which cleverly introduces new symmetries at the time of grounding. Our main idea is to partition the ground atoms and force the inference algorithm to treat all atoms in each part as indistinguishable. We show that by systematically and carefully refining (and growing) the partitions, we can build advanced any-time and any-space MAP inference algorithms. Our experiments on several real-world datasets clearly show that our new algorithm is superior to previous approaches and often finds useful symmetries in the search space that existing lifted inference rules are unable to detect.
Markov logic networks (MLNs) [5] allow application designers to compactly represent and reason about relational and probabilistic knowledge in a large number of application domains including computer vision and natural language understanding using a few weighted first-order logic formulas. These formulas act as templates for generating large Markov networks -the undirected probabilistic graphical model. A key reasoning task over MLNs is maximum a posteriori (MAP) inference, which is defined as the task of finding an assignment of values to all random variables in the Markov network that has the maximum probability. This task can be solved using propositional (graphical model) inference techniques. Unfortunately, these techniques are often impractical because the Markov networks can be quite large, having millions of variables and features.
Recently, there has been growing interest in developing lifted inference algorithms [4,6,17,22] for solving the MAP inference task [1,2,3,7,13,14,16,18,19]. These algorithms work, as much as possible, on the much smaller first-order specification, grounding or propositionalizing only as necessary and can yield significant complexity reductions in practice. At a high level, lifted algorithms can be understood as algorithms that identify symmetries in the first-order specification using lifted inference rules [9,13,19], and then use these symmetries to simultaneously infer over multiple symmetric objects. Unfortunately, in a vast majority of cases, the inference rules are unable to identify several useful symmetries (the rules are sound but not complete), either because the symmetries are approximate or because the symmetries are domain-specific and do not belong to a known type. In such cases, lifted inference algorithms partially ground some atoms in the MLN and search for a solution in this much larger partially propositionalized space.
In this paper, we propose the following straight-forward yet principled approach for solving this partial grounding problem [21,23]: partition the ground atoms into groups and force the inference algorithm to treat all atoms in each group as indistinguishable (symmetric). For example, consider a first-order atom R(x) and assume that x can be instantiated to the following set of constants: {1, 2, 3, 4, 5}. If the atom possesses the so-called non-shared or single-occurrence symmetry [13,19], then the lifted inference algorithm will search over only two assignments: all five groundings of R(x) are either all true or all false, in order to find the MAP solution. When no identifiable symmetries exist, the lifted algorithm will inefficiently search over all possible 32 truth assignments to the 5 1
Recently, there has been growing interest in developing lifted inference algorithms [4,6,17,22] for solving the MAP inference task [1,2,3,7,13,14,16,18,19]. These algorithms work, as much as possible, on the much smaller first-order specification, grounding or propositionalizing only as necessary and can yield significant complexity reductions in practice. At a high level, lifted algorithms can be understood as algorithms that identify symmetries in the first-order specification using lifted inference rules [9,13,19], and then use these symmetries to simultaneously infer over multiple symmetric objects. Unfortunately, in a vast majority of cases, the inference rules are unable to identify several useful symmetries (the rules are sound but not complete), either because the symmetries are approximate or because the symmetries are domain-specific and do not belong to a known type. In such cases, lifted inference algorithms partially ground some atoms in the MLN and search for a solution in this much larger partially propositionalized space.
In this paper, we propose the following straight-forward yet principled approach for solving this partial grounding problem [21,23]: partition the ground atoms into groups and force the inference algorithm to treat all atoms in each group as indistinguishable (symmetric). For example, consider a first-order atom R(x) and assume that x can be instantiated to the following set of constants: {1, 2, 3, 4, 5}. If the atom possesses the so-called non-shared or single-occurrence symmetry [13,19], then the lifted inference algorithm will search over only two assignments: all five groundings of R(x) are either all true or all false, in order to find the MAP solution. When no identifiable symmetries exist, the lifted algorithm will inefficiently search over all possible 32 truth assignments to the 5 ground atoms and will be equivalent in terms of (worst-case) complexity to a propositional algorithm. In our approach, we would partition the domain, say as {{1, 3}, {2, 4, 5}}, and search over only the following 4 assignments: all groundings in each part can be either all true or all false. Thus, if we are lucky and the MAP solution is one of the 4 assignments, our approach will yield significant reductions in complexity even though no identifiable symmetries exist in the problem.
Our approach is quite general and includes the fully lifted and fully propositional approaches as special cases. For instance, setting the partition size k to 1 and n respectively where n is the number of constants will yield exactly the same solution as the one output by the fully lifted and fully propositional approach. Setting k to values other than 1 and n yields a family of inference schemes that systematically explores the regime between these two extremes. Moreover, by controlling the size k of each partition we can control the size of the ground theory, and thus the space and time complexity of our algorithm.
We prove properties and improve upon our basic idea in several ways. First, we prove that our proposed approach yields a consistent assignment that is a lower-bound on the MAP value. Second, we show how to improve the lower bound and thus the quality of the MAP solution by systematically refining the partitions. Third, we show how to further improve the complexity of our refinement procedure by exploiting the exchangeability property of successive refinements. Specifically, we show that the exchangeable refinements can be arranged on a lattice, which can then be searched via a heuristic search procedure to yield an efficient any-time, any-space algorithm for MAP inference. Finally, we demonstrate experimentally that our method is highly scalable and yields close to optimal solutions in a fraction of the time as compared to existing approaches. In particular, our results show that for even small values of k (k bounds the partition size), our algorithm yields close to optimal MAP solutions, clearly demonstrating the power of our approach.

The paper considers lifted MAP inference for relational models.More precisely, it proposes to make use of partitions and partition lattices for symmetry-aware MAP inference. Overall, I very much like the direction of the paper. And the experimental results shows that approach works well; some of theresults are actually quite impressive. However, there are also some downsides:(1) The ideas exploited are floating in the literature and this should be acknowledged (2) The baselines in the experimental evaluation are rather weak; at least the rocket system should be used as another baselineLet me clarify this in more details. The paper is too short on related work. The discussion of the related work has tobe extended. At the current stage, it is not clear where thepresented work goes beyond what is known, in particular when it comes to algebraic lifted MAP approaches. The long list of references on the first page is not very informative there. More importantly,it is not even clear what "in a vast majority of cases, the inferencerules are unable to identify several useful symmetries" actually means running the risk of providing false statements about some of the other approaches. That is, the connections are not formally defined so far in the literature and without this, the claim is not proven at all. Moreover, many of the positive claims made about the presented workalso holds for the algebraic approaches as presented in [2, 13, 15]and also Martin Mladenov, Amir Globerson, Kristian Kersting: Lifted Message Passing as Reparametrization of Graphical ModelsUAI 2014Martin Mladenov, Kristian Kersting:Equitable Partitions of Concave Free Energies.UAI 2015All of them show that we can solve the relaxed MAP LPefficiently using (fractional) automorphisms. Plugging them intoa brunch-and-cut/bound ILP solver would result in a very similar if not identical approach, modulo the efficient recompilation of the lifted network and partial grounding. They could becombined with SA hierarchies as e.g. done inUdi Apsel, Kristian Kersting, Martin Mladenov: Lifting Relational MAP-LPs Using Cluster Signatures.AAAI 2014Moreover, the domain approach presented in the present paper appears to be similar if not identical to the domain graph approach presented in some of the above mentioned related approaches. The authors should clarify the differences. Indeed, one of the big differences is the refinement of the lifted during inference. However, much of the lattice approach is alreadypresented in the above mentioned work, indeed from an algebraicperspective and not from a logical perspective, but this is a detail.Moreover, the idea of relifting during inference has been presented already in [7].That is, the paper misses a discussion of related work. This should carefully acknowledge the origins of ideas and in turn showecase the originality of the presented idea. Without, it reads more like putting well-known ideas together. Nevertheless, putting the ideas together into a single system is really nice and scales inference very well.


The Self-Normalized Estimator for Counterfactual Learning
This paper identifies a severe problem of the counterfactual risk estimator typically used in batch learning from logged bandit feedback (BLBF), and proposes the use of an alternative estimator that avoids this problem. In the BLBF setting, the learner does not receive full-information feedback like in supervised learning, but observes feedback only for the actions taken by a historical policy. This makes BLBF algorithms particularly attractive for training online systems (e.g., ad placement, web search, recommendation) using their historical logs. The Counterfactual Risk Minimization (CRM) principle [1] offers a general recipe for designing BLBF algorithms. It requires a counterfactual risk estimator, and virtually all existing works on BLBF have focused on a particular unbiased estimator. We show that this conventional estimator suffers from a propensity overfitting problem when used for learning over complex hypothesis spaces. We propose to replace the risk estimator with a self-normalized estimator, showing that it neatly avoids this problem. This naturally gives rise to a new learning algorithm -Normalized Policy Optimizer for Exponential Models (Norm-POEM) -for structured output prediction using linear rules. We evaluate the empirical effectiveness of Norm-POEM on several multi-label classification problems, finding that it consistently outperforms the conventional estimator.

- A question to clarify in the rebuttal: from Section 4.1 in [1], it is said that the CRM approach only makes sense for delta in [-1,0] (and thus they propose to rescale a general loss to [-1,0]). My guess is that it should be the same for NORM-POEM here -- i.e. (8) and (3) both uses the re-scale delta in [-1,0], correct? In this case, I do not understand what it means to use "delta > 0" in the experiment in Table 1. Please clarify.== Other comments ==- There is something I find missing in the setup for "batch learning from logged bandit feedback" of Section 3 (or from [1]): I feel the feedback should also be seen as a random variable, rather than a deterministic function as presented on line 110. Here is my rationale. Suppose that the partial feedback is coming from a classification problem where we just do not know the labels, we only get feedback by trying a prediction and seeing what is the loss. Suppose that there is a true classification loss Delta(y',y), and that the way the feedback is generated is, for a given input x, we make a prediction y using h_0; some god agent labels also the input to y'; and then tells us the loss Delta(y',y) that we incur (so here delta(x,y) is Delta(y',y) for the y' given to this example). On the other hand, often in classification, we suppose that the labeling can be noisy and so there is not necessarily a unique y' assigned to each x. This means that somewhere else in the log, we could have the same x as input, the same y that we played, but with a different delta(x,y) as the god agent just had given a different label for this one... This is why I think that in general delta(x,y) should be seen as a random number (in the classification example above the mean of this random variable for a fix x & played y should be the expectation of Delta(y',y) for y' distributed according to p(y'|x), the true noisy labeling distribution).This perhaps does not change anything about the risk estimator (2); but at least the setup in (1) with the true risk should be presented with an additional expectation over the randomness of delta, to be more general.- Line 170-171: I think it should be h(y|x) / h_0(y|x). Also, it might be worthwhile to mention that the lack of linearity was already pointed out in [1].- Line 276: you should probably put argmin instead of argmax given that you are minimizing the risk...- Line 319: replace bars with hats in the notation to be consistent with the previous convention. === Update after rebuttal ==Thanks for the clarifications.


Deep Convolutional Inverse Graphics Network
This paper presents the Deep Convolution Inverse Graphics Network (DC-IGN), a model that aims to learn an interpretable representation of images, disentangled with respect to three-dimensional scene structure and viewing transformations such as depth rotations and lighting variations. The DC-IGN model is composed of multiple layers of convolution and de-convolution operators and is trained using the Stochastic Gradient Variational Bayes (SGVB) algorithm [10]. We propose a training procedure to encourage neurons in the graphics code layer to represent a specific transformation (e.g. pose or light). Given a single input image, our model can generate new images of the same object with variations in pose and lighting. We present qualitative and quantitative tests of the model's efficacy at learning a 3D rendering engine for varied object classes including faces and chairs.

Summary: the paper proposes a CNN for learning explicit image representations as an inverse graphics problem.The image representation has interpretable explicit representations, in particular pose angles and lighting angles, along with implicit representations (texture, appearance).This is done in an autoencoder framework with reconstruction error.To make a particular latent dimension focus on one aspect (e.g. azimuth), in each mini-batch, training data is presented with only variations in one aspect, while activations in other nodes are surpressed to a fixed value (the mean value).Experiments on two datasets showing reconstructions of a 3D object at varying poses and illumination directions.Quality: The methodology is fine, but some interesting cases are not tested in the experiments. - the pose variables are trained independently.How well can they be used in conjunction?For example, changing elevation and azimuth at the same time? - L344-348 - I would have liked to see an actual quantitative evaluation about how well the actual pose could be estimated.The GT pose and pose variables are in different spaces, correlation between the GT and inferred could be calculated.Or a simple transformation function from one space to the other could be learned. - Sec 4.1.1 - For the comparison with a standard network, the latent variable with the largest variance is used as the surrogate for azimuth (L359).It would be interest to also use the linear subspace with the largest variance (i.e., PCA on the latent vector space), since it is possible the azimuth is encoded in multiple latent variables. - no comparisons to other inverse-graphics models [10,17,16,12].Originality: appears to be original.The training method for specializing a node for a particular transformation appears novel.Clarity: - in (1) and (2), why is the mean and variance coupled? i.e., \Sigma = diag(exp(\mu)) - Sec 4.1, it's unclear whether the "static test image" are the same faces as those in the training set. - L376: The error 2.7722e-4 is misleading.The images in Fig 7 have a lot of white space, which are easy to predict and artifically decreases the error rate.The error rate should be for only the pixels in the bounding box of the object.Significance: The paper is interesting, but I'm left to wonder how significant is the result.What is the end-goal of such a network?If it is for viewing objects at different viewpoints, it seems limited since the quality of the images is poor and low resolution (looking at Fig 7).If it is for learning the mapping from image to graphics code (pose parameters), then this is not evaluated in the experiments so it's unclear its advantage over directly predicting the pose parameters with a CNN.


End-to-end Learning of LDA by Mirror-Descent Back Propagation over a Deep Architecture
We develop a fully discriminative learning approach for supervised Latent Dirichlet Allocation (LDA) model using Back Propagation (i.e., BP-sLDA), which maximizes the posterior probability of the prediction variable given the input document. Different from traditional variational learning or Gibbs sampling approaches, the proposed learning method applies (i) the mirror descent algorithm for maximum a posterior inference and (ii) back propagation over a deep architecture together with stochastic gradient/mirror descent for model parameter estimation, leading to scalable and end-to-end discriminative learning of the model. As a byproduct, we also apply this technique to develop a new learning method for the traditional unsupervised LDA model (i.e., BP-LDA). Experimental results on three real-world regression and classification tasks show that the proposed methods significantly outperform the previous supervised topic models, neural networks, and is on par with deep neural networks.

=== Update after rebuttal ===I thank the authors for a comprehensive rebuttal and extra experiments. It has addressed most of my concerns, and I have updated my score. The authors should make sure to properly tone down the claims about improved training for LDA (vs. sLDA), as well as mention the perplexity experiment. It seems to me that we do not really understand very well what is happening in these models at this stage; this perplexity experiment is just scratching the surface (and should be presented as such). I am also a bit puzzled by the use of alpha = 1.001 (vs. exactly 1 e.g.). As a prior, there is almost no difference between the two as a prior; but I can see that it changes a lot the MAP. In particular, using alpha = 1 (uniform distribution), the MAP is just maximum likelihood and could have zero entries; whereas any alpha > 1 creates a barrier in the objective function and prevents any theta_k to be too close to zero.This being said, I think this paper makes interesting contributions, and so even though more careful experiments are needed to understand better what is happening, this can be left as future work and people can already build on the ideas introduced here for topic modeling.== additional comments[# of dominant topics experiment] The "# of dominant topics" experiment is interesting and should be included in the final version. As a note, a prior with alpha = 1.001 with K=100 gives on average about 60 dominant topics as a prior (defined as the number of topics to include to get 90% of the mass when sorted in decreasing order). This becomes 45 for alpha = 0.5 and 15 for alpha = 0.1 (which shows that these alpha are still too big to be used when K = 100). This experiment indicates that given the topic learned, the likelihood is still enforcing somewhat sparse topic proportion (I think that for an alpha so close to 1, the MAP should not be too different than the maximum likelihood solution). Though at K=100, one can already see a big difference in sparsity between BP-LDA alpha = 1.001 and Gibbs-LDA alpha=0.5.[perplexity experiment] The authors should also report the perplexity for BP-LDA with alpha=0.5 and alpha=0.1 to control the difference in alpha. They should also try more topics. The reason that the perplexity is becoming worse with more topics is that alpha should change according to the number of topics. In my experience, properly estimating alpha yields a perplexity more robust to the number of topics (and can even get a better fit with more topics). To get some ball park, the alpha's learned by maximum likelihood using variational inference on standard text document datasets are much smaller than these values. For example, on the NIPS dataset from http://ai.stanford.edu/~gal/data.html, they are of the order of 1e-3 for a K=200 topic model. Finally, as I said in my previous review, it would interesting in future work to propagate the backprop to updating the alpha's as well...=== end of update ===This paper makes an interesting contribution to the supervised topic modeling literature. The idea of using backpropagation by unrolling a few steps of an iterative procedure (which computes a fixed point; or maximize some objective; or do some message passing) has been used before in several areas (as the cited [16]), but as far as I know, it has not been used for supervised topic modeling, so this is a fresh outlook, and I like it. It is also interesting that they are able to get significantly better results than linear regression (I hope this is ridge regression!), as usually supervised topic models struggle to improve over standard discriminative approaches. Their results in Figure 2b) where they basically do the same as logistic regression (within the variation) is more typical (though the other supervised topic model approaches do not do well on this one for some reasons). * Missing experiment:The reason that I am not putting a higher rating is because I think an important experiment is missing given that the authors claim that they their approximate MAP method to learn the parameters of the unsupervised LDA model is "outperforming previous learning methods". The classification / regression experiment is not meaningful for comparing unsupervised LDA learning techniques (it is great for sLDA, and the results are impressive; but unsupervised LDA is meant for an *unsupervised* evaluation!). The authors should thus also report the average test set per-word log-likelihood for LDA with the topic parameters learned by BP-LDA vs. Gibbs-LDA on their dataset. Note that they are definitively not allowed to do the PLSA-like technique of maximizing the posterior over theta on the test data to evaluate its likelihood (this is cheating!) -- they should just use the usual marginalization over theta for LDA. I suggest they use the code from http://homepages.inf.ed.ac.uk/imurray2/pub/09etm/ "Evaluation Methods for Topic Models", Wallach et al. ICML 2009, with some of their best methods. The authors should report these results in their rebuttal -- I am quite curious to see whether the superior prediction performance came at the cost of worse text modeling accuracy. If the authors can report on these results, I am willing to increase my rating (irrespective of whether their perplexity results are good or not).* Novelty claim correction:Using convex optimization to do MAP over theta for LDA is not new: this was already proposed in "Complexity of Inference in Latent Dirichlet Allocation", Sontag & Roy, NIPS 2011 (see Section 3.1). They also had proposed to use the exponentiated gradient algorithm to do the convex optimization, which is equivalent to mirror descent with KL divergence, i.e. yields exactly the same update as (12). The authors should properly correct this novelty claim and mention this prior work.* Using alpha > 1 is not a good text model:I also do not think that using alpha > 1 is a good idea for proper text modeling. It might be fine to get good classification / regression performance (it gives more dense features to the regressor); but if one would also evaluate the generative likelihood for the document, it won't be as good. All the work on topic modeling that I know always found that the individual hyperparameters of the Dirichlet prior over theta should be < 1 to give better perplexity (especially if a large number of topics is used). This was also mentioned in Section 3.2 of the [Sontag & Roy 2011] paper; see also Section 3 of"On Smoothing and Inference for Topic Models", Asuncion et al. UAI 2009. I am willing to bet that if the authors do the perplexity experiment, they could get better results by using alpha < 1 (at least for the Gibbs sampler).Also, note that it was also mentioned before that estimating the hyperparameters of the Dirichlet prior over theta (with asymmetric components) made a big difference for text modeling; see "Rethinking LDA: Why Priors Matter", Wallach et al., NIPS 2009. This was for unsupervised LDA. I am not sure whether it also could make a difference for sLDA -- but given that the authors have now an efficient backpropagation framework to learn all the parameters in a discriminative fashion; it might also be interesting to also backpropagate the gradient to estimate the alpha_k hyperparameters as well.Using alpha > 1 means that a sampled theta is not sparse: this means that a specific document contains all topics with some probability, which certainly seems like a bad generative modeling assumption! I understand that the authors made this assumption because they wanted to claim that their MAP inference over theta was convex. On the other hand, given that they only do a small number L of iterations (and so they certainly do not optimize to convergence), and that the objective is not convex anyway in U and Phi, I do not think that having the inner MAP inference non-convex would be such a big problem. Using a finite number of iterations L is a way to lower bound the smallest value that theta could take in any case (as was suggested in Section 3.2 of [Sontag & Roy 2011]), and so the problem of negative infinities would operationally be avoided. They also seem to report better results for smaller alpha; so I would be curious to see Figure 4 to also report results for alpha < 1.=== other comments ===- Correction line 076: "this paper is the first work to perform a fully end-to-end discriminative training of LDA" -- it should be "of sLDA". They certainly do not do discriminative training of LDA: for LDA, they do an approximate MAP for Phi by removing the marginalization over theta that would normally be required by the model and replacing it with an approximate MAP over theta. This is still generative training, and so should not be called discriminative. In some sense, by doing MAP for theta, they are doing a kind of regularized PLSA model (probabilistic latent semantic analysis), which was the non-Bayesian precursor to LDA. In PLSA, there is a fixed theta per document, found by maximum likelihood. Here, they do regularized maximum likelihood for theta with the Dirichlet prior acting as a regularizer. It is presented as an approximation to LDA; but really, I would see it more as just regularized (potentially supervised) PLSA.- Lines 120-122: the authors should clarify that Blei and McAuliffe had argued that using bar{z} instead of theta to tie the words to the variable to predict y was a better idea given their previous experience with two-signals modeling. I understand that the authors' framework could not handle bar{z} as the input variable for the distribution on y, as it is discrete, justifying their modification, but it is worthwhile to clarify this point. In their experiments, when they talk about "sLDA", do they mean the approach from [3] which used bar{z} as the input variable and also variational inference?- Figure 3 a): are the error bars coming from different folds? What about the variation arising from just different random initialization for this non-convex problem? (Same thing for the sampling approach). Also, what about using a l2-regularizer on U to avoid overfitting for their method? - Lines 396-399: do they use the same number of steps L when computing the test features; or they do full MAP for it?


Winner-Take-All Autoencoders
In this paper, we propose a winner-take-all method for learning hierarchical sparse representations in an unsupervised fashion. We first introduce fully-connected winner-take-all autoencoders which use mini-batch statistics to directly enforce a lifetime sparsity in the activations of the hidden units. We then propose the convolutional winner-take-all autoencoder which combines the benefits of convolutional architectures and autoencoders for learning shift-invariant sparse representations. We describe a way to train convolutional autoencoders layer by layer, where in addition to lifetime sparsity, a spatial sparsity within each feature map is achieved using winner-take-all activation functions. We will show that winner-take-all autoencoders can be used to to learn deep sparse representations from the MNIST, CIFAR-10, ImageNet, Street View House Numbers and Toronto Face datasets, and achieve competitive classification performance.

The authors present a fully connected auto-encoder with a new sparsity constraint called the lifetime sparsity. For each hidden unit across the mini-batch, they rank the activation values, keeping only the top-k% for reconstruction. The approach is appealing because they don't need to find a hard threshold and it makes sure every hidden unit/filter is updated (no dead filters because their activation was below the threshold).Their encoder is a deep stack of ReLu and the decoder is shallow and linear (note that usually non-symmetric auto-encoders lead to worse results). They also show how to apply to RBM. The effect of sparsity is very effective and noticeable on the images depicting the filters.They extend this auto-encoder in a convolutional/deconvolutional framework, making it possible to train on larger images than MNIST or TFD. They add a spatial sparsity, keeping the top activation per feature map for the reconstruction and combine it with the lifetime sparsity presented before.Having worked on auto-encoders in the past, I really like the paper and I think this will be very interesting to many people in this field (researchers in auto-encoder representation learning). I have a few questions/requests:* can you present the reconstruction of a full image on image net using your architecture? i.e. presenting the original image on one side and the reconstructed version on the side. * you only evaluate with an SVM on top of the representation. Does the pre-initialization improves the performance of the same architecture but randomly initialized if you fine-tune the model? can you provide results in that scenario?minor comments L52 "no hyper-parameter to be tuned" what about the initialization technique you choose? the learning rate? the number of hidden units? the activation function? I would consider those as hyper-parameters. L230 "it requires all the dictionary atoms to contribute in the reconstruction" if I look at Fig 5f, there are a few feature maps that are totally black (meaning no activation at all, or very close to 0). Will the corresponding atoms contribute to the reconstruction? same idea at L236, "every filter will get updated", maybe precise that the filter needs to be activated. Maybe you can measure during training which proportion of the filters are not activated. L295 can you point to the url if the paper gets accepted? thanks. Table 2b, CONV-WTA = Stacked CONV-WTA 128 & 2048 maps? precise in the caption. Fig 4a, filters look very weird, can you explain?precise how ranking at each iteration affect the training time (should be very negligible but useful to state that it does compare to normal auto encoder training)


Fast and Accurate Inference of Plackett-Luce Models
We show that the maximum-likelihood (ML) estimate of models derived from Luce's choice axiom (e.g., the Plackett-Luce model) can be expressed as the stationary distribution of a Markov chain. This conveys insight into several recently proposed spectral inference algorithms. We take advantage of this perspective and formulate a new spectral algorithm that is significantly more accurate than previous ones for the Plackett-Luce model. With a simple adaptation, this algorithm can be used iteratively, producing a sequence of estimates that converges to the ML estimate. The ML version runs faster than competing approaches on a benchmark of five datasets. Our algorithms are easy to implement, making them relevant for practitioners at large.

Via a connection to Markov Chains, the paper exploits spectral techniques for learning Placket-Luce models efficiently and with lower error than existing methods.The summary mostly says it all.To the best of my knowledge the algorithmic contributions of the paper are novel as well as the theorem proving convergence.The results are also practically useful when compared to existing work and the paper is well-situated w.r.t. general related work and trends.Perhaps the icing on the take then is that the algorithm is straightforward to implement.For this reason, I believe that other researchers and practitioners will find strong reasons to use this work and build on it.I have two (minor) questions:(1) One question relates to the error metric above Eq (9): I don't fully understand the definition of the theta vector... it appears that a scalar is assigned to a vector... I am intepreting the notation as the assignment to the i'th element of theta.What is most confusing though is the purpose of t... why is it necessary to "center" the theta_i purpose of error comparison?(2) The authors state in Figure 2 that I-LSR is robust to changes in graph structure.While I agree that the impact of graph structure is more prominent for MM, I-LSR's RMSE is not immune to changes in graph structure... am I missing something?And one simple comment:Because the implementation of the stationary distribution computation is critical to the efficiency of Algorithms 1 and 2, some discussion of this should ideally be moved from the Appendix and into the main body of the paper (perhaps as a footnote to save space).


Efficient Compressive Phase Retrieval with Constrained Sensing Vectors
We propose a robust and efficient approach to the problem of compressive phase retrieval in which the goal is to reconstruct a sparse vector from the magnitude of a number of its linear measurements. The proposed framework relies on constrained sensing vectors and a two-stage reconstruction method that consists of two standard convex programs that are solved sequentially. In recent years, various methods are proposed for compressive phase retrieval, but they have suboptimal sample complexity or lack robustness guarantees. The main obstacle has been that there is no straightforward convex relaxations for the type of structure in the target. Given a set of underdetermined measurements, there is a standard framework for recovering a sparse matrix, and a standard framework for recovering a low-rank matrix. However, a general, efficient method for recovering a jointly sparse and low-rank matrix has remained elusive. Deviating from the models with generic measurements, in this paper we show that if the sensing vectors are chosen at random from an incoherent subspace, then the low-rank and sparse structures of the target signal can be effectively decoupled. We show that a recovery algorithm that consists of a low-rank recovery stage followed by a sparse recovery stage will produce an accurate estimate of the target when the number of measurements is O(k log d k ), where k and d denote the sparsity level and the dimension of the input signal. We also evaluate the algorithm through numerical simulation.

In this paper the authors propose a two stage approach to the compressive phase retrieval problem. The idea is that if the measurement matrix factorizes into the product of a Gaussian matrix and and RIP matrix one can use a convex program to recover the RIP matrix times the sparse signal and then use standard techniques to recover the sparse signal from such RIP measurements. The paper text is readable and the results are correct. However, my main concern is that this approach heavily relies on modeling assumptions of the measurement ensemble that do not correspond to a realistic model of interest.I only have one main comment. (The proofs seem correct to me):First sentence of the abstract: "We propose a new approach to the problem of compressive phase...", last sentence of the second paragraph, and first sentence of 1.2 "the main challenge in the CPR problem in its general formulation is to design an accurate estimator that has optimal sample complexity and computationally tractable"I disagree with this claim. If the paper is to be accepted I would recommend some caution about such statements and claims. Indeed the main challenge is to get to optimal sample and computational complexity. However, in this process you are not allowed to pick your algorithm based on your sensing mechanism! This is certainly not elusive. In particular if one can choose the measurement models there are already papers that do get to optimal sample complexity and this one would not be the first. e.g. see"Fast and Robust Compressive Phase Retrieval with Sparse-Graph Codes" Dong Yin, Kangwook Lee, Ramtin Pedarsani, and Kannan RamchandranLet me clarify what I mean. Imagine you get just generic Gaussian measurement vectors this approach simply can not work because the matrix does not factorize. For example, In a realistic setup you have a signal which is sparse in an over-complete basis and you wish to recover that signal from quadratic measurements of the order of sparsity (say from a few Gaussian or Fourier measurements). That is the overall measurement matrix w.r.t the sparse signal is a fat matrix times a fat matrix. Not a tall matrix multiplied by a fat matrix. So even if the RIP assumption holds on the over-complete dictionary it is still not a useful model. Simply put model (2) is unrealistic.The challenge is sparse phase retrieval is really can we come up with an algorithm that can handle sparsity o(k) e.g. an algorithm that works generically like for Gaussian measurements.


Barrier Frank-Wolfe for Marginal Inference
We introduce a globally-convergent algorithm for optimizing the tree-reweighted (TRW) variational objective over the marginal polytope. The algorithm is based on the conditional gradient method (Frank-Wolfe) and moves pseudomarginals within the marginal polytope through repeated maximum a posteriori (MAP) calls. This modular structure enables us to leverage black-box MAP solvers (both exact and approximate) for variational inference, and obtains more accurate results than tree-reweighted algorithms that optimize over the local consistency relaxation. Theoretically, we bound the sub-optimality for the proposed algorithm despite the TRW objective having unbounded gradients at the boundary of the marginal polytope. Empirically, we demonstrate the increased quality of results found by tightening the relaxation over the marginal polytope as well as the spanning tree polytope on synthetic and real-world instances.

Overview: The present manuscript introduces an approach to approximate marginal inference in graphical models that is based on the Frank-Wolfe/conditional gradient method.The main objective is to optimize over the full marginal polytope (a constraint set that grows exponentially) instead of the local polytope (a relaxation of the marginal polytope that typical message passing algorithms optimize over).The presented approach works by repeatedly solving a linearization of the original problem, which in this specific case corresponds to MAP inference, with the natural/theta parameters determined by the current iterate.A specific problem that should be addressed in this context is that the gradient is unbounded at the boundary of the marginal polytope. A barrier-like approach, inspired by interior point methods, is introduced to handle this situation and aid convergence. The empirical results demonstrate improvements versus optimization over the local polytope.Positive points: + This paper provides interesting new perspectives on optimization-based/variational approximate marginal inference in graphical models.While a lot of recent work has focused on "smoothing" the MAP problem to achieve faster convergence (thereby effectively marginalizing), in this work, the smooth marginalization problem is solved by a sequence of non-smooth MAP problems. This highlights again the close connection between the two problems (MAP/marginalization) and provides interesting perspectives. + Related work is discussed in sufficient detail, and the work is well embedded into its scientific context. + The theory regarding the "barrier" approach is developed in great detail in the supplementary material.It is clear that a lot of effort went into development of the approach. + The material developed in the paper seems to be technically correct.Negative points: - I have some criticism regarding the approach in general (even though I find it interesting, from a pure curiosity-based research perspective).The use of the Frank-Wolfe method is indicated if optimization over a *non-linear* objective function is not plausible due to the difficulty of the constraint set, but optimization of a linear objective function over the same constraint set can be handled gracefully.However, this is not the case here.Optimization of a linear objective function over the exact marginal polytope is NP-hard as well.How much is really gained here?In fact, one could argue that it is even harder to solve the MAP problem, sine it is non-smooth. Many recent approaches have focused on smoothing the MAP problem, effectively turning it into a smooth marginalization problem, precisely to improve convergence.What must be done in practice, with the proposed approach, is to solve the inner MAP problem approximately, thereby voiding guarantees.One might argue that it is more elegant to establish a well-defined approximation in the first place, and to then solve the approximation exactly.- The manuscript is purely concerned about approximation quality, and not at all about computational cost. As an example, when evaluating the impact of using approximate MAP solutions in the inner loop, the authors run three solvers (QPBO, TRW-S, ICM) and use the result that realizes the highest energy.Clearly, this is not very practical outside of synthetic experiments.- In the discussion (section 6), the authors refer to their approach as performing "exact marginal inference".This is not the case: While the constraint set is exact, the objective function still uses an approximation of the exact entropy, the TRW bound.I also do not find it true that tightening over the spanning tree polytope has remained relatively unexplored in recent work.This was done both in the original work of Wainwright et al. (2005), as well as the more recent work of Jancsary and Matz (2011).- Figures 1 and 2 requires further polishing: The caption lines run over the space allocated to each specific sub-figure, making the captions very hard to read.In Figure 2 (a), the label of the Y coordinate states "Error in Marginals", but the caption states "Error in logZ".In general, the experiments section and the figures therein leave a rather disorganized impression. I assume that this is partially due to the lack of space, but it seems that further improvements could be achieved here.- A question regarding Figures 2 (c) and (d), and the statement in Section 5.1: How can the optimization of the edge occurrence probabilities rho possibly not have any effect on the tightness of the bound on on logZ if optimizing over the local polytope? Rho is expected to have a significant influence on the quality of the entropy approximation, both when optimizing over the marginal polytope *and* when optimizing over the local polytope.Both in Wainwright et al. (2005) and Jancsary and Matz (2011), the improvement of the bound was considerable in simulations.Is this an artifact of the specific type of interactions chosen, or does it perhaps point to a problem in the code/experimental setup?- A question regarding Figure 3: I assume that for TRBP, no optimization over rho was performed (this relates to my previous question - why should optimization over rho not help)?This is a different problem now than in Figure 2, so even if optimization over rho did not help there, one cannot in general conclude that it would not help for the Chinese characters problem.It seems somewhat unfair to compare the proposed method *with* optimization over rho to TRBP *without* such optimization.- Minor point: Footnote 2 is a fragment -"which converges which iterates differing by 0.002"


Double or Nothing: Multiplicative Incentive Mechanisms for Crowdsourcing
Crowdsourcing has gained immense popularity in machine learning applications for obtaining large amounts of labeled data. Crowdsourcing is cheap and fast, but suffers from the problem of low-quality data. To address this fundamental challenge in crowdsourcing, we propose a simple payment mechanism to incentivize workers to answer only the questions that they are sure of and skip the rest. We show that surprisingly, under a mild and natural "no-free-lunch" requirement, this mechanism is the one and only incentive-compatible payment mechanism possible. We also show that among all possible incentive-compatible mechanisms (that may or may not satisfy no-free-lunch), our mechanism makes the smallest possible payment to spammers. Interestingly, this unique mechanism takes a "multiplicative" form. The simplicity of the mechanism is an added benefit. In preliminary experiments involving over several hundred workers, we observe a significant reduction in the error rates under our unique mechanism for the same or lower monetary expenditure.

Authors consider a setting in which workers can either choose to answer or to "skip" the question. They define the "no free-lunch condition", which states that a payment should give reward zero if all answers (in the ground truth subset) are wrong.The payment rule is very simple: either zero reward if one or more answers are wrong or a reward that grows (decreases for each skipped answer) multiplicatively. Theoretical results regarding the correctness and uniqueness of the proposed method are derived. The results are generalized to the case of different degrees of confidence and evaluated in a real experiment.It is indeed surprising that the payment rule that has to give zero reward as soon as the worker answers incorrectly one question.The key idea is very simple and the authors did a good job on the theoretical side. I have doubts, however, that NIPS should be the conference for this type of work. Nevertheless, I have checked in depth until page 14 in the supplementary material, which includes the proof of uniqueness (the most important). I would like to hear the opinion of the authors on the following points:1- I can replace the factor $T^{G-C}$ by $T^{k(G-C)}$, for arbitrary $k$. This is a more general scoring rule that also satisfies the no-free lunch condition. For k=1, we recover the proposed rule. For k > 1 we penalize more the skips and or k < 1 (and here comes a problem) we penalize less the skips. I say this is a problem because in the limit of k->0 we do not penalize skipping, and the rule would give full reward for a all-skipped answer, something clearly not desired.2- Concerning the empirical evaluation, there is significant selection bias in the experiments: only the good workers "having at least 95% of their prior work approved by the requesters" are selected for the experiment (lines 1115-1116 suppl). This seems to be a critical point because the main motivation is to defer spammers, and this is not really evaluated. Authors should mention that as an (additional) limitation in the beginning of section 5.Related to this, if the main goal is to compare with the traditional method, wouldn't be worthy to make a simulation where the p's are generated randomly without the confounders of a real experiment?3a- As authors note in lines 234-238, this payment rule is indeed restrictive. For example, given N questions, performing three different tasks is different than performing one task only, since failing one single question will lead to zero reward in the later and will not in the former. It is mentioned that in reality, "workers play a repeated game", but this is not considered at all in the analysis/evaluation.3b- Finally, and related to the previous point, note that the proposed rule is more informative than the standard and can leak information, in the sense that zero reward means that one question of the gold standard was answered incorrectly. This could be used by a group of cheaters to learn the subset of gold standard questions.I have other minor remarks:- In Axiom 1, why not replacing the inequality and equality just with the condition $\sum_{i=1}^G\one\{x_i=-1\} = G$? - line 510 suppl: $kappa$ should be $\kappa$ - line 577 suppl: $j\in[\eta+\gamma]$ should be$j\in[1,\eta+\gamma]$. This should also be corrected many times later on. - line 577 suppl: $j\in\{\eta+1,\eta+\gamma\}$ should be $j\in\{\eta+1,\hdots,\eta+\gamma\}$ - It is not clear why the wrong answers are changed (eg, lines 580-588 suppl). - line 738 suppl: Algorithm ?? - (lines 1199, 1209) suppl: the user is informed that the bonus is not altered if a question is skipped, but that is not true, since skipping decreases reward exponentially.


Sample Efficient Path Integral Control under Uncertainty
We present a data-driven optimal control framework that is derived using the path integral (PI) control approach. We find iterative control laws analytically without a priori policy parameterization based on probabilistic representation of the learned dynamics model. The proposed algorithm operates in a forward-backward manner which differentiate it from other PI-related methods that perform forward sampling to find optimal controls. Our method uses significantly less samples to find analytic control laws compared to other approaches within the PI control family that rely on extensive sampling from given dynamics models or trials on physical systems in a model-free fashion. In addition, the learned controllers can be generalized to new tasks without re-sampling based on the compositionality theory for the linearly-solvable optimal control framework. We provide experimental results on three different tasks and comparisons with state-of-the-art model-based methods to demonstrate the efficiency and generalizability of the proposed framework.

Summary  The paper presents a new method for path integral control. The proposed method leverages that the system control matrix G is often known, and the uncontrolled dynamics (or dynamics under a reference controller) can be learnedusing a Gaussian Process. The constraint on the reward function takes a more general shape than in previous PI approaches which means among others that noise and controls can act in different subspaces. The authors also showhow their framework can be used for generalizing from known to new tasks, and evaluate the method on three simulatedrobotics problems, where their method compares favourably to SOTA reinforcement learning and control methods.    Quality   Generally, the derivations seem correct and principled (although I'm unsure about the task generalization, see below).Related work is sufficiently discussed, the authors point out the similarities and differences to related work andcompare to state of the art methods. The experiments are convincing, altough some details are missing (see below).  The task generalization seems odd: (13) states that the exponent of the reward functions will be linearly combinedto yield the exponent of the reward function of the new task. However, it's not clear to me how that results in the exponent of the value function being the same kind of linear combination (14). In any case, this should be explained,but I think that actually there might be an error here ((14) also states psi is the exponent of the cost function,which holds only for the last time step as shown in the equation around line 247).As defined by the authors, the task generalization is specific to the case that the task is defined by a target, although it seems that othertask variations would be possible as long as a suitable task similarity kernel can be defined.  The experiments considers relevant and realistic tasks, and compare to SOTA methods. The technical details are a bit sparsein the experimental section: the paper should mention what the dynamics and reward functions are, and how long sampled trajectoriesare, in order for the experiment to be reproducible. (Possibly in the appendix). One baseline is iterative PI control,is this one of the methods in table 1? It would be insightful to see how it compares on these qualitative aspects. The comparison plots in Figures 1 and 2 should show error bars. The reported score is the exponent of the value fc, psi. Is this the psi as calculated by the robot? What if the robot is overly optimistic in its calculation of psi? It wouldbe more objective to report the average (or cumulative (discounted)) reward (that is also what the algorithm sets out to optimize).      Clarity  Generally, the paper is well-written and explains the proposed methods rather well. The comparison to other methods, e.g. in Table 1, helps understanding the relationship to the SOTA. There are a couple of grammatical and formatting errors, see below under Minor points. One confusing point is that the GP has different regression targets in (2) (line 094) and line 208,which doesn't include the reference controls. This should be made consistent or explained. The authors should explain how the hyperparameters of the GP are set. It's unclear what's meant by "the posterior distribution can be obtained by constraining the joint distribution" -- is conditioning meant?Algorithm 1 is confusing - here, it looks as if an open-loop control sequence is optimized while the rest of the paper discusses learning a feed-back controller. If it's really just an open-loop sequence u_1 ... u_T that's returned in line 13 of the algorith, how can the algorithm deal with the noise? This shouldbe clarified or corrected.      Originality and Significance    Although the main ingredients of the proposed algorithm have been used in earlier algorithms (propagation through an learnedstochastic forward model as in [15], path integral control), the algorithm combines these in a novel way. As far as I know, this is an original formulation that seems to attain very good performance. By evaluating on realistic problems against strongbenchmarks, the proposed algorithm can be concluded to be a significant improvement over prior work.    Comments on the rebuttal:Thanks for clarifying (14). Still, it seems there is a typo and Psi should be Phi (or Psi_t+dt should be included in the middle part). To me, it's therefore unfortunately still unclear what's meant here. I changed my confidence& score accordingly. Thanks for clarifying the open-loop part, I would really change the notation here to avoid confusion. I'm also still confused about what meant by the psi in figure (1), as I understand it this is the exponent of the final costs, but average cost would be more standard. Maybe the average cost could additionally be reported in the supplementary material?  Minor points    * Although generally well-written, the paper has some grammar issues. I recommend letting someone proofread it  (line 040 "has been existed", line 143 (while -> in contrast ?), line 231 - comma instead of full stop or reformulate,   line 345 is based on ... model -> are based on ... models )   * the authors should avoid creating empty sections (e.g. between sec 2 and sec 2.1)  * there are some formatting issues (margins in line 128, table 1, brackets not scaled correctly in (8))  * would be clearer if the authors state how to obtain \tilde{G} from G* line 192 table.1. -> Table 1.  * notation: in (13) x_t is written boldface on the right-hand side but italic on the left-hand side.  * line 339: Pilco requires an optimizer for policy evaluation -> is policy improvement meant?  * line 332 "6 actuators on each joint" -> is "one actuator per joint" meant?


Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Recurrent Neural Networks can be trained to produce sequences of tokens given some input, as exemplified by recent results in machine translation and image captioning. The current approach to training them consists of maximizing the likelihood of each token in the sequence given the current (recurrent) state and the previous token. At inference, the unknown previous token is then replaced by a token generated by the model itself. This discrepancy between training and inference can yield errors that can accumulate quickly along the generated sequence. We propose a curriculum learning strategy to gently change the training process from a fully guided scheme using the true previous token, towards a less guided scheme which mostly uses the generated token instead. Experiments on several sequence prediction tasks show that this approach yields significant improvements. Moreover, it was used succesfully in our winning entry to the MSCOCO image captioning challenge, 2015.

Recent work on neural machine translation and other text generation tasks has trained models directly to minimize perplexity/negative log-likelihood of observed sequences. While this has shown very promising results, the setup ignores the fact that in practice the model is conditioning on generated symbols as opposed to gold symbols, and may therefore be conditioning on contexts that are quite different from the contexts seen in the gold data. This paper attempts to remedy this problem with by utilizing generated sequences at training time. Instead of conditioning on the gold context it utilizes the generated context. Unfortunately at early rounds of the algorithm this produces junk, so they introduce a "scheduled sampling" approach that alternates between the two training methods based on a predefined decay schedule inspired by curriculum learning.The strength of this paper is in its simplicity and the comprehensive empirical testing. The important model and inference assumptions are defined clearly, and the details about the internal architecture of the model are appropriately elided. It seems like it would be very straightforward to re-implement this approach on LSTM's or any other non-Markov model.  Empirically, the method seems to work quite well. There is a relatively large gain across several quite different tasks, and the schedule part seems to have a significant effect as always sampled does quite poorly.The parsing results still pretty far behind state-of-the-art, but they use a very reduced input representation (no features). The speech results also seem to be using a somewhat unique setup, but the improvement here is quite large. - I would be to know how performance changes based on the footnote 1. It seems like flipping on a token level is very different than flipping on an example level, since the worst-case distance between gold tokens is much lower.The main weakness is the lack of comparison to other methods that attempt a similar goal.For one, the authors are too quick to dismiss early-update perceptron (Collins and Roark, 2004) with beam search as being not applicable, "as the state sequence can not easily be factored". While the factoring is utilized in parsing, nothing about beam search requires this assumption to work.(This connection between beam search and DP is also made on l.130. Beam search is rarely used for HMMs, at least in NLP, and when it is, it is often exactly when it is not possible to use dynamic programming.) The continuous nature of the state shouldn't effect the use of this algorithm, and in fact there is a paper at ACL this year "Structured Training for Neural Network Transition-Based Parsing" that that using this method on neural-net model that makes similar assumptions.Secondly, I did not feel like an appropriate distinction was made with SEARN and reinforcement learning type algorithms. The related work talks about these as as "batch" approaches. While the SEARN paper itself may have chosen to use a batch multi-class classifier (since they are fast), that does not mean it couldn't be applied in the SGD case. It seems like the key idea of SEARN is to interpolate the current prediction from the model with gold based to produce a sampled trajectory. The major difference is that they may learn the policy versus using a schedule.


Model-Based Relative Entropy Stochastic Search
Stochastic search algorithms are general black-box optimizers. Due to their ease of use and their generality, they have recently also gained a lot of attention in operations research, machine learning and policy search. Yet, these algorithms require a lot of evaluations of the objective, scale poorly with the problem dimension, are affected by highly noisy objective functions and may converge prematurely. To alleviate these problems, we introduce a new surrogate-based stochastic search approach. We learn simple, quadratic surrogate models of the objective function. As the quality of such a quadratic approximation is limited, we do not greedily exploit the learned models. The algorithm can be misled by an inaccurate optimum introduced by the surrogate. Instead, we use information theoretic constraints to bound the 'distance' between the new and old data distribution while maximizing the objective function. Additionally the new method is able to sustain the exploration of the search distribution to avoid premature convergence. We compare our method with state of art black-box optimization methods on standard uni-modal and multi-modal optimization functions, on simulated planar robot tasks and a complex robot ball throwing task. The proposed method considerably outperforms the existing approaches.

Summary: The paper introduces an algorithm (MORE) for black box optimization that constructs local quadratic ``surrogate'' models based on recent observations.By using a (simpler than the true function) quadratic model, the iterative step of refining the reward parameters can be computed in closed form (though the models themselves seem to be built with a form of sampling). This approach allows for a more sample efficient and robust search procedure, which is shown to outperform state-of-the-art methods in terms of samples and converged parameters on a number of simple functions as well as some very complex robotics tasks. Review: The new algorithm performs much better than the state-of-the-art algorithms in a wide range of experiments and is applicable in a very important problem setting.I appreciate the wide range of problems that the authors used and I think they make a very strong case for the new algorithm.I do think there is some room for improvement in what experimental conditions were considered, how some of the results are presented, and in the general description of the algorithm's properties, which I cover here:First and foremost, the experiments section does not contain any results on computational speed and in fact, computation time is not mentioned until the next-to-last sentence in the paper.But there, it is mentioned that MORE takes more time than all the other approaches (though we are told it is fast enough, taking less than a second).It would be better to see some actual comparisons of computation time from the experiments (perhaps in place of one of the robotic arm diagrams), and the computation time issue ought to be mentioned much earlier rather than buried at the very end of the paper.I do not understand why experiments were not conducted using the exact solution of the quadratic modeling problem described in the paragraph starting on line 196.I do understand this would require many more samples than the dimensionality-reduction approach, but it seems possible the exact solution might converge to a better set of parameters.Why not show this approach on some of the graphs (at least in the simple problems) and also report how many more samples it needs than reduced-dimension approach?Or, if the number of samples is always prohibitive, state that definitively to mark the exact solution as intractable.There is a disconnect between the description of the approach in the early sections, which focus on the ability to do closed form calculations, and the computation of the posterior (line 249) of the quadratic models, which requires sampling.I understand that the change in parameters can now be calculated in closed form, which is a nice achievement, but it seems like the burden of sampling may have just been pushed deeper into the algorithm (in the posterior calculation).If this is the case, it should be mentioned earlier on that the algorithm still requires sampling at some level.A basic premise of the paper (line 106) is that keeping the previous and next reward functions close to one another ``controls'' the exploration-exploitation dilemma.But there are certainly cases (say when the current function creates terrible behavior) where the proper thing to do is to completely change the function parameters.A better description around line 106 of when this algorithm is allowed to make such large leaps (or why it does not) would make the approach more understandable.At line 101, it is mentioned that CMA-ES is ``well known as an ad-hoc algorithm'' but no citation is given and no real evidence of that fact is presented.Graphs 4b and 4c seem to be referenced incorrectly in the text (b is referenced as the entropy graph, but its really c, and the reverse for the high dimensional graph)Typos: Line 21 - frail to converge prematurely - converge prematurely. Line 55 -allow to - allow us to Line 107 - search distribution - search distributions Line 110 - control - controlling Line 111 - Such control - Such a control Line 154 - from using - of using Line 155 - decreased the - a decrease in the