Efficient state-space modularization for planning: theory, behavioral and neural signatures
Even in state-spaces of modest size, planning is plagued by the "curse of dimensionality". This problem is particularly acute in human and animal cognition given the limited capacity of working memory, and the time pressures under which planning often occurs in the natural environment. Hierarchically organized modular representations have long been suggested to underlie the capacity of biological systems 1,2 to efficiently and flexibly plan in complex environments. However, the principles underlying efficient modularization remain obscure, making it difficult to identify its behavioral and neural signatures. Here, we develop a normative theory of efficient state-space representations which partitions an environment into distinct modules by minimizing the average (information theoretic) description length of planning within the environment, thereby optimally trading off the complexity of planning across and within modules. We show that such optimal representations provide a unifying account for a diverse range of hitherto unrelated phenomena at multiple levels of behavior and neural representation.

This paper provides a new algorithm for sub-dividing a state space for planning and search. The algorithm is shown to provide qualitative fits to features of human navigation data and neural data from rodents in a simple learning task.
The paper is very ambitious and develops a computational model of how the state space can be carved up (aggregated?) for planning. This model is applied to some intriguing data on human and rodent spatial navigation and seems to nicely pull together disparate threads from the literature. Unfortunately, the exposition was sufficiently abstract, so that following the thread from the model to the results (simulations) was challenging, leaving unclear exactly how the model was explaining the behaviour and making evaluation difficult. I unpack these and a couple of other issues below.  1. It is not entirely clear how the different ideas introduced in the paper (modularity, centrality, description length) fit together into a single model of behaviour and the brain. From the text, it was not clear to me how the simulations and predictions for the different behavioural tasks were generated. I think the account in the text is just too abstract with insufficient detail of how to get from those three equations to a full model of the behaviour (which they obviously have).  2. Almost no detail is given on the brain simulations (Fig 3C). The results look great, but I have no idea how they were generated from the model as spelled out in the early sections.  These first two issues may be addressed in the supplemental materials (which I did not read), but my understanding is that the core paper needs to stand alone, with the supplementals supplementing the main text, not replacing key aspects of the results.  3. It was not clear to me how their concept of modularization was different than simple state aggregation. That is, the novelty and distinctiveness of the modularization approach was not apparent, and this parallel is even briefly acknowledged in footnote 1, yet the approach is still deemed novel. The paper could still be interesting even if the modularization approach just builds on or even just implements state aggregation with interesting connections to human and animal behaviour, but that was not done here.  4. The optimal modularization presented in Figure 4D seems like it might depend strongly on the “states” that are included in the model. Why are explore, freeze, rest, and groom only behaviours permitted near the start sequence? Unlike some of the experiments with humans on navigation, the behavioral repertoire of the animals is not strongly limited by their location. Animals can engage in these behaviours at any time (though they might not in practice). This needs further justification. Also, the states there are almost all actions, not really states.


Interaction Networks for Learning about Objects, Relations and Physics
Reasoning about objects, relations, and physics is central to human intelligence, and 1 a key goal of artificial intelligence. Here we introduce the interaction network, a 2 model which can reason about how objects in complex systems interact, supporting 3 dynamical predictions, as well as inferences about the abstract properties of the 4 system. Our model takes graphs as input, performs object-and relation-centric 5 reasoning in a way that is analogous to a simulation, and is implemented using 6 deep neural networks. We evaluate its ability to reason about several challenging 7 physical domains: n-body problems, rigid-body collision, and non-rigid dynamics.

The authors aim to develop a learning-based physics engine. The engine developed in the paper take as input an interaction graph. This graph has nodes corresponding to objects and edges corresponding to relations between the objects. At each step of the simulation, a trained MLP is used to predict the effects of inter-object interactions. Then, a separate trained MLP is used to predict the state of the object at the next time-step taking into account both the previous state of the object and the effects of its relationship with other objects. The authors train and test the proposed model on three different types of scenarios: n-body systems; balls bouncing in a box; and strings composed of springs that collide with rigid objects. They demonstrate accurate predictions and generalization ability of the model one time-step into the future.
                  A. Experiments         * It seems only one timestep into a future is explored (lines 252-253). The authors should have tried to evaluate the model over a longer timescale. Especially in the case of balls bouncing in a box, it seems much harder to correctly predict the future positions as errors in positions tend to accumulate over the tilmestep.         * The qualitative results (Fig. 2) indicates the model inaccurately predicts over a longer timescale - a property rather important for a  physics engine.                  B. Model         * The presented model works under quite a strong assumptions where an 'interaction graph' is given. Works such as Fragkiadaki et al. [5] (ICLR'16) demonstrate similar abilities but under weaker modelling assumption (from a raw visual input).         * It also seems the model requires quite a lot of training data (1 million). Hence it is hard to tell if the model has any possibility to generalize outside of the 'simulation' domain.          C. Motivation         * It is unclear for me what is the main motivation/purpose behind the submission. If simulation and 'interaction graph' are available then arguably we can use 'laws of physics' without a need to learn anything. Although other works (e.g. [5]) still use simulation, the interactions are learnt from the data only, and hence such a line of research ([5]) may have an impact for real-world scenarios. Likewise, in real-world scenarios extracting an 'interaction graph' may become a quite difficult problem -- and such graph is likely to be 'corrupted by noise'. Such issues are not addressed in the submission.    ---  I liked the rebuttal, and modified my score a bit. However, there are things that still bother me.  Regarding the experiments:  The submission lacks of quantitative results on long-term predictions (either with MSE or by proposing an alternative measure). The MSE results given in Table 1 (and majority results in the supplementary material) already exhibit small errors. Hypothetical future work that builds upon the submission would arguable explore longer-term predictions, and need a comparison. Moreover, qualitative results may be good in the 'intuitive physics' domain, but to build 'physics engine' or to allow reproducibility/comparisons automatic measures showing deviations from the ground truth are also important.   I also think the Table from supp. should be moved to the main paper -- it shows quite important results.  It would also be great if the authors make the dataset / code publicly available.   Regarding the motivation/impact:  I agree with the authors that predicting functional forms in the interaction networks might be useful in many scenarios ranging from intuitive physics, learning engines of systems with unknown simulation, or social networks, but none of the aforementioned tasks have been explored in the submission. For instance, in the 'intuitive physics from images' domain, the authors have to extract a noisy graph representation of an image. Is IN competitive with the work Fragkiadaki et al. in this scenario? I see the experiments shown in the submission as valuable additional experiments done in a more controlled environment but they miss real applications. Therefore, I feel the work is somehow incomplete, but I also agree it has a good potential, and is a nice initial study in a controlled environment under the assumption that the graph is given.


Reconstructing Parameters of Spreading Models from Partial Observations
Spreading processes are often modelled as a stochastic dynamics occurring on top of a given network with edge weights corresponding to the transmission probabilities. Knowledge of veracious transmission probabilities is essential for prediction, optimization, and control of diffusion dynamics. Unfortunately, in most cases the transmission rates are unknown and need to be reconstructed from the spreading data. Moreover, in realistic settings it is impossible to monitor the state of each node at every time, and thus the data is highly incomplete. We introduce an efficient dynamic message-passing algorithm, which is able to reconstruct parameters of the spreading model given only partial information on the activation times of nodes in the network. The method is generalizable to a large class of dynamic models, as well to the case of temporal graphs.

The paper deals with the problem of diffusion graph weighting, from activity data observed from the studied networks. More specifically, it focus on an adaptation of NetRate (which corresponds to an extension of the Independent Cascade model with  delays of infection, here in the discrete case, following a geometric law on these  transmission delays) to the case where only the activity of a subset of the nodes of the network can be monitored. The claim is that non observed nodes shouldn't be ignored when learning the transmission probabilities of the diffusion graph.
Very interesting and well written paper, which deals with an important problem when learning diffusion models: it may be impossible to observe the activity of every node of the network. For instance, if the source is Twitter, the streaming API only allows one to monitor the activity of about 4000 users over several millions.   The proposed approach is elegant and well sounded. I enjoyed to review this paper that brought me new knowledge about learning in such complex situations. The proposal is maybe not too much innovative since it mostly employs already published techniques, but from my point of view the followed methodology deserves to be published in some main machine learning venue, at least for its pedagogical aspect. Moreover, a good theoretical analysis of the algorithm is given for the case of tree network  structures.    I have two concerns about the applicability of the method in real-world scenarios:      - The model is based (as NetRate and a lot of other models) on the hypothesis that the earlier a node is infected the more likely it is to be. This is induced by the fact that a single parameter controls the activation probability and the time of this activation. In such models, the non-infected nodes are usually considered as infected at a horizon time T. My experience is that this parameter $T$ has a very great impact on the results and is very difficult to tune. However, considering asynchronous diffusion models such as Saito's ones leads to greatly more complex likelihood formulations. I do not know if the present work could be transposed in such a setting.        - The model considers a geometric law for the time delays of transmission between nodes. This is a strong assumption about the regularities on transmission dynamics that may not hold in  real life.   Three questions/coments about the experiments:      - What is the MonteCarlo method used for HTS ? From my point of view this should be an MCMC, such as Gibbs Sampling, since it has to consider activation times of observed infected nodes.     - How are generated the cascades from the real world graph? Using a IC model with a geometric law on diffusion delays as considered in the paper ? I regret that real "real world" data have not been used here, as this does not allow to assess the model for real life tasks...     - Only very small networks have been considered here. It would be nice to experiment it on greatly larger networks to assess how it scales  For me the paragraph "missing information in time" should be removed since it presents very few interest: it appears as obvious that the algorithm can be easily adapted with a different discretization of the time. At least, it should be reworked (I spent more time to understand this small paragraph than the whole rest of the paper!):    - what is the difference between $\cal K$ and k here ? For me, $k$ should be used in place of $\cal K$ everywhere    - for me $\delta_k$ should be noted $\delta_i^c$, it is the interval in which $\tau_i^c$ is included. Then, it would allow one to skip the indicator function in the following equations, by giving $\Delta_{k_i^c}$ that would be more clear     - line 215: the upperbound of $\tau_i^c$ should be $\tau^{k+1}$ rather than $\tau^{k}$   - Last but not least: I do not understand why non-infections are ignored here. It is claimed that, provided the final time T is not included in the observations, the second term of (9) can be ignored. I disagree with this: every non infected node at the end of the last observed step should be considered as not infected. Thus, a term $P_S^i$ should be given for them. Else, every probability is likely to be greatly over-estimated.. No ?     Minor comments:     - "Using expression (11)" line 199 is somewhat misleading (at least it confused me a little for the comprehension of the proof). From my point of view, it would be more clear without this indication.     - Formula (9): \leq T-1 rather than \leq T in the first term, no ? (or strict inequality)    - Line 75: "the its"    - Formula 4 : I have some problems with the notation of the sum. For me the subscript notation for "each possible set of hidden activation times" is not correct here.     - I would add a $\cal O$ as a subscript of \Sigma in formula (9)       - Line 282: overestimated => underestimated (and if I am right the opposite operation line 280)


High resolution neural connectivity from incomplete tracing data using nonnegative spline regression
Whole-brain neural connectivity data are now available from viral tracing experiments, which reveal the connections between a source injection site and elsewhere in the brain. These hold the promise of revealing spatial patterns of connectivity throughout the mammalian brain. To achieve this goal, we seek to fit a weighted, nonnegative adjacency matrix among 100 µm brain "voxels" using viral tracer data. Despite a multi-year experimental effort, injections provide incomplete coverage, and the number of voxels in our data is orders of magnitude larger than the number of injections, making the problem severely underdetermined. Furthermore, projection data are missing within the injection site because local connections there are not separable from the injection signal. We use a novel machine-learning algorithm to meet these challenges and develop a spatially explicit, voxel-scale connectivity map of the mouse visual system. Our method combines three features: a matrix completion loss for missing data, a smoothing spline penalty to regularize the problem, and (optionally) a low rank factorization. We demonstrate the consistency of our estimator using synthetic data and then apply it to newly available Allen Mouse Brain Connectivity Atlas data for the visual system. Our algorithm is significantly more predictive than current state of the art approaches which assume regions to be homogeneous. We demonstrate the efficacy of a low rank version on visual cortex data and discuss the possibility of extending this to a whole-brain connectivity matrix at the voxel scale.

This paper presents a structured regression model for inferring neural connectivity from viral tracing data. Combining a nonnegativity constraint, a spatial smoothness regularizer, a low-rank assumption, and a mask that handles ambiguity within the injection site, the authors derive a model that can leverage observed tracer data to infer the underlying connectivity. While the individual components of this model have been explored in other structured regression problems, this paper presents a thorough and well-executed application of these ideas to an interesting problem.
The underlying model is a non-negative linear regression, y = Wx + \eta, where \eta is drawn from a spherical Gaussian model.  The weight matrix, W, is assumed to be nonnegative and, in probabilistic terms, drawn from a spatially smooth prior. Optionally, a low-rank assumption may be incorporated into the weight model, which can dramatically improve memory efficiency for large-scale problems.  While the individual components of this model (nonnegative regression, Laplacian regularized least squares, low-rank constraints) are well-studied, I think this is a nice combination and application of these techniques to a real-world, scientific problem. The presentation of the model, the synthetic examples, and the real world applications (and supplementary movies) are particularly clear.  While it is certainly valid to directly construct a objective function that captures both the reconstruction error and the domain-specific constraints and inductive biases, I think a probabilistic perspective could elucidate a number of potential extensions and connections to existing work. For example, Jonas and Kording [1] tackle a similar connectomics problem by formulating a probabilistic generative model in terms of latent variables of each neuron. While the setup is somewhat different (theirs is an unsupervised modeling problem whereas yours is framed as a regression problem), one could imagine a similar set of latent variables serving as a prior for your weight matrix.  Similarly, from a probabilistic perspective, the Gaussian noise model is not particularly realistic given that the fluorescence observations are nonnegative. Likewise, the spherical covariance seems likely to be a strong assumption. While these may justifiable and computationally advantageous simplifications, it is worth discussing. Finally, the orthogonal projector, P_{\Omega}, seems like a rather fancy way of saying that your loss function is only summing over observable voxels.  Minor comments: - The labels in Figure 3 are not very legible.  [1] Jonas, Eric, and Konrad Kording. "Automatic discovery of cell types and microcircuitry from neural connectomics." eLife 4 (2015): e04250.


Select-and-Sample for Spike-and-Slab Sparse Coding
Probabilistic inference serves as a popular model for neural processing. It is still unclear, however, how approximate probabilistic inference can be accurate and scalable to very high-dimensional continuous latent spaces. Especially as typical posteriors for sensory data can be expected to exhibit complex latent dependencies including multiple modes. Here, we study an approach that can efficiently be scaled while maintaining a richly structured posterior approximation under these conditions. As example model we use spike-and-slab sparse coding for V1 processing, and combine latent subspace selection with Gibbs sampling (selectand-sample). Unlike factored variational approaches, the method can maintain large numbers of posterior modes and complex latent dependencies. Unlike pure sampling, the method is scalable to very high-dimensional latent spaces. Among all sparse coding approaches with non-trivial posterior approximations (MAP or ICAlike models), we report the largest-scale results. In applications we firstly verify the approach by showing competitiveness in standard denoising benchmarks. Secondly, we use its scalability to, for the first time, study highly-overcomplete settings for V1 encoding using sophisticated posterior representations. More generally, our study shows that very accurate probabilistic inference for multi-modal posteriors with complex dependencies is tractable, functionally desirable and consistent with models for neural inference.

The paper considers the problem of sparse coding, also known as dictionary learning. It is here expressed as a maximum likelihood problem where one wishes to fit to some training data the parameters of a Bernoulli-Gaussian (“spike and slab”) prior on a sparse latent variable of (potentially large) dimension H, the components W of a dictionary, and the variances of the observation noise. To circumvent the intractability of the problem for large H, a technique is proposed which combines ideas from iterative hard thresholding (via a “selection function” which actually keeps the H’ < <  H components most correlated to the observation given the current parameters) and expectation maximization with Gibbs sampling. Numerical experiments show that, with an appropriate choice of H’, the approach behaves well both in controlled synthetic experiments (in low dimension) and in one denoising experiments with large H. Further, a discussion of the shape and nature of the learned dictionary atoms is provided.
Overall this is an interesting and fairly clear algorithmic and experimental paper paper presenting an approximate EM technique for dictionary learning in a Bernoulli-Gaussian setting.    The main idea is that EM requires sampling to approximate certain intractable expectations/integrals, and that such sampling is inaccurate or difficult in high dimensions H. It is proposed to perform sampling only after a selection step which consists in finding a subspace (dependent on the considered training vector as well as the currently estimated parameters of the model) of intermediate dimension H’ << H.   As far as I could check the selection step, expressed implicitly in (10)-(11) corresponds to computing certain weighted correlations of the training sample y_n with columns of the current estimate of the dictionary, and to keep the H’ largest correlations. This essentially boils down to one step of hard thresholding, reminiscent of certain recent techniques for dictionary learning (see, e.g., recent work of K. Schnass such as [A]).    Numerical experiments on synthetic data show state of the art performance of the proposed approach. An discussion on the shape of the atoms learned on the “house” image is provided.  On the one hand this is interesting and intriguing, as one can see for example some “curved” atoms similar to those appearing in the work of Jost et al [B], and I am curious see if the approach can be extended to factor invariances (translation, rotation) in the learning stage. On the other hand, it is a bit of a pity that the experiment is only conducted on a single image, since this seriously limits the interpretability of the results.    The paper is concluded by the potentially interesting Figure 3 which aims at presenting a synthetic view on existing algorithms based on two axes: “accuracy of posterior approximation” and “latent dimension” (a measure of scalability). I welcome this type of synthetic representations in principle. However I am a bit bothered by the fact that the positioning on the axes (especially the horizontal one) seem to be subjective rather than objective. It would be really interesting to draw such a picture, but one would need to agree for example on the notion of “accuracy of posterior approximation”.  One way to do it may be to conduct large-scale empirical simulations and use agnostic performance measures.   Since the proposed technique is claimed to be scalable, I would have expected some mention of computation times besides pure log-likelihood figures .   Please carefully revise the english writing, e.g.: the sentence line 21-22 has two verbs, line 43 “to in general study”, line 80 “which have in similar forms be derived …”line 181 “implementation following and execute”, line 206 “obtaine”,line 244 “the for MAP based approaches required choice”.  [A] Schnass, K. (2014, January 24). Local Identification of Overcomplete Dictionaries. arXiv.org. [B] Jost, P., Vandergheynst, P., Lesage, S., & Gribonval, R. (2006). MoTIF : an Efficient Algorithm for Learning Translation Invariant Dictionaries (Vol. 5, pp. V–V). Presented at the Acoustics, Speech and Signal Processing, 2006. ICASSP 2006. IEEE International Conference on, Toulouse, France: IEEE. http://doi.org/10.1109/ICASSP.2006.1661411


Gaussian Process Bandit Optimisation with Multi-fidelity Evaluations
In many scientific and engineering applications, we are tasked with the optimisation of an expensive to evaluate black box function f . Traditional methods for this problem assume just the availability of this single function. However, in many cases, cheap approximations to f may be obtainable. For example, the expensive real world behaviour of a robot can be approximated by a cheap computer simulation. We can use these approximations to eliminate low function value regions cheaply and use the expensive evaluations of f in a small but promising region and speedily identify the optimum. We formalise this task as a multi-fidelity bandit problem where the target function and its approximations are sampled from a Gaussian process. We develop MF-GP-UCB, a novel method based on upper confidence bound techniques. In our theoretical analysis we demonstrate that it exhibits precisely the above behaviour, and achieves better regret than strategies which ignore multi-fidelity information. MF-GP-UCB outperforms such naive strategies and other multi-fidelity methods on several synthetic and real experiments.

The authors considered Bayesian optimization framework. The main difference of the considered problem statement consists in that multi-fidelity black-boxes are available, realizing the same function. The data source with the highest fidelity at the same time is the most expensive to evaluate. It is assumed that costs of evaluation of data sources with different fidelities are given.  The authors proposed multi-fidelity bandit problem statement, they formalized the notion of regret when explicit costs of different fidelity observations are given. Then the authors proposed an approach for Bayesian optimization, which explicitly takes into account costs of observations and relationships between black-boxes with different fidelities. The approach is based on Gaussian process upper confidence bound algorithm. Asymptotics of the corresponding regret is provided w.r.t. increasing capital of a resource, available for black-boxes evaluations. Extensive computational experiments provide evidence about efficiency of the proposed approach.
Comments: - line 88. It seems that the comma should be placed before the word «where», not after this word. - line 101. In the formula for a Gaussian distribution (at the end of the line) there is a redundant power «2». It seems that after the word «with» the comma should not be used. - line 105. The definition of the regret depends on B, which is defined in A3. In practice, if I would like to monitor the dependence of the regret on iterations number, how should I set a value of B? - In formula (2) the authors introduce variables $\lambda^{(m)}$. However, starting from line 119 there is no any explanation what $\lambda^{(m)}$ stands for (although it is quite easy to understand, but formally it should be written). - Will the considered regret bound and theoretical results about it be still valid if for the case of lower fidelity functions we set the instantaneous rewards to be  equal not to -B, but  to some set of arbitrary numbers? - Formula (3). At the end of the formula the dot should be placed. - line 252. «...if the algorithm, is stuck at fidelity m for too long then …». What is «too long» here? Any possibility to make a decision automatically? Or a human intervention is needed? - Are there any recommendations how to set values of $\lambda^{(m)}$? Of course, I understand, that these values are defined by to which extent a high fidelity model is more expensive to evaluate then low fidelity models. However, this information is not always precisely known. At the same time values of $\lambda^{(m)}$ influence the outcomes of the proposed algorithm. That is why I am asking about the corresponding authors' experience. - line 313, Type Ia Supernovae. M = 3 fidelity functions are considered. In previous examples these functions were defined e.g. by the sample size, used to estimate the functions values. How is it done in this test case? - line after line 396 (Appendix, A.1 Table of notations), the seventh row from below. There is some problem with a reference to some equation. - line 491. «Let $\sigma_{t-1}$ denote the posterior…» -> «Let $\sigma_{t-1}^2$ denote the posterior..." -  line 532. «vol(vol(…»-> «vol(…» - In the proof of theorem 10 I would like to ask the authors to explicitly: a) write where in the proof assumption A2 (line 105) is used, b) comment on whether we consider the statement of the theorem conditional only on those GP trajectories, which fulfill assumptions A2 and A3 (line 105), or not.  Conclusions: - The topic of the paper is very important. E.g. in engineering practice it is a typical situation when black-boxes of different fidelities are available. - As far as I know this is the first time when a notion of a cumulative regret for the multi-fidelity case is rigorously introduced. - The proofs seem correct. Results of experiments confirm theoretical conclusions. The paper is well written. - Therefore, I would recommend to publish this paper in the NIPS proceedings.


On Regularizing Rademacher Observation Losses
It has recently been shown that supervised learning linear classifiers with two of the most popular losses, the logistic and square loss, is equivalent to optimizing an equivalent loss over sufficient statistics about the class: Rademacher observations (rados). It has also been shown that learning over rados brings solutions to two prominent problems for which the state of the art of learning from examples can be comparatively inferior and in fact less convenient: (i) protecting and learning from private examples, (ii) learning from distributed datasets without entity resolution. Bis repetita placent: the two proofs of equivalence are different and rely on specific properties of the corresponding losses, so whether these can be unified and generalized inevitably comes to mind. This is our first contribution: we show how they can be fit into the same theory for the equivalence between example and rado losses. As a second contribution, we show that the generalization unveils a surprising new connection to regularized learning, and in particular a sufficient condition under which regularizing the loss over examples is equivalent to regularizing the rados (i.e. the data) in the equivalent rado loss, in such a way that an efficient algorithm for one regularized rado loss may be as efficient when changing the regularizer. This is our third contribution: we give a formal boosting algorithm for the regularized exponential rado-loss which boost with any of the ridge, lasso, SLOPE, ∞ , or elastic net regularizer, using the same master routine for all. Because the regularized exponential rado-loss is the equivalent of the regularized logistic loss over examples we obtain the first efficient proxy to the minimization of the regularized logistic loss over examples using such a wide spectrum of regularizers. Experiments with a readily available code display that regularization significantly improves rado-based learning and compares favourably with example-based learning.

The paper makes contributions involving Rademacher observations (previously dubbed "rados"), and their relationship to regularization. It shows that the compression of the dataset's information into the set of rados has a regularizing effect, defines rado-losses which are of special importance in the theory, and demonstrates they hold practical value.
The technical contribution of this paper is clear - I recommend acceptance because of the potential value. The theorems apply only to the four losses mentioned, and I was not able to go through the proofs in full detail so I have some doubts about how they would generalize to efficient algorithms otherwise; however, I had more significant doubts after reading the original Nock et al. 2015 paper, and clearly the theory has grown since then. It is indeed interesting that the new boosting algorithm can be implemented as efficiently as the standard ones, but I view this as a side consequence of the formulation and have therefore focused on the remainder in my evaluation.   There are several places in the paper with typos (e.g. line 101, 217), and the material can get dense for a general machine learning audience, though it is mathematically clean and precise. E.g., the discussion of the affinely generated functions in Sec. 2 would be much clearer if re-ordered, so that the motivation for the definitions and choices is clear (e.g. rados represent powersets, so the powerset indexing would clear up significantly). For similar reasons I suggest defining (1) and (2) as a different script letter instead of L, due to Eqs. (11,12) and the possible confusion with losses.  Following from the paper are several interesting and open questions with possible connections to other areas of machine learning. I am interested in the connection to the two-player game formulation, because it suggests different parametrizations of the problem by choosing the z's differently for other learning settings. The z's are currently chosen in a similar way to the boosting game, this conflation of information happens due to linear constraints on the z_i's (weak learning assumption), so they are chosen in this setting with benefits. But the results in Sec. 2 are general and there are other similar game-theoretic formulations in areas like (e.g. online) learning. Are there other ways of condensing the data, besides rados, that are similarly beneficial for those problems?


Multi-armed Bandits: Competing with Optimal Sequences
We consider sequential decision making problem in the adversarial setting, where regret is measured with respect to the optimal sequence of actions and the feedback adheres the bandit setting. It is well-known that obtaining sublinear regret in this setting is impossible in general, which arises the question of when can we do better than linear regret? Previous works show that when the environment is guaranteed to vary slowly and furthermore we are given prior knowledge regarding its variation (i.e., a limit on the amount of changes suffered by the environment), then this task is feasible. The caveat however is that such prior knowledge is not likely to be available in practice, which causes the obtained regret bounds to be somewhat irrelevant. Our main result is a regret guarantee that scales with the variation parameter of the environment, without requiring any prior knowledge about it whatsoever. By that, we also resolve an open problem posted by Gur, Zeevi and Besbes [8]. An important key component in our result is a statistical test for identifying non-stationarity in a sequence of independent random variables. This test either identifies nonstationarity or upper-bounds the absolute deviation of the corresponding sequence of mean values in terms of its total variation. This test is interesting on its own right and has the potential to be found useful in additional settings.

The paper studies the stochastic Multi Armed Bandit problem, where the distributions of the arms could change over time. The authors approach the problem by first proposing a testing procedure to detect if there is a significant change in the means in an offline set (Test 1) and online series (Test 2) of independent random variables. They use these tests to design an online alg for the problem in the case of two arms The alg could be outlined as follows:  - It dynamically divides the time horizon into blocks; each block consists of an exploration phase, and a possibly null exploitation phase.  - During the exploration phase, the alg randomly pulls each arm with equal probability. If a change in either of the arms is detected, it goes to the next block (nil exploitation phase). If no (essential) change is detected and one arm is "significantly" better than the others, it goes to the exploitation phase.   - The exploitation phase favors playing the better arm, but the length of the phase and the probability of playing the perceived better arm are weighed according to the probability that the exploration phase makes the correct identification.   The main novelty of the algorithm is the proposal of Test 2, which requires the decision maker to actively and explicitly detect the change in the arms' distributions. This appears a novel aspect (at least to my knowledge) that is not featured in the current stochastic and adversarial MAB literature. While the paper is very closely related to a recent work by Gur, Zeevi and Besbes (2014), a main difference is that the amount of variation in the arms distributions is not known in the current paper's setting.
Minor typos:  Line 238 - 239: In the definition of \epsilon_2(n_1, n_2) on page 6 (between line 238 and 239), the quantity is undefined when n_2 = n, n_1 = 1. Is there a typo?  Line 302: \Delta \geq \tau^{-\lambda^2} should be \Delta \geq \Theta(\tau^{-\lambda^2})  Line 424: The reference for Hoeffding inequality is missing.  Suggested additional references:  A. Garivier and E. Moulines, "On upper-confidence bound policies for non-stationary bandit problems" in Algorithm Learning Theory (ALT), ser. Lectures notes in Computer Science, vol. 6925, 2011   Comments:  The proposed algorithm is novel in the sense that it actively detects if there is any change in the arms' distribution, which is quite different from the current literature, including the one above. Designing online algorithm that competes well against the dynamic benchmark is clearly an important topic in MAB, and the paper makes a major step by designing such an algorithm without knowing the variation budget. Indeed, previous works such as [1, 7] referred in the manuscript requires knowing the variation budget beforehand to tune the learning rate, and it is quite surprising that one can achieves a sublinear regret without the knowledge. Thus, I think this submission will be interesting to the MAB community. The paper is quite clearly written, given the level of technicality.   Finally, what is the regret order bound with K arms?


Sub-sampled Newton Methods with Non-uniform Sampling
We consider the problem of finding the minimizer of a convex function F :
where a low-rank factorization of ∇ 2 f i (w) is readily available. We consider the regime where n d. We propose randomized Newton-type algorithms that exploit non-uniform sub-sampling of
, as well as inexact updates, as means to reduce the computational complexity, and are applicable to a wide range of problems in machine learning. Two non-uniform sampling distributions based on block norm squares and block partial leverage scores are considered. Under certain assumptions, we show that our algorithms inherit a linear-quadratic convergence rate in w and achieve a lower computational complexity compared to similar existing methods. In addition, we show that our algorithms exhibit more robustness and better dependence on problem specific quantities, such as the condition number. We empirically demonstrate that our methods are at least twice as fast as Newton's methods on several real datasets.

The focus of this paper is improving convergence times for Newton's method by approximating the Hessian during each iteration.  The proposed algorithm applies to smooth, strongly convex objectives subject to convex constraints.  An important assumption is that the Hessian is a sum of known low-rank matrices and one additional matrix.  The main idea is to approximate the Hessian by intelligently subsampling (and properly rescaling) the low rank matrices.  The Newton step is then solved approximately.  Prior works considered uniform sampling for approximating the Hessian.  This paper studies novel non-uniform sampling techniques based on the norm or standard leverage scores of the Hessian components.  In the theoretical results section, the paper builds off recent work on fast matrix approximation methods to derive sufficient asymptotic sample sizes for the algorithm to achieve a local linear convergence rate with high probability.  An important point about this result is that unlike uniform sampling, the sufficient sample size for the proposed method does not depend on the problem's total number of training examples.  The section also clearly compares the time complexities of the proposed and related algorithms.  The theory allows the different sampling schemes to be compared based on their dependence on the problem's condition number, which is interesting.  The paper only focuses on local convergence guarantees and only very briefly refers to other work regarding what to do in the initial phase of the algorithm.  In the empirical section, the paper considers logistic regression problems with l2 regularization.  The proposed Hessian sampling approaches are compared to uniform sampling, Newton's method, and LBFGS.  The experiments focus on obtaining highly accurate solutions for data sets with roughly 100 features and 100k examples.  The proposed methods seem to clearly outperform uniform sampling, while standard Newton is a closer competitor.
There are some obvious criticisms for this paper.  First, the algorithm focuses on finding highly precise solutions to loss minimization problems with many more examples than features.  For most real world prediction tasks, it is well-known that highly precise solutions are not necessary.  Second, it is not clear from the experiments whether the algorithm significantly improves upon the conventional Newton's method.  It seems Newton's method consistently displays the same or faster local convergence speed compared to the proposed methods, and Newton's method is only slower in early stages.  This observation is not discussed, and there are not sufficient experiment details to understand why this is the case (for example whether the same subproblem solver is used for both methods, which steps of the algorithms dominate iteration times, what type of backtracking is used).  This is especially an issue, since the analysis focuses only on the local convergence phase, and it is not clearly stated what the algorithms do in the initial phase (see footnote 1 in the paper).  On a related note, the sampling size for each method is chosen to optimize convergence progress during the initial stage of the algorithm (footnote 5 in the paper), and the uniform sampling method looks competitive in this phase.  However, the algorithms are then evaluated based on their local convergence speed, calling into question the fairness of the parameter tuning.  Finally, compared to Newton's method, the proposed method has additional parameters to tune (sample size, when to use stale leverage scores), and the data set sizes are quite small.  For these reasons, I am not convinced the algorithm will be useful in practice.  As a whole, however, the paper has several appealing aspects.  Assuming that the conclusions of the empirical section are correct, I was surprised that the somewhat complicated partial leverage scores sampling scheme significantly outperforms uniform sampling.  I imagine these subsampling and matrix approximation ideas could be useful in other areas of optimization and machine learning, and this paper could serve as a template for applying this ideas.  The theory is also interesting in the sense that the different methods can easily be compared based on their dependence on the number of training examples as well as the problem's condition number.  The sections of the paper also complement each other fairly well, as the theory can be used to explain several of the empirical results.  Finally, the presentation is for the most part very clear.  Some suggestions/questions for the authors to potentially consider are: 1. Think about defining the objective as \sum_i f_i(a_i^T w) + R(w), so that the Hessian structure is more explicit.  Also, what is the benefit of defining the A_i's as matrices and not vectors (i.e. k_i >= 1 vs. k_i = 1)?  This "block" notion was consistently confusing to me, and it is not clear to me what additional objectives this allows. 2. Also consider removing the constraint from the problem formulation. The constraint does not seem to be important to the paper, and it causes some confusion when stating time complexities of the subproblem solvers. 3. Lemma 2 is presented without much discussion.  To make the theory section easier to follow, consider briefly discussing the implications of meeting conditions C1 and C2 immediately after the lemma. 4. The sample size (in)dependence on n is framed as an important point.  For me at least, it was not obvious how the sr(A) and \tau(A) quantities in the bounds related to n, if at all.  It may be helpful to make this more clear in the text. 5. Consider including empirical results (likely in an appendix) that demonstrate the trade-offs between the time required and approximation error for each way to construct the Hessian.  This would be helpful in understanding the convergence vs. time curves. 6. It seems to be that for row norm squares sampling in the logistic regression case, the norms of the feature vectors can be cached to perform the sampling in O(n) time.  If this is true, it might be worth mentioning. 7. Consider including empirical results that illustrate the sample size bound's independence on n. 8. Think about removing the last paragraph of Section 4 to add additional commentary elsewhere.


The non-convex Burer-Monteiro approach works on smooth semidefinite programs
Semidefinite programs (SDPs) can be solved in polynomial time by interior point methods, but scalability can be an issue. To address this shortcoming, over a decade ago, Burer and Monteiro proposed to solve SDPs with few equality constraints via rank-restricted, non-convex surrogates. Remarkably, for some applications, local optimization methods seem to converge to global optima of these non-convex surrogates reliably. Although some theory supports this empirical success, a complete explanation of it remains an open question. In this paper, we consider a class of SDPs which includes applications such as max-cut, community detection in the stochastic block model, robust PCA, phase retrieval and synchronization of rotations. We show that the low-rank Burer-Monteiro formulation of SDPs in that class almost never has any spurious local optima. This paper was corrected on April 9, 2018. Theorems 2 and 4 had the assumption that M (1) is a manifold. From this assumption it was stated that
, which is not true in general. To ensure this identity, the theorems now make the stronger assumption that gradients of the constraints A(Y Y ) = b are linearly independent for all Y in M. All examples treated in the paper satisfy this assumption. Appendix D gives details.

This paper considers a specific class of semi-definite programs (minimize linear objective subject to linear equally and positive semi-definiteness constraints). This convex optimization problem is replaced by a non convex one where the optimization variable is replaced by the outer product of a matrix and its transpose. The paper shows that under certain conditions (size of the factorization large enough. compact and smooth search space), all second-order critical points are globally optimal.   More specifically, the paper is very closely related to prior work by Burer and Monteiro. I think [1] proposed the change of variables X=YY^T, proposed also an BFGS augmented Lagrangian algorithm for solving the nonconvex problem, and showed experimentally that it reliably gives a global minimum if number of columns of Y is large enough. Then, [2] showed that the change of variables does not introduce extraneous local minima, that rank deficient local minima are global minima, and that a minor modification of BFGS is such that if a local minimum is obtained at each stage of the algorithm, then any accumulation point is a rank deficient local minima, and hence a global minima. So the main open problem with Burer and Monteiro appears to be that guaranteeing that the algorithm returns a local minima at each iteration. The core contribution of this paper is to address this issue. This is done by making two additional assumptions: (1) the set {X: A(X) = b} is compact, and (2) the set {Y: A(YY^T) = b} is a smooth manifold. These two assumptions allow the use of a Riemannian trust region method, which is guaranteed to converge to a second order critical point (new result from Boumal, P.-A. Absil, and C. Cartis in preparation which I did not have access to). As a consequence, it is not necessary to check that one is at a local minima at each step of the algorithm: one has guaranteed convergence to a second order critical point (as per new results from Bournal). Then, this paper shows that second-order critical points are globally optimal. So the core contribution is to not only show that rank-deficient local minima are global, but to also provide an algorithm that can find them.  [1]  S. Burer and R.D.C. Monteiro. A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Mathematical Programming (Series B), 95:329–357, 2003.  [2] S. Burer and R.D.C. Monteiro. Local minima and convergence in low-rank semidefinite programming. Mathematical Programming, 103(3):427–444, 2005.
I believe that (modulo some notational issues described below) the paper is well written, interesting, and worth knowing about. I found it interesting that the paper treats the constraint set as a manifold and uses Riemannian optimization techniques not just as an algorithm (Riemannian trust region) but also to establish conditions for optimality. Also, I liked that the paper provides specific bounds on how far the value of the objective is from the optimal value, and the fact that the framework is applicable to important applications, e.g., Max-cut.  On the other hand, a key limitation of the paper is that it addresses a restricted set of optimization problems (sub-class of SDPs, whose constraint in factorized form is a compact and smooth manifold). In addition, the paper is less novel than it appears given recent results that are not discussed. In particular, [1] considers the potentially more general case where the matrix to be factorized is not symmetric, and so the factors are not constrained to be equal, the objective function is not constrained to be quadratic, [2] extends [1] to account for conic constraint and gauge functions, and [3] further extends the results to non-differentiable objectives and uses simpler alternating proximal gradient methods for minimizing the objective. While the results in this paper need need not be a corollary of [1,2,3], especially because [1,2,3] use some form of regularization, I believe a discussion of the differences would help further appreciate the novelty of the present paper since [1,2,3] also have results like “if the size of the factorization is large enough, local minima are global”.   [1] F. Bach, J. Mairal, and J. Ponce, “Convex sparse matrix factorizations,” arXiv:0812.1869v1, 2008. [2] F. Bach, “Convex relaxations of structured matrix factorizations,” arXiv:1309.3117v1, 2013. [3] B. Haeffele, E. Young, and R. Vidal, “Structured low-rank matrix factorization: Optimality, algorithm, and applications to image processing,” in International Conference on Machine Learning, 2014.  Other comments are that in the Proof of Theorem 4, S(Y) seems to be undefined. Then, in footnote 1, lambda_min(S) is mentioned, so S seems to be a matrix rather than a function. Overall, notation is undefined and sloppy at times.  Minor comments:   P2L33: The acronym  L-BFGS has not been introduced. P2L70: The statement "Since (SDP) is a relaxation of (QCQP) (up to parameterization)” is not clear. In what sense is a relaxation and what parameterization are you referring to? P3L79: The statement "cost matrix admissible for our theory” is not unclear until one reads Theorem 1. Such terminology should be introduced in line 70.


Interpretable Nonlinear Dynamic Modeling of Neural Trajectories
A central challenge in neuroscience is understanding how neural system implements computation through its dynamics. We propose a nonlinear time series model aimed at characterizing interpretable dynamics from neural trajectories. Our model assumes low-dimensional continuous dynamics in a finite volume. It incorporates a prior assumption about globally contractional dynamics to avoid overly enthusiastic extrapolation outside of the support of observed trajectories. We show that our model can recover qualitative features of the phase portrait such as attractors, slow points, and bifurcations, while also producing reliable longterm future predictions in a variety of dynamical models and in real neural data.

The authors demonstrate a fitting approach using basis functions and a regularization term to robustly fit nonlinear dynamical systems. The inclusion of the global regularization term is their main contribution, and they clearly illustrate how it makes their procedure robust against test cases outside of the training range (Table 1 & Fig 6 in supplementary material). They demonstrate their approach for various types of dynamics: a fixed-point attractor, FitzHugh-Nagumo, a ring attractor and Lorentz chaotic dynamics. They also very briefly discusses how the approach can be applied to neural recordings.
Overall I found the paper to be solid and rather enjoyable, and I would qualify it as a strong candidate for a poster. The authors' method of plotting velocity fields by decomposing the velocity into direction and speed, which they've apparently introduced, is especially effective. It made their arguments and conclusions much easier to follow, and will hopefully be picked up by others.  In my opinion stating that this approach leads to “interpretable models” might be somewhat overselling the results – the interpretability of the results is still hampered by the fact that models are composed by 10-100 more or less arbitrary basis functions. That being said, their capacity to reproduce salient features of the phase diagram certainly makes them more interpretable than, say, recurrent neural networks.  *** Update following authors' response *** "Interpretable model" can mean many things. For instance, it can mean that the value of each c_i, σ_i,… is interpretable – the meaning I originally had in mind – or that the model results in an interpretable phase diagram – the meaning the authors had in mind. It should suffice to define the desired meaning the first time this expression is used.  Discussions are for the most part appropriately detailed, although sometimes they feel somewhat condensed, perhaps due to the large number of examples the authors discuss within their allowed 8 pages. Specific issues are noted below.  *** Section 2 — Model - The linearization in eq. (2) seems incorrect to me. One typically linearizes an update function (F here) around some other point (say x*) in the neighborhood of x_t, so that we have something of the form x_{t+1} = x_t + A(x*)(x_t - x*) + B(x*)(x_t - x*)u_t + ε_t We can then collect the constant terms as b(x_t) = -A(x*)x* - B(x*)x* Note that it is critical that x* and x_t be different in general, otherwise the linear terms are always zero. This distinction seems to be made in the given ref. [9], but it is not done here. Thus eq. (2) needs to be corrected, or if it is correct, a more detailed derivation needs to be given. These issues are further illustrated by the comment that A is not unique: if A(x_t) is already defined as the Jacobian of F_x at x_t, how can it not be unique ? Fixing the derivation of (2) should help clarify this comment.  *** Update following authors' response *** I think the issue boils down to clarifying that b(x_t) and B(x_t) are not linearizations around x_t, but rather around some other point h(x_t). It would then be clear that they are not unique, as they depend on the function h. One should probably write b(x)=Fx(h(x)) on line 38.  - line 39: B should be a function from ℝ^d → ℝ^d x ℝ^{di}.  - The previous issue might also be addressed by simply removing the first part of section 2 up to “redundant and over parametrizes the dynamics.” Indeed, this whole section is never used for the rest of the paper. I found eq. (3) a clearer starting point for the arguments than eq. (2).  - The authors should state that W_g, W_B and τ all depend on the step size Δ.  - Eq. (4). I feel that the regularization term should be exp(-τ² x_t). If it is correct as is, the authors should comment on the benefit of writing the regularization constant in this way (rather than just using a constant β ≡ exp(-τ²) ).  - line 61: Unless B is constant, the Jacobian should also include a term ∂B/∂x.  - line 63: As far as I am aware, “ghost point” is not a frequent term in studies of dynamical systems. Perhaps the authors could provide a reference, or a short definition ? Same comment regarding “slow point”, which first appears on line 92.   *** Section 4.2 — Nonlinear oscillator model - Since the text refers to figures 2a-2d, the subfigures should show alphabetic labels.  *** Section 4.3 — Ring attractor dynamics To me this is the most interesting of the presented examples. The fact that only a subset of qualitative features are reproduced makes this a good test case to explore what this fitting procedure can and cannot do. Unfortunately the authors limit their discussion to a description, simply stating that the fit and true models show qualitatively different behavior. I think it would be worth trying to distill a what parts of the dynamics are difficult to fit and why. For instance, why is it that the radial dynamics seem to completely dominate Fig 3a, even along the ring (where dr/dt should be almost zero) ?  - Fig 3b : As far as I can tell, there are in fact two fit and two true trajectories in this figure (distinguished by dashed and full lines). The legend says there is one of each.  *** Section 4.4 — Chaotic system - Fig 4c : The true trajectories are almost impossible to see. Also the line styles in the legend should match those in the figure.  *** Section 5 — Learning V1 neural dynamics This section feels incomplete. The LDS model used is claimed to be the “best”, but no justification or reference is given to support this claim. In fact it is not even stated which LDS model is used – this could be included in the supplementary material, similar to what was done for section 4.1. The example studied in this section is by far the paper's most complex, and yet its explanation is one of the shortest. Consequently, it is difficult to picture what it is exactly that the model is fitting. Ideally a figure would be added to illustrate the inputs, but at the very least the second paragraph should be lengthened to include a more precise description of the inputs. As it stands, it is not even specified what time of recording the data stems from (only that it is “large scale”).  *** Supplementary material - As far as I understand, the interest in giving eq. (15) in terms of the x_i is to relate these dynamics to physical quantities. Since the authors never do this in the text, I would remove the middle expression in eq. (15). Similarly, I think defining x_2 in eq. (16) is superfluous; moreover, x_1 is never properly defined. The expressions in terms of s_1 and s_2 already appear as-is in ref. [20]. Also, the parameters J_{A,ext} and c' which appear in eq. (17) are never defined. - In fig 7, green and red dots should be above the blue lines rather than below.  *** Grammar  Some minor grammatical typos were noted, which in no way affect the quality of the paper. Possible corrections are listed here for the author's reference. 4: Our model incorporates a prior assumption 7: bifurcations 17: dynamical systems [5,6] 27-28: These features encourage the model 36: of an autoregressive 40: in the multivariate RBF 60: dynamics are locally 61: via the Hartman-Grobman 82-83: where the tasks typically involve 150: The initial state of each trajectory is picked from a standard normal 151: We use 19 trajectories 152: and the last one for testing. 166: The input was 3 dimensional 173: This end of sentence could be better formulated: “robustly predicts well.”


Quantized Random Projections and Non-Linear Estimation of Cosine Similarity
Random projections constitute a simple, yet effective technique for dimensionality reduction with applications in learning and search problems. In the present paper, we consider the problem of estimating cosine similarities when the projected data undergo scalar quantization to b bits. We here argue that the maximum likelihood estimator (MLE) is a principled approach to deal with the non-linearity resulting from quantization, and subsequently study its computational and statistical properties. A specific focus is on the on the trade-off between bit depth and the number of projections given a fixed budget of bits for storage or transmission. Along the way, we also touch upon the existence of a qualitative counterpart to the Johnson-Lindenstrauss lemma in the presence of quantization.

NIPS 2016 Review of Paper ID 1404 “Quantized Random Projections and Non-Linear Estimation of Cosine Similarity”  This manuscript studies the problem of estimating cosine similarities when the projections are quantized to an arbitrary number of bits. Previous work has examined random projections with full precision (i.e. real-valued projections) and random projections that have been quantized to a single bit (i.e. binary-valued responses). This paper focuses on the middle ground. In particular, they examine a maximum likelihood estimator- presumably of cosine similarity under the quantization non-linearity--the trade-off between bit depth vs. the number of projections for storage or transmission (i.e. compression).
The topic is of potential interest, but the authors wrote this manuscript with a specialized audience in mind. I am a non-specialist. As a consequence, I am not in a good position to evaluate the merits or the importance of the work to the authors’ academic community. However, I am in a position to evaluate the writing and the presentation of the submitted manuscript. It is clear that the manuscript would benefit from more thorough and methodical writing. Figures and figure captions are not carefully explained; it is often not clear whether the data presented in the figures buttress the claims and conclusions made by the authors. Sometimes, parameters are used that are never defined. In general, while the statement of the problem is clear and the mathematical development is largely clear and easy to follow, the numerical results presented by the authors are very difficult to make sense of.     Line 61: “for its first resp. second derivative.”  What is the first resp second derivative? Typo?  Line 129: “resp.”  resp.?   Line 131: “The assumption that the norms are known is reasonable since  1) information about the norms is lost because of quantization (a total loss occurs for b = 1 ) and 2) it  is not hard to satisfy in practice as they can be computed by one linear scan during data collection.”  Confusing, possibly incoherent sentence. How can it be reasonable to assume that the norms are known if information about the norms is lost.   Figure 3 Caption: What do the number in the first subplot’s legend indicate?   Line 268; “We perform 100 resp. 20 independent replications for synthetic resp. real data”  What does ‘resp.’ mean? The authors use it repeatedly in the manuscript. I do not know what it means and I could not decipher either from context or a google search what the authors intend it to mean.   Line 279: “Some of these findings  are reflected in Figures 4 and 5. We refer to the supplement for additional figures.”  Which findings? I could not figure out what the figures showed. The authors assert that their results can be summarized by the fact that many bit quantization is superior to one bit quantization. This seems like a sensible conclusion, but it is not clear how Figure 4 and 5 show that. Consider figure 5. First, the y-axis has a different scale for each subplot, so comparisons between b=2 and b=4 are not easy in the farm or rcv1 datasets. This is non-standard and should be rectified. Second, if one does take the time to compare the numbers for the MLE estimator (black curve), the numbers are nearly identical for both datasets. That is, it is not at all clear from the figure that a larger number of bits (2 vs 4) indeed yields better retrieval of the K nearest neighbors. It would also be nice to know whether the 0.92 fraction retrieved in the farm dataset is considered ‘good performance’.   Line 290: “For SVM classification, we  consider logarithmically spaced grids between 10^3  and 10^3  for the parameter C”  The SVM parameter C is never described or explained.  Line 301: “there is empirical evidence that using three or a higher number potentially matches the performance achieved in the full precision case.”  Three or a higher number of what? Bits?  Appendix Figure 1: “(Re-scaled) Empirical MSE k _  (b_MLE􀀀__ )2  for (averaged over 104  i.i.d. data sets with k  = 100) compared to the inverse information.”  Nonsense first sentence of caption.  Appendix Figure 5: “ verage fraction of K  = 10 nearest neighbors retrieved vs. total # of bits (log2  scale) of our approach (MLE) relative to that based on the Hamming distance and the linear approach.”  Typos.


Efficient Neural Codes under Metabolic Constraints
Neural codes are inevitably shaped by various kinds of biological constraints, e.g. noise and metabolic cost. Here we formulate a coding framework which explicitly deals with noise and the metabolic costs associated with the neural representation of information, and analytically derive the optimal neural code for monotonic response functions and arbitrary stimulus distributions. For a single neuron, the theory predicts a family of optimal response functions depending on the metabolic budget and noise characteristics. Interestingly, the well-known histogram equalization solution can be viewed as a special case when metabolic resources are unlimited. For a pair of neurons, our theory suggests that under more severe metabolic constraints, ON-OFF coding is an increasingly more efficient coding scheme compared to ON-ON or OFF-OFF. The advantage could be as large as one-fold, substantially larger than the previous estimation. Some of these predictions could be generalized to the case of large neural populations. In particular, these analytical results may provide a theoretical basis for the predominant segregation into ONand OFF-cells in early visual processing areas. Overall, we provide a unified framework for optimal neural codes with monotonic tuning curves in the brain, and makes predictions that can be directly tested with physiology experiments.

This work analyzes information transmission in neurons with monotonic tuning curves under resource and noise constraints. Given a small set of assumptions it demonstrates analytically that splitting into ON and OFF channels maximizes information transmission in a neural population with limited resources.
This is a high quality submission with a potentially high impact on the field. It generalizes and extends previous results [19, 30]. Moreover, it provides a possible theoretical foundation for results obtained with statistical models of natural stimuli [13]. What I especially like about this paper is the generality of obtained results. The authors do not limit the analysis to a single constraint but consider a number of them at the same time: dynamic range, average activity level and a family of noise models. The results are independent of stimulus distribution and make very little assumptions about the shape of tuning curves (monotonicity). Despite the "density" of the paper the presentation is very clear.  There are a few questions/remarks I have, which I think could be addressed in the text in order to further improve the quality the paper and interpretability of the results:  1) Are proposed coding constraints independent? There seems to be a relationship between the maximum attainable $r$ and average $K(r)$. If yes - can we treat them independently as the authors do?  2) The results presented in the paper are independent of the stimulus distribution. It is of course a strength, however a number of previous studies (e.g. [13]; Ratliff et al., PNAS 2010) suggest a strong connection between the statistics of natural stimuli and the presence of "ON-OFF" channels. How shall one interpret present results in light of previous statistical findings?  3) The above question becomes especially relevant with respect to section 4 - "Optimal coding of large neural population". You show that an optimal solution for a population of neurons is to divide it into two equally large ON and OFF subpopulations. It is known however, that there is a larger proportion of OFF units in the retina. Moreover it seems to be explained by the excess of dark spots in natural images (Ratliff et al., PNAS 2010) i.e. stimulus distribution. How would you explain that discrepancy?   4) Please provide a brief justification in the main text for the assumption of even number of neurons ($N = 2k$).  5) Line 207 includes a remark "show in figure". While I understand that it's a leftover from the editing process, I agree it would be good to show on a picture how ON and OFF tuning curves tile the input space (since, as I understand this is the solution).  6) I like the reference to opponent-channel codes in the auditory system. There have been results published suggesting that this sort of representation maximizes information transmission given the statistics of the natural auditory input [Harper, McAlpine, Nature 2005; Mlynarski, Plos CompBio. 2015]. I would however urge the authors to be careful with this comparison. Spatial tuning curves in the auditory brainstem and cortex are non-monotonic (they peak at extremally lateral positions, sometimes outside of the so-called "physiological range"), and they strongly overlap at the midline (which you address in the text already).   Minor comments:  It might be a good idea to add a small color legend to your figures explicitly saying that yellow / red spectrum corresponds to different constraints.   Figure 2 - I would mention that it is for a uniform variable $u$, not $s$  Linr 152 - supplementary material mentions that due to monotonicity of the tuning curve one of A^+ or A^- regions has to be empty. It would be good to write that in the main text for clarity.


Supervised Learning with Tensor Networks
Tensor networks are approximations of high-order tensors which are efficient to work with and have been very successful for physics and mathematics applications. We demonstrate how algorithms for optimizing tensor networks can be adapted to supervised learning tasks by using matrix product states (tensor trains) to parameterize non-linear kernel learning models. For the MNIST data set we obtain less than 1% test set classification error. We discuss an interpretation of the additional structure imparted by the tensor network to the learned model.

This paper deals with the use of tensor networks to perform supervised learning. Specifically, the authors shows how tensor networks can be adapted to produce useful representation of data and solve classification problems. The main idea is to consider data feature maps that are built from the tensor product of local features and represent them as a higher-order tensor. A classification problem then consists in viewing the model parameters as a tensor network and learn them using a matrix-product state (MPS) decomposition. An optimization algorithm to find the MPS decomposition is described and an experiment on MNIST digit data set is given. Finally, a discussion on the class of functions in which the solution is searched is provided.
The idea of using tensor networks for machine learning problems is interesting. Unfortunately, there are many important issues that prevent the publication of the manuscript as it is now. My main concerns are the following:   1- Some parts of the paper are hard to read. The paper applies and adapts some idea about tensor networks that are used/developed by the Physics community to ML problems. The authors have done a good job in illustrating the idea and the intuition by many figures. However, I think that for a ML audience, giving more details on tensor networks, MPS decomposition algorithms and their use for Physics problems (in short, more details on the papers cited and published in Physics journals) will improve the readability and the accessibility of the paper. A section can be dedicated to this.  2- The notation is also heavy to follow and dos not help understanding. For example:  * In equation 2, what is the relation between x and x_j? Is x_j a component of the vector x? What is N: is it the number of attribute?  * In equation 3, \phi is indexed by s_j but depends only in x_j. For this example, is it the same \phi for all the s_j?  * A needs to be defined in equation 5  * line 68: ‘in classifying labeled data’ should be ‘unlabeled data’ (from a given training set)  * …  3- Experiments are weak. To assess the performance of the method, more data sets have to be used and not only the simple MNIST digit data. Also, some informations about the experimental setting are missing. For example, How the data set was divided in training and testing data?  4- I think that the choice of the local feature map is crucial. It is not clear how this feature map can be chosen. The one given in Eq. 3 seems to be adapted only for image data. The same for the parameters d and m. How these parameters can be chosen?  5- Is the MPS decomposition unique? If no, how this affects the learning process?  6- MPS decomposition for feature extraction and classification were used in [4]. I think that the novel contributions compared to [4] need to be clearly stated. Line 210 is too short and does not give enough details to see the difference between the two works.  7- ANOVA kernels are based on the product of a set of ‘local’ kernels. The product of the kernels allows to find a solution in a Hilbert space  constructed from the tensor product of the reproducing Hilbert spaces associated to each local kernel. I Think that it is interesting to study the link between the space of functions obtained by such kernels and the tensor network representation proposed here.


DeepMath -Deep Sequence Models for Premise Selection
We study the effectiveness of neural sequence models for premise selection in automated theorem proving, one of the main bottlenecks in the formalization of mathematics. We propose a two stage approach for this task that yields good results for the premise selection task on the Mizar corpus while avoiding the handengineered features of existing state-of-the-art models. To our knowledge, this is the first time deep learning has been applied to theorem proving on a large scale.
• A demonstration for the first time that neural network models are useful for aiding in large scale automated logical reasoning without the need for hand-engineered features.
• The comparison of various network architectures (including convolutional, recurrent and hybrid models) and their effect on premise selection performance.
• A method of semantic-aware "definition"-embeddings for function symbols that improves the generalization of formulas with symbols occurring infrequently. This model outperforms previous approaches.
• Analysis showing that neural network based premise selection methods are complementary to those with hand-engineered features: ensembling with previous results produce superior results.

This paper considers the problem of premise selection for automated theorem proving: given a set of axioms (premises) and a conjecture, select a subset of axioms to provide the theorem prover that will allow proving the conjecture. The approach is to embed axioms and conjectures into real-valued vectors, and then map the concatenation of embeddings to a classification of whether the axiom is relevant for proving the conjecture. A variety of neural network architectures are explored for representing the axioms and conjectures, including recurrent and convolutional neural network approaches. In addition, a two-stage approach is developed, where embeddings for identifiers are created by embedding the identifier’s definition. Results show that when a small number of premises are returned (up to 64), a k-NN baseline performs best. When up to 1024 premises are returned, one of proposed models proves 62.95% of conjectures versus the k-NN baseline’s 59.34%.
Pros:  * The problem is interesting and does not appear to have previously received attention from the NIPS community  * The paper is clearly written  Cons:  * The approach is not particularly novel; it is mostly a case of applying a range of existing neural network components to the premise selection problem.  * The improvements achieved over a k-NN baseline are small, and I have some reservations about how the experimental results are reported (see below)  Experiments  * Numbers are presented for the performance of previous work in Section 3, but they don’t appear to be comparable to the results reported in the experiments (Section 6). Is there a way to meaningfully compare these numbers? If not, it's confusing to include them.  * Table 1 is not a fair way of reporting results. This presentation is asking us to take a maximum of test performance over all the different neural network variants, and to compare that to the performance of a single baseline method. However, there is quite a lot of variance in the neural network performance relative to the difference between neural network and baseline performance. Thus, there is a bias towards selecting one of the neural networks as the best method, even though it could just be due to the fact that we’re taking a max over many noisy performances. In section 3 it is mentioned that a portfolio of simple learners significantly outperform any single learner. If such an extensive search is being done over tweaks to the proposed method, then an equally extensive search should be done over baseline methods.  Presentation  * Please number Figures 5 and 6  * Figure 5 isn’t readable in black and white  * I find the title off-putting. “DeepMath” is a very general term relative to what the paper actually does (premise selection for automated theorem proving). It sounds like a title meant to grab headlines as opposed to being descriptive of the scientific contributions.


Learning Additive Exponential Family Graphical Models via ℓ 2,1 -norm Regularized M-Estimation
We investigate a subclass of exponential family graphical models of which the sufficient statistics are defined by arbitrary additive forms. We propose two ℓ 2,1 -norm regularized maximum likelihood estimators to learn the model parameters from i.i.d. samples. The first one is a joint MLE estimator which estimates all the parameters simultaneously. The second one is a node-wise conditional MLE estimator which estimates the parameters for each node individually. For both estimators, statistical analysis shows that under mild conditions the extra flexibility gained by the additive exponential family models comes at almost no cost of statistical efficiency. A Monte-Carlo approximation method is developed to efficiently optimize the proposed estimators. The advantages of our estimators over Gaussian graphical models and Nonparanormal estimators are demonstrated on synthetic and real data sets.

This paper proposes parameter estimators for exponential family graphical models in which the univariate and pairwise sufficient statistics are representable as linear combinations of pre-defined basis functions.  Two regularized maximum likelihood estimators are presented, the first for joint estimation of all parameters, the second for conditional estimation of the parameters associated with each node.  A Monte-Carlo approximation to certain expectations is used to simplify the algorithms. Finite-sample statistical guarantees are established on the estimation error, which agree with existing results on Gaussian and nonparanormal graphical models. Experimental results show greater flexibility in modeling nonlinear interactions between variables. I think this last aspect is the main strength of the paper compared to existing approaches, in addition to the theoretical results.
Most important: 1. In Theorem 1, the error rate is as expected since max{q,r} ||theta*||_{2,0} is approximately the number of parameters in the model.  I think the factor of max{q,r} should not be omitted from Section 1.1, last paragraph so as not to be misleading.  I am surprised however that the same factor of max{q,r} is not present in the error rate in Theorem 2, being replaced instead by (q+r) inside the logarithm. This result should be checked since if I understand correctly, the number of parameters is still something like q or r times ||theta*_s||_{2,0}. If Theorem 2 is correct as currently stated then a more thorough justification in words is needed.  Major comments:  2. I think it may be inaccurate to call the model "semi-parametric" because the set of basis functions is finite and fixed.  Moreover the dimensions of the bases used in the experiments are small (< 10).  3. Related to the point above, I believe the proposed approach shifts the burden onto choosing an appropriate set of basis functions for the data at hand. For the simulated data in Section 4.1, the bases are chosen to be a perfect fit or at least a good fit to the true form of the interactions.  A case where the basis is not well-matched would be of interest.  In general, the choice of basis is an issue deserving of more attention.  4. The definition of groups is not clear in the Notations subsection and remains so until below (7) where the (2,1)-norm is defined. Why are node groups penalized the same as edge groups?  5. In Proposition 1, definition of sigmahat, should it be sigmahat^2 instead? The RHS looks like a variance.  6. I am not seeing how the number of Monte Carlo samples m was chosen in the experiments.  7. In Section 4.2, the use of stock category as ground truth for links seems a bit questionable since stocks in different categories could also be strongly dependent, while not all stocks in the same category may be dependent.  Minor issues:  8. Section 1, paragraph 1: "undirected graph consists" --> "undirected graph consisting"  9. In Section 1.2, the second half of the paragraph could be better organized. Do [9,27] belong to the nonparanormal approach?  It appears that [7,20,8] are fully non-parametric methods.  This could be emphasized with a brief discussion of the advantages/disadvantages of non-parametric models vs. semi-parametric ones and the proposed model.  10. Below (10) in the definition of ||theta||_{2,1}, I think the summation should be over t in N(s).  11. Just above Proposition 1, is "A.1" an appendix in the supplementary material?  12. In Theorems 1 and 2, the constants c_0 and ctilde_0 are undefined. In Theorem 2, there may be missing tildes e.g. on delta.  13. In Section 4.1, is performance reported on the training sample, after the tuning sample has been used to select lambda_n?


Joint M-Best-Diverse Labelings as a Parametric Submodular Minimization
We consider the problem of jointly inferring the M -best diverse labelings for a binary (high-order) submodular energy of a graphical model. Recently, it was shown that this problem can be solved to a global optimum, for many practically interesting diversity measures. It was noted that the labelings are, so-called, nested. This nestedness property also holds for labelings of a class of parametric submodular minimization problems, where different values of the global parameter γ give rise to different solutions. The popular example of the parametric submodular minimization is the monotonic parametric max-flow problem, which is also widely used for computing multiple labelings. As the main contribution of this work we establish a close relationship between diversity with submodular energies and the parametric submodular minimization. In particular, the joint M -best diverse labelings can be obtained by running a non-parametric submodular minimization (in the special case -max-flow) solver for M different values of γ in parallel, for certain diversity measures. Importantly, the values for γ can be computed in a closed form in advance, prior to any optimization. These theoretical results suggest two simple yet efficient algorithms for the joint M -best diverse problem, which outperform competitors in terms of runtime and quality of results. In particular, as we show in the paper, the new methods compute the exact M -best diverse labelings faster than a popular method of Batra et al., which in some sense only obtains approximate solutions.

The authors describe an improved sequential and a parallel algorithm for the case joint diverse best M solutions problem for binary submodular energy functions and nodewise diversity functions that are permutation invariant and concave.
This work builds primarily on the work of Kirillov but feels a bit incremental in some ways as the main theoretical result is very reminiscent of previous work.  The fact that it leads to a better algorthimic approach is important, I think that the comparison with previous work could be stated more clearly in the draft.  I would have liked to see an actual complexity analysis of the algorithms.  That is, the authors claim that it leads to an improvement that scales with M, but that doesn't seem to be presented theoretically.  What is the computational complexity of the algorithms involved?  Does this improve the theoretical asymptotic complexity, or is it just a constant time improvement?  In the worst case I think that they may be the samThis kind of analysis should be possible to state for the simple pairwise binary models discussed in this work.  The same goes for the parallel algorithm.    The draft is difficult to read because of a number of typos.  This can certainly be corrected with some effort.  Also, it appears that the main theoretical contribution is imprecisely stated.  This should be corrected as it makes it difficult to read/check the proofs.  Miscellaneous typos/comments:  -  line 61, "and proposes a solver to it" -> "and proposes a solver for it" -  line 66, "e.g. Hamming distance" -> "e.g., Hamming distance" -  line 104, "i.e. each variable may take only" -> "i.e., each variable may take only", this typo occurs in many places throughout the draft. -  line 135-136, "In this paragraph we briefly summarize existing efficient minimization approaches for (2)" ->  this is the end of the paragraph so it doesn't make any sense. -  line 142, "this algorithm requires to sequentially solve M" ->  "this algorithm requires sequentially solving M" -  line 161, "which plays a crucial role for obtaining the main results of this work" -> "which plays a crucial role in obtaining the main results of this work" -  line 176, I'm not sure that the definition of \Gamma here makes sense given that \Gamma should be a subset of R^{|V|}.  In fact, I think that there is a problem with the definition of the parametric problem in line 165 as "i" does not appear anywhere in the definition. -  line 204, "monotonous" -> "monotone"


Learning Parametric Sparse Models for Image Super-Resolution
Learning accurate prior knowledge of natural images is of great importance for single image super-resolution (SR). Existing SR methods either learn the prior from the low/high-resolution patch pairs or estimate the prior models from the input low-resolution (LR) image. Specifically, high-frequency details are learned in the former methods. Though effective, they are heuristic and have limitations in dealing with blurred LR images; while the latter suffers from the limitations of frequency aliasing. In this paper, we propose to combine those two lines of ideas for image super-resolution. More specifically, the parametric sparse prior of the desirable high-resolution (HR) image patches are learned from both the input low-resolution (LR) image and a training image dataset. With the learned sparse priors, the sparse codes and thus the HR image patches can be accurately recovered by solving a sparse coding problem. Experimental results show that the proposed SR method outperforms existing state-of-the-art methods in terms of both subjective and objective image qualities.

The proposed solution to single image super-resolution starts from and is inspired by [14] and [4] to design a method which employs sparse priors learned from both training images and input LR image and learned mapping functions between LR patches and the sparse codes of the preferable HR patches.
The method is novel, the performance is impressive on 3 standard datasets, and the paper reads well.  My concerns are expressed below and hopefully will be considered for improving the paper.  1) The parameters need to be not evaluated or discussed; at least some of them should be discussed if not supported by experiments.  2) In Table 1, I think that the SCSR method is the one from [2] and not [3]. why not having the results for x2, x4?  3) In Table 1, the PSNR/SSIM values for A+[7] and BSD100 are computed after saving to JPEG compressed images, while for the other methods (SRCNN, KK, NCSR) before.  In the following papers (appeared first in 2015 on arXiv) are reported better numbers for A+: [Ref1] Timofte et al, Seven ways to improve example-based single image super resolution, CVPR 2016 [Ref2] Kim et al, Accurate image super-resolution using very deep convolutional networks, CVPR 2016  4) Combining internal and external information for superior restoration performance is not new; It was shown, among others, in [Ref1] for super-resolution, and by (Mosseri et al, ICCP 2013) for denoising.  5) The experiment from 5.2, Tab. 2, is questionable. For a fair comparison it is necessary to report also the results for the example-based methods [3,7,8,10] after retraining for the conditions from the test.   6) I find necessary a discussion and comparison in run-times and/or time complexity of the methods. How much slower is the proposed method when compared with NCSR, A+ or SRCNN?


Statistical Inference for Pairwise Graphical Models Using Score Matching
Probabilistic graphical models have been widely used to model complex systems and aid scientific discoveries. As a result, there is a large body of literature focused on consistent model selection. However, scientists are often interested in understanding uncertainty associated with the estimated parameters, which current literature has not addressed thoroughly. In this paper, we propose a novel estimator for edge parameters for pairwise graphical models based on Hyvärinen scoring rule. Hyvärinen scoring rule is especially useful in cases where the normalizing constant cannot be obtained efficiently in a closed form. We prove that the estimator is p n-consistent and asymptotically Normal. This result allows us to construct confidence intervals for edge parameters, as well as, hypothesis tests. We establish our results under conditions that are typically assumed in the literature for consistent estimation. However, we do not require that the estimator consistently recovers the graph structure. In particular, we prove that the asymptotic distribution of the estimator is robust to model selection mistakes and uniformly valid for a large number of data-generating processes. We illustrate validity of our estimator through extensive simulation studies.

This paper studies the problem of estimating parameters in a pairwise graphical model and constructing confidence intervals for the parameters. As part of this an asymptotically normal estimator is constructed. The key progress made in this paper is that inference (i.e., confidence intervals) is done in a setting where computation of basic quantities (e.g. partition function) is intractable. (In contrast, for Gaussian graphical model one can efficiently compute the normalizing constant.)  Specifically, an estimator based on Hyvarinen score is given for estimation of a single edge in the pairwise undirected graphical model. The new scoring rule uses the conditional density of two variables given the rest. A first step forms a preliminary Markov blanket for a pair of variables, and the estimator then re-optimizes over parameter value, which has a sort of decoupling effect. The estimator is shown to satisfy asymptotic normality subject to regularity assumptions on the model (Corollary 2). Finally, experiments on simulated data show the estimator behaves as one would expect, and an experiment on protein signaling data agrees with results from previous papers analyzing that dataset.    The estimator is
This paper was a pleasure to read: the results address a problem of significant interest, the writing is excellent, and the overall level of scholarship is high with context and background carefully placed within the broader literature. The proofs are fairly simple, making good use of existing technology developed in the cited work, but the estimator is novel.   It would be nice to remark if possible on what does the assumption SE (line 176) really mean as far as modeling power of the model.