Legendre Decomposition for Tensors
We present a novel nonnegative tensor decomposition method, called Legendre decomposition, which factorizes an input tensor into a multiplicative combination of parameters. Thanks to the well-developed theory of information geometry, the reconstructed tensor is unique and always minimizes the KL divergence from an input tensor. We empirically show that Legendre decomposition can more accurately reconstruct tensors than other nonnegative tensor decomposition methods.

Main ideas of the submission  The manuscript presents an approximation of nonnegative multi-way tensorial data (or high-order probability mass functions) based on structured energy function form that minimizes the Kullback-Leibler divergence. Comparing against other multilinear decomposition methods of nonnegative tensors, the proposal approach operates on multiplicative parameters under convex objective function and converges to a globally optimal solution. It also shows interesting connections with graphical models such as the high-order Boltzmann machines. Two optimization algorithms are developed, based upon gradient and natural gradient, respectively. The experiment shows that under the same number of parameters, the proposed approach yields smaller RMSEs than the other two baseline non-negative tensor decomposition methods.      Although the authors frame the proposed approach under tensor decomposition and draw a connection to statistical models, I found that it is easier to interpret it from the perspective of graphical models. Basically, the main technical point seems to be the incorporation of partial ordering constraints on the inclusion of interaction terms such that the model complexity of higher-order Boltzmann machines is under control. It seems the tensor decomposition view tends to overcomplicate the idea, and I’m not quite convinced by its benefits.   Meanwhile, the experiments seem problematic and not compelling.    First, the capacity of tensor models not only depends on the number of parameters but also on the operations on them. Usually, the low-dimensional components could also be interpretable, and able to capture certain underlying structure of data (e.g., multi-linear). Moreover, we do not always want an accurate reconstruction of the observed tensor (e.g., denoising). In this manuscript, the possible interactions are restricted by a partial order that looks quite artificial. In practice, would this ordering naturally arise? What kind of characteristic of the multi-variate density do we expect the Legendre decomposition to capture?    Second, it seems the two baseline methods considered in the experiments are both based on least-square (LS) objective functions and multiplicative updates (Lee and Seung, 1999), while this manuscript adopts KL divergence. The LS objective may not be appropriate for non-negative smaller values. In addition, those algorithms could converge to a non-KKT point (Gonzalez and Zhang, 2015). Therefore, the worse performance of competitors could either be attributed to the objective function or the multiplicative updates. Without additional experimental evidence, it is hard to say whether the proposed method truly outperforms the competitors in terms of model capacity. To make a fair comparison, there exists KL divergence based nonnegative tensor decomposition methods, such as CP-APR (Chi and Kolda, 2012), an important baseline missing in this paper.   The faster convergence of the natural gradient algorithm than its gradient counterparts is not too surprising.   On page 3 line 83, what is the main reason for using the forward KL divergence KL(P, Q)? How would the zero-avoiding property affect the results in tensor reconstruction? How does it compare to the variational Bayes methods for Boltzmann machines?    The is certainly room for improvements in the clarity and self-containess of this paper. I found the notations in Section 2.1 and 2.2 particularly hard to follow. Also on page 8 line 267-268, the author claims that the “Legendre decomposition effectively uses the index order information”. I’m having trouble understanding what is exactly index order information and how does it improve the reconstruction.       Reference   Lee, Daniel D., and H. Sebastian Seung. "Learning the parts of objects by non-negative matrix factorization." Nature401.6755 (1999): 788.  Chi, Eric C., and Tamara G. Kolda. "On tensors, sparsity, and nonnegative factorizations." SIAM Journal on Matrix Analysis and Applications 33.4 (2012): 1272-1299.  Gonzalez, Edward F., and Yin Zhang. Accelerating the Lee-Seung algorithm for nonnegative matrix factorization. 2005.  [After Rebuttal] The CP-APR comparison included in the rebuttal serves as a more genuine competitor than the other two. The experimental results provide additional evidence on the margin of improvements (despite small) of the proposed approach. Accordingly, I will raise my rating to “weak accept”. The connection between tensor formulation and graphical models looks quite interesting, and now we know that the partial ordering constraint is at least useful for image data.


Boosted Sparse and Low-Rank Tensor Regression
We propose a sparse and low-rank tensor regression model to relate a univariate outcome to a feature tensor, in which each unit-rank tensor from the CP decomposition of the coefficient tensor is assumed to be sparse. This structure is both parsimonious and highly interpretable, as it implies that the outcome is related to the features through a few distinct pathways, each of which may only involve subsets of feature dimensions. We take a divide-and-conquer strategy to simplify the task into a set of sparse unit-rank tensor regression problems. To make the computation efficient and scalable, for the unit-rank tensor regression, we propose a stagewise estimation procedure to efficiently trace out its entire solution path. We show that as the step size goes to zero, the stagewise solution paths converge exactly to those of the corresponding regularized regression. The superior performance of our approach is demonstrated on various real-world and synthetic examples.

In this paper, the authors propose an algorithm for tensor regression. The problem of tensor regression with low-rank models have been approached in several recent works. The novelty in the present paper is that, besides the low-rank CP (PARAFAC) property, sparsity in the factor matrices are also imposed, thus providing interpretability to the results. In order to estimate the CP model factors with sparsity constraints, the authors propose a deflation approach which means to estimate one rank-1 term of the decomposition at a time, giving that the other terms are fixed. For the rank-1 term estimation step, the authors use a stagewise technique inspired in the algorithm for Lasso (Zhow & Yu, 2007). Finally, the experimental results are presented using simulated as well as real-world datasets. I found the paper somewhat relevant to NIPS, although its contribution can be seen as an improvement of previously proposed regression techniques. Particularly, I found the following issues in the current version of the paper: - The authors made use of the term “boosted” in the title but it is not well justified in the paper. In particular, the explanation in lines 158 – 160 is not clear. Does it mean that using a combination of a forward plus a backward step provides a boosting effect? The Authors have clarified this in their responses. - Line 98: The sentence “3) It leads to a more flexible and parsimonious model, thus makes the model generalizability better” is kind of vague and should be more precise. What does it mean to have a more flexible model? What does it mean to have a more parsimonious model? What do the authors mean by better generalizability? Are the authors referring to the fact that the model is less prone to overfitting? If so, this must be clearly stated and demonstrated, at least, with experiments. The Authors have clarified this in their responses. - Line 114-115: The sentence “Here for improving the convexity of the problem and its numerical stability …” is not clear enough. These concepts must be discussed more precisely. Is the model not convex without using the elastic-net? What does it mean to be numerically unstable and why the elastic-net model helps to avoid it?The Authors have clarified this in their responses and will provide further clarification in the final version. - Line 116 – 117: The authors say that “we fix alpha>0 as a small constant”. What does it mean a "small value"? What is the range of this parameter and how should we judge if a value is small or large? The Authors have clarified this in their responses and will make more specific recommendations on the choice of alpha in the final version of the paper. - Algorithm 1: This algorithm is called “Fast” by the authors. The authors should provide some complexity analysis to justify the term “fast” in the algorithm name. - I found the experimental results based on real-world signals somewhat confusing. The authors state that the goal in this case is to predict the Montreal Cognitive Assessment scores. The authors evaluate the results in terms of root mean squares error (RMSE), sparsity and computation time. This error (RMSE) is not defined in the paper. Is it computed by calculating the error between the ground truth score and the one predicted by the model? The Authors have clarified this in their responses and will provide a definition in the final version of the paper. - Line 66, the index in the lower end in the last sum must be changed i_1 -> i_N - Line 35 in supplemental information: “Some Experimental Results” -> “Additional Experimental Results”


Fast and Effective Robustness Certification
We present a new method and system, called DeepZ, for certifying neural network robustness based on abstract interpretation. Compared to state-of-the-art automated verifiers for neural networks, DeepZ: (i) handles ReLU, Tanh and Sigmoid activation functions, (ii) supports feedforward, convolutional, and residual architectures, (iii) is significantly more scalable and precise, and (iv) and is sound with respect to floating point arithmetic. These benefits are due to carefully designed approximations tailored to the setting of neural networks. As an example, DeepZ achieves a verification accuracy of 97% on a large network with 88, 500 hidden units under L ∞ attack with = 0.1 with an average runtime of 133 seconds.

The paper develops a new approach for the problem of verifying input-output properties of neural network, an emerging area that has received significant interest and attention in the ML community recently. The approach builds on previous work that uses abstract interpretation (sound abstractions of the transformations at each layer of a neural network) that can be used to reason about worst-case behavior of the network with respect to desired input-output properties (for example, that the network should continue to produce the same label despite adversarially chosen small perturbations of the input). If the property is satisfied by the sound abstraction, the property will be satisfied by the underlying network as well.  Strengths of the paper: Well written - the central idea of zonotopes is clearly explained, and the property of area minimizing parallelogram abstractions of various neural network transformations (relu, sigmoid, tanh) is explained geometrically and derived. The paper computes certificates that are sound even accounting for floating point arithmetic - a property that many competing approaches (like those of Kolter et al, Raghunathan et al, Dvijotham et al) do not satisfy. The paper applies to a broad class of transformations commonly found in feedforward neural networks, unlike previous approaches from that use tools from the formal methods community (like Katz et al, Elhers et al etc.)  Weaknesses of the paper: Justifying the use of zonotopes: The authors claim that zonotopes provide a sound abstraction that offers the right tradeoff between precision and computation, but do not provide much justification for why this is the case. For example, it was not clear to me why enforcing a parallelogram shape in the abstraction is necessary - for example, the ReLU is abstract better by the triangle used in the PLANET (Elhers et al) paper. I understand that this helps preserve the affine interval expression structure - however, it is not clear why this interval expression structure is crucial for the approach to work or scale. Comparison to prior work: When specialized to the case of piecewise linear activation functions, the approach seems to be equivalent (or very close to) that of https://www.icml.cc/Conferences/2018/Schedule?showEvent=2976 . This paper also proves that the approach is equivalent to a that form (Kolter and Wang, 2017) https://arxiv.org/abs/1711.00851 . While the authors extend the approach to nonlinear activation functions and ensure that the computations are rigorous under finite precision arithmetic, a careful discussion and comparison to this previous work is certainly warranted. Without it, the novelty of the approach in the paper comes under question. Experimental evaluation: I have several concerns with the experimental evaluation section of the paper: It is well-understood from the works of (Kolter and Wang, 2017) and (Raghunathan et al, 2017) that networks that are trained to be verifiable are significantly easier to verify than networks trained arbitrarily. The Given how close the approach is to that from https://www.icml.cc/Conferences/2018/Schedule?showEvent=2976, a comparison to this approach on ReLU networks is certainly warranted.  It would be good to compare against other approaches that use formal methods and are sound to floating point arithmetic (like ReLUplex from Katz et al, PLANET from Elhers et al). In the comparison against [Dvijotham et al, 2018], the authors only implement the version of the algorithm with optimization based bound propagation. The approach also works with interval arithmetic based bound propagation and is likely much faster in this variant -  Final results: Published numbers for verified robustness against adversarial attacks on MNIST and CIFAR (from https://www.icml.cc/Conferences/2018/Schedule?showEvent=2976 and Kolter and Wang) are significantly better than those from the paper - for example, these papers establish verified error rates of around 5% against perturbations of size .1 or larger while those in the paper are significiantly weaker. While this is likely due to the training procedure used, it would have been good for the authors to apply their methods to the networks trained by (Kolter and Wang), (Mardry et al) which are publicly available to see if they indeed produce state of the art numbers on verified accuracy/robustness. Another possibility is the need to account soundly for finite precision arithmetic, but the reviewer feels that this needs to be justified through numerical studies (or by implementing a version of the authors’ approach that does not guarantee soundness against finite precision, only against precise real arithmetic). Other specifications on the input: The examples considered in the paper assume that the constraints on the input can be given by lower and upper bounds. It is clear that the zonotopes can work with other types of input constraints - however, it is not clear how gracefully the performance (in terms of precision loss) would degrade (for example, with an l2 constraint on the inputs instead of linf).  As a side note, at least in the ML community, “zonotopes” is not a commonly used term. Given the central idea of the paper can be captured in the idea of bounding nonlinear tranforms with affine interval expressions, perhaps a simpler name is more appropriate for the ML community?  Overall evaluation:  Clarity: The paper is clearly written.  Quality: The theoretical results are accurate and well presented. However, the quality of the experimental section can be significantly improved.  Novelty: The paper adds novelty in addressing finite precision arithmetic and general activation functions relative to previous verification approaches. However, the lack of a clear comparison to https://www.icml.cc/Conferences/2018/Schedule?showEvent=2976 casts a shadow on how novel the core idea of zonotopes, particularly in the context of neural network verification.  Significance: The paper is an interesting albeit marginal addition to the state of the art, in that the paper offers a scalable/general approach to robustness certification that is sound against floating point arithmetic.


Domain-Invariant Projection Learning for Zero-Shot Recognition
Zero-shot learning (ZSL) aims to recognize unseen object classes without any training samples, which can be regarded as a form of transfer learning from seen classes to unseen ones. This is made possible by learning a projection between a feature space and a semantic space (e.g. attribute space). Key to ZSL is thus to learn a projection function that is robust against the often large domain gap between the seen and unseen classes. In this paper, we propose a novel ZSL model termed domain-invariant projection learning (DIPL). Our model has two novel components: (1) A domain-invariant feature self-reconstruction task is introduced to the seen/unseen class data, resulting in a simple linear formulation that casts ZSL into a min-min optimization problem. Solving the problem is non-trivial, and a novel iterative algorithm is formulated as the solver, with rigorous theoretic algorithm analysis provided. (2) To further align the two domains via the learned projection, shared semantic structure among seen and unseen classes is explored via forming superclasses in the semantic space. Extensive experiments show that our model outperforms the state-of-the-art alternatives by significant margins.

The authors present an algorithm for zero-shot learning based on learning a linear projection between the features of a pre-trained convnet and a semantic space in which to do nearest neighbors classification.  Overall, I found the exposition of the paper very confusing, and I am still struggling to understand the exact set-up in the experiments and theorems after several re-readings. I'm not 100% certain from the paper what the authors are actually doing and what they have available as training data. In particular, the authors denote the test images by D_u, which they are sure to point out are unlabeled. They then define l_i^{(u)} to be the label of test point x_i^{(u)} - how can the label be included in a set of test images that is unlabeled? What information from D_u is provided when learning the projection matrix W, and what is held out until test time? What is the algorithm used to map the projection of a feature to a class? I *think* what is happening is that the semantic features y_j for each unseen class j are provided at training time, but not the label for which class each test image belongs to, but this is not well-explained. It is also not clear to me how these vectors y_j are chosen in the experiments. I *think* that the datasets they work with have pre-defined attribute vectors for every class - providing a concrete example of such an attribute vector would help explain the source of these y_j terms.  In the loss function in Eq 1, I am not clear on why W and W^T are used for the forward/backward projection from visual features to semantic features, rather than W and its Moore-Penrose pseudoinverse. Is the learned W approximately orthogonal?  Line 152: "contentional" is not a word. I am not sure what the authors are trying to say here.  Section 3.3: This section is especially confusing. I suppose the confusion stems from the fact that f_i^{(t)} is a vector, and you are doing an optimization over elements of the vector, that is, just taking the minimum element. Not to mention some of the math seems wrong - you define the gradient of the minimum of the element of the vector to be 1/number of elements with that value. The minimum is a piecewise-linear function, and if multiple elements of the vector have the same (minimum) value, the minimum does not have a well-defined gradient at that point and instead has a *sub*gradient, which is a set of directions that fall below the function.  Section 3.4: This section is far too dense, to the point of being almost unreadable. I would recommend reformatting this and, if space is not available, just state the main result of the proof and move the body to the supplementary materials. Also I am not sure that Proposition 1 is novel enough to warrant a proof here - presumably the theory of convergence for Sylvester equations is well-studied and the appropriate theorem could simply be cited here.  I appreciate the inclusion of ablation studies. It should be more common in NIPS submissions.  Overall, while I'm somewhat skeptical of how realistic the transductive ZSL setting is (since information about unseen classes is not likely to be available at training time) and I am not certain that the derivation of the updates is correct, the empirical performance does seem very good, and the results are a significant improvement on the aPY dataset.


See and Think: Disentangling Semantic Scene Completion
Semantic scene completion predicts volumetric occupancy and object category of a 3D scene, which helps intelligent agents to understand and interact with the surroundings. In this work, we propose a disentangled framework, sequentially carrying out 2D semantic segmentation, 2D-3D reprojection and 3D semantic scene completion. This three-stage framework has three advantages: (1) explicit semantic segmentation significantly boosts performance; (2) flexible fusion ways of sensor data bring good extensibility; (3) progress in any subtask will promote the holistic performance. Experimental results show that regardless of inputing a single depth or RGB-D, our framework can generate high-quality semantic scene completion, and outperforms state-of-the-art approaches on both synthetic and real datasets.

I want to like this paper. I think its well motivated and using a staged approach to this problem seems quite sensible. Additionally, they show quite good quantitative results on a challenging dataset. That being said, the model itself is not overly unique (not the worst thing) and the quantitative / qualitative results don't really cement the main purpose of the paper. Because the authors changed the network itself AND the approach, its not clear whether any superiority is due to multiple inputs (they use RGBD while other papers use RGB or D), a different network (perhaps this paper has a truly advantageous network structure) or the main idea: that predicting 2D first is better, is what really leads to good performance.  === Main Ideas === The main idea of the work is to produce 3D semantic segmentation voxel grids, but do so via a staged approach: (1) 2D semantic segmentations are first estimated. (2) these are projected into 3D space (3) the unseen (occluded) voxels are inferred.  The authors attempt three variants of this approach:  1. RGB or Depth --> Incomplete 3D --> Complete 3D 2. RGB and Depth pathways --> Incomplete 3D --> Complete 3D 3. RGB and Depth pathways --> Incomplete 3D from RGB, Incomplete 3D from depth --> Complete 3D   === Quality === I like the idea overall: by breaking up the network into components, its much easier to debug and analyze than in pure end-to-end models. That being said, I really wish the authors had more clearly validated the efficacy of each model component. NYUDV2 is, in reality, a 2D dataset with depth. So if SATNet is good at predicting filled in voxel grids, it might do one of two things well: (1) estimate 2D semantic segmentations well or (2) somehow correct mistakes made in (1) but in 3D space. Filling in occluded pixels for this problem, while providing nice qualitative results, cannot help performance, because NYUDV2 has no labels for occluded pixels.   Indeed, the clearest way to demonstrate that the proposed 3-stage approach is a good one would be to (a) show that better 2D segmentations result in better 3D results (surely this is true) or (b) that their 3D network is very good at fixing mistakes. If their model happens to do (a), then they need to clearly demonstrate this superiority in terms of 2D results and explain how they were able to obtain superior 2D results. If their model happens to do (b) this would make the paper ever more compelling.  Overall, I'm left a bit wanting in terms of the quality. I want to like the paper and I think the authors approach is a potentially very good one, but I feel that much can be done to more clearly cement the main motivation of the work (staged approach is better than end-to-end) and demonstrate why indeed this would be the case.  === Clarity ===  Overall, the paper is easy to read. Minor grammatical issues throughout but these do not impact the quality of the paper: - ln 77: "with the assist" => "with the assistance" - ln 81: "the semantic for semantic completion" => "the voxels for semantic completion" (I assume?) - ln 114: "it takes few time" => "it takes little time"  There are a number of literary choices made by the authors that are extraneous and actually make the paper less clear overall: - ln 91: This is the first time I've seen the acronym DUC. Aside from paper [38], this is quite uncommon and actually makes the paper less clear. I would recommend just referring to the Decoder Network.  - Naming / Branding the entire approach SATNet is fine. Giving names to the individual sub-components (SeeNet, ThinkNet, SeeNetFuse, ThinkNetFuse) seems extraneous and very confusing. Its far clearer to simply describe the components of the network, Figure 5 does a very good job of showing this off without the need to sub-brand parts of the network.  - ln 85: "Images have higher resolution than volumes.". This is plainly untrue. Either one can be constructed to have an arbitrarily high resolution. It happens that in practice, researchers tend to use voxel grids that have a smaller spatial resolution, mostly due to computational constraints. This might be what the authors meant?  Minor Nit: the following reference is missing: A contour completion model for augmenting surface reconstructions which does volume / voxel reconstruction, partially semantically.   === Originality === The paper is mildly original. Many works solve problems in a staged approach. To my knowledge, this is the first to do so on this particular task.  === Significance === The problem of 3D semantic scene completion is an increasingly popular one. The authors demonstrate some very promising quantitative results which would be generally interesting to the community.


Chain of Reasoning for Visual Question Answering
Reasoning plays an essential role in Visual Question Answering (VQA). Multi-step and dynamic reasoning is often necessary for answering complex questions. For example, a question "What is placed next to the bus on the right of the picture?" talks about a compound object "bus on the right," which is generated by the relation <bus, on the right of, picture>. Furthermore, a new relation including this compound object <sign, next to, bus on the right> is then required to infer the answer. However, previous methods support either one-step or static reasoning, without updating relations or generating compound objects. This paper proposes a novel reasoning model for addressing these problems. A chain of reasoning (CoR) is constructed for supporting multi-step and dynamic reasoning on changed relations and objects. In detail, iteratively, the relational reasoning operations form new relations between objects, and the object refining operations generate new compound objects from relations. We achieve new state-of-the-art results on four publicly available datasets. The visualization of the chain of reasoning illustrates the progress that the CoR generates new compound objects that lead to the answer of the question step by step.

*Summary* Questions in Visual Question Answering often require to reason about referring expressions that relate some object to other objects in the image, e.g. “The bus on the right of the person that is wearing a hat”. This paper argues that previous architectures for VQA are not explicitly biased to capture such chains of reasoning and propose a novel architecture that interleaves a relational reasoning step (to form novel relationships between objects) with a refinement operator (to prevent exponential blow-up). The model improves the state-of-the-art on four VQA datasets:  VQA v1 and v2, Coco QA and TDIUC. In the ablation studies, the authors show that the chain of reasoning structure outperforms the stack-based and parallel-based attention structure.    *Originality* The observation that referring expressions are compositional and that neural networks for VQA should incorporate compositional priors is not novel (see e.g. compositional attention network [Hudson and Manning, ICLR2018]). Nevertheless, it is interesting that this paper applies a relational prior on top of extracted *object* features, as opposed to prior work that has proposed to apply it on top of convnet features [Relation Networks, Santoro et al]. Also, the relational reasoning module is different from prior work, and these details seem to matter; The ablation study highlights the benefit of the proposed module over a vanilla relation network module.  *Clarity* The paper is well-structured but could benefit from an additional pass of proofreading. The model description is mathematically precise, and I believe one could reproduce the experimental results from the paper description.   *Quality* Experiments are thoroughly executed, improving the state-of-the-art on four VQA datasets. Also, the ablation study compares the relative improvements of several hyperparameters of the proposed module, as well as relative improvements over related architectures.  *Significance* The proposed model is strong performant, and could be widely applied to a range of language-vision tasks.   Some of my remaining concerns: - Although evaluation is thorough, I believe some other datasets would be better suited to evaluate the proposed model. Instead of Coco QA, I would have suggested evaluating on CLEVR [Johnson et al, CVPR17] or the oracle task of GuessWhat [de Vries, CVPR17].  - In the ablation studies, the authors mention that the chain of reasoning structure outperforms the stacked attention baseline and parallel attention baseline. Although I remember the stacked attention network architecture by heart, I did not figure out what the parallel architecture would be, so maybe you can add a few lines to that paper to make it more self-contained?  Post-rebuttal Feedback I'm keeping my scores.


Monte-Carlo Tree Search for Constrained POMDPs
Monte-Carlo Tree Search (MCTS) has been successfully applied to very large POMDPs, a standard model for stochastic sequential decision-making problems. However, many real-world problems inherently have multiple goals, where multiobjective formulations are more natural. The constrained POMDP (CPOMDP) is such a model that maximizes the reward while constraining the cost, extending the standard POMDP model. To date, solution methods for CPOMDPs assume an explicit model of the environment, and thus are hardly applicable to large-scale realworld problems. In this paper, we present CC-POMCP (Cost-Constrained POMCP), an online MCTS algorithm for large CPOMDPs that leverages the optimization of LP-induced parameters and only requires a black-box simulator of the environment.
In the experiments, we demonstrate that CC-POMCP converges to the optimal stochastic action selection in CPOMDP and pushes the state-of-the-art by being able to scale to very large problems.
and in the black-box simulation of the history. In contrast, solution methods for CPOMDPs, e.g. dynamic programming [11], linear programming [18], and online uniform-cost search [23], are not yet advanced up to this level, partly due to requiring an explicit model of the environment. This prevents CPOMDP from being a practical approach for modeling real-world applications.
In this paper, we present an MCTS algorithm for CPOMDPs, which precisely addresses the scalability. To the best of our knowledge, extending MCTS to CPOMDPs (or even CMDPs) has remained unexplored since it is not straightforward to handle the constrained optimization in tree search. This challenge is compounded by the fact that optimal policies can be stochastic. 1 In order to develop MCTS for CPOMDPs, we first show that solving CPOMDPs is essentially equivalent to jointly solving an unconstrained POMDP while optimizing its LP-induced parameters that control the tradeoff between the reward and the costs. From this result, we present our algorithm, Cost-Constrained POMCP (CC-POMCP), for solving large CPOMDPs that combine traditional MCTS with LP-induced parameter optimization. In the experiments section, we demonstrate that CC-POMCP converges to the optimal stochastic action selection using a synthetic domain and that it is able to handle very large problems including constrained variants of Rocksample (15,15) and Atari 2600 arcade game, pushing the state-of-the-art scalability in CPOMDP solvers.

This paper addresses a potentially important problem by giving an algorithm that can solve large constrained  POMDPs with online methods. A constrained POMDP, which augments a traditional POMDP with multi-attribute cost constraints, is an important extension that can help model a wider range of real-world phenomena than a POMDP can.  Having such an algorithm for solving large CPOMDPs is a very valuable contribution. The authors provide, in this paper, a derivation of an unconstrained objective to be solved (resulting from taking the dual of the CPOMDP's linear program), backed by theoretical justification, and an adaptation of the online search algorithm, POMCP, that incorporates cost constraints by approximately optimizing the objective.  The paper is extremely well-written, free of typos, and clear in its presentation. I especially liked the separation between the derivation of the objective being minimized (Section 3: eq. 3 and 4, leading up to the three-step process outlined at the end) and the approximation algorithm CC-POMCP (Section 4). I noticed there was no related work section, except for the paragraph starting "Although the CPOMDP..." in the Introduction, but maybe this is okay due to the minimal previous work on this subject.  (I don't know of any obviously missing references.)  I am assuming that the derivation, starting from section 3, is your novel work (the theorem 1 procedure is very cool).  If that is not true, then please indicate more clearly what aspects of the paper are novel.  I was somewhat surprised by the result that the optimal policy for a CPOMDP is generally stochastic.  It would be great if you could give a small example illustrating that fact, just for general edification of the reader.  Several experiments are conducted to show the effectiveness of the proposed algorithm.  They are well done, but ultimately somewhat disappointing. - The Rocksample domains, even though they are widely used to benchmark POMDP solvers, are actually really surprisingly easy to solve.  The reason is that the uncertainty about the individual rocks is basically completely uncoupled, and so it's really more of a large collection of small POMDPs with something like a TSP to connect them.  Further, the small POMDPs are relatively easy, given the actual observation probabilities, and there's very little branching in the optimal policy. - Given the nice ability of CPOMDPs to handle MA problems, it would have been good to see a domain that illustrates that.  I could imagine something like oil exploration where:  information about one test well actually give information about others (unlike the rocks) and where there are budgets for number of wells, distance traveled, total energy expended, etc. - The Pong domain seems very contrived, and seems to me to do a disservice to a very beautiful algorithm and paper.  When describing your main algorithm CC-POMDP, I think it would be very helpful to readers if you were to identify the differences from the standard POMCP algorithm. For example, perhaps you could use a different color to indicate those differences.   --------------- After reading the author response and other reviews, my opinion above still holds.


A Retrieve-and-Edit Framework for Predicting Structured Outputs
For the task of generating complex outputs such as source code, editing existing outputs can be easier than generating complex outputs from scratch. With this motivation, we propose an approach that first retrieves a training example based on the input (e.g., natural language description) and then edits it to the desired output (e.g., code). Our contribution is a computationally efficient method for learning a retrieval model that embeds the input in a task-dependent way without relying on a hand-crafted metric or incurring the expense of jointly training the retriever with the editor. Our retrieve-and-edit framework can be applied on top of any base model. We show that on a new autocomplete task for GitHub Python code and the Hearthstone cards benchmark, retrieve-and-edit significantly boosts the performance of a vanilla sequence-to-sequence model on both tasks.

This paper addresses the task of highly structured output prediction in the contexts of, here, source code generation.  The authors propose a two-step process for this, consisting of a retrieval model that retrieves the closest seen training instance to the test instance, and an edit model, that edits the retrieved training instance.  Quality: - The authors approach the relatively recently proposed task of generating Python source code. Unlike others, they work in a setting with less supervision, where they assume no access to the Abstract Syntax Tree.  - The key innovation here is the task-specific retrieval model, trained using a variational autoencoder that is then further fine-tuned such that the output distribution of the learned encoder resembles the oracle encoder. I like this idea and the proposed model a lot and can see how it would be useful beyond the immediate scope of the paper. - It is clear how the proposed method differs from previous work, and related work is adequately cited  - What wasn't clear to was why the retriever only retrieves one training example to be edited per testing instance. This seems to rely very heavily on the retriever's performance and also on the training set containing one perfectly related training instance. Wouldn't it be better to retrieve several and then try and then have the edit model merge them as well? - The authors could have shown more ablation studies for their results. They make several assumptions in their approach for which detailed results are not shown, e.g. the encoding prior (Section 3.1.1) - They assume that there is an oracle editor, but don't describe how the oracle decoder is constructed from the training data. Also, they didn't discuss the dependence of their approach on the oracle editor or how could train the model in a scenario without access to an oracle editor.   Clarity: - The submission is overall well-written and well-motivated. The figures provided nicely illustrate the approach. - In several places in the paper, abbreviations are used without being introduced (e.g. "AST" in abstract) and variables are introduced after formulae they are used in, particularly in Section 3. Small rewrites could make the paper more readable in those places.  Originality: - This work is a novel combination of existing techniques, combining variational autoencoders, joint training, and sequence models. Previous work using retrieval as part of their approach saw this mainly as a black-box component and did not try to jointly train the retrieval component with the main model.  Significance: - As already stated above, I believe the general idea as well as the model proposed here will be used beyond the immediate scope of this work (Python code generation).  =================  Thanks to the authors for clarifying my questions. My remaining concerns are addressed and I would be happy to see this paper appear at NIPS. In light of this, I increased my score by 1.


Scalable Laplacian K-modes
We advocate Laplacian K-modes for joint clustering and density mode finding, and propose a concave-convex relaxation of the problem, which yields a parallel algorithm that scales up to large datasets and high dimensions. We optimize a tight bound (auxiliary function) of our relaxation, which, at each iteration, amounts to computing an independent update for each cluster-assignment variable, with guaranteed convergence. Therefore, our bound optimizer can be trivially distributed for large-scale data sets. Furthermore, we show that the density modes can be obtained as byproducts of the assignment variables via simple maximum-value operations whose additional computational cost is linear in the number of data points. Our formulation does not need storing a full affinity matrix and computing its eigenvalue decomposition, neither does it perform expensive projection steps and Lagrangian-dual inner iterates for the simplex constraints of each point. Furthermore, unlike mean-shift, our density-mode estimation does not require inner-loop gradient-ascent iterates. It has a complexity independent of featurespace dimension, yields modes that are valid data points in the input set and is applicable to discrete domains as well as arbitrary kernels. We report comprehensive experiments over various data sets, which show that our algorithm yields very competitive performances in term of optimization quality (i.e., the value of the discrete-variable objective at convergence) and clustering accuracy.

The paper proposed a noval and scalable approach to solve the Laplacian K-modes problem. The primary strategy is to adopt another way to relax the original discrete problem and further minimize a upper bound of the objective. The method is guaranteed to converge and evaluated to show its superiority over some benchmark clustering algorithms in both clustering performance and algorithmic efficiency. The statements are clear and content is easy to follow. The technical part is also correct.    My main criticism is about the significance.  -First, I am not sure if Laplacian K-modes energy is a widely used metric for clustering. Laplacian K-modes energy follows a form that simultaneously maximizes weights within each cluster (mode part) and minimizes weights across clusters (Lap part). There are several objectives falling in this category like correlation clustering. However, in experiments, the authors only compare LK objective with some very preliminary objectives NCUT, k-means, k-modes that are in other categories of clustering algorithms (NCUT: balanced min-cut; k-means, k-modes: max weights within clusters). - Second, although the proposed approach has good scalability, the derivation is kind of ad-hoc and all tricks are for scalability. In the first-step relaxation, an entropy term is subjectively added to handle the constrains. With this term, the objective can hardly be a tight relaxation of Laplacian K-modes energy. The proposed approach only holds convergence guarantee while do not has performance guarantee and the trick to obtain such convergence is not difficult to think of. Also, the efficient method to estimate mode is also ad-hoc. I think the proposed method may be useful in some practical scenarios but it is not significant enough for a strong acceptance of NIPS. Hence, I give a borderline overall score.


Nearly tight sample complexity bounds for learning mixtures of Gaussians via sample compression schemes *
We prove that Θ(kd 2 /ε 2 ) samples are necessary and sufficient for learning a mixture of k Gaussians in R d , up to error ε in total variation distance. This improves both the known upper bounds and lower bounds for this problem. For mixtures of axis-aligned Gaussians, we show that O(kd/ε 2 ) samples suffice, matching a known lower bound. The upper bound is based on a novel technique for distribution learning based on a notion of sample compression. Any class of distributions that allows such a sample compression scheme can also be learned with few samples. Moreover, if a class of distributions has such a compression scheme, then so do the classes of products and mixtures of those distributions. The core of our main result is showing that the class of Gaussians in R d has a small-sized sample compression.

This paper is well written and issues a classical and well known studied problem from a new point of view.  The compression based analysis which is indeed deep, although less popular, is gaining more attention on the last couple of years. This paper join this line of work by extending it, proving some interesting properties if sample-compression schemes (such as the them being close under mixture or product) and go on and demonstrates the power of the obtained results by proving state of the art sample complexity upper bound for mixture of Gaussians. I think that the NIPS community will indeed benefit from this paper.  Remarks:  - there exists a line of work regarding the generalization guarantees of compression based learners. Most of them to the classification case but some for more general setting. How does your results differ from those? See for example      On statistical learning via the lens of compression,    PAC-Bayesian Compression Bounds on the Prediction Error of Learning          Algorithms for Classification,    Nearly optimal classification for semimetrics,    Sample Compression for Real-Valued Learners,    Adaptive Learning with Robust Generalization Guarantees, - Although the “robust compression implies  agnostic learning” is indeed of interest it isn’t in use on the specific case of mixture of Gaussians, I think that it should be stated more clearly as I got a bit of confused, especially as the proof is given for that case and not for the (more simple) realizable case - On line 317 you use a notion of a d dimensional ball. I know that this is standard notion but still it may be of help to mention you intention to make it more clear what each parameter means (so for example when you multiply the ball, you mean just scaling it?


Efficient High Dimensional Bayesian Optimization with Additivity and Quadrature Fourier Features
We develop an efficient and provably no-regret Bayesian optimization (BO) algorithm for optimization of black-box functions in high dimensions. We assume a generalized additive model with possibly overlapping variable groups. When the groups do not overlap, we are able to provide the first provably no-regret polynomial time (in the number of evaluations of the acquisition function) algorithm for solving high dimensional BO. To make the optimization efficient and feasible, we introduce a novel deterministic Fourier Features approximation based on numerical integration with detailed analysis for the squared exponential kernel. The error of this approximation decreases exponentially with the number of features, and allows for a precise approximation of both posterior mean and variance. In addition, the kernel matrix inversion improves in its complexity from cubic to essentially linear in the number of data points measured in basic arithmetic operations.

In this paper, the authors introduce an improvement to the standard random fourier features (RFF) scheme for kernel approximation. Rather than rely on Monte Carlo estimates of the integral involved in Bochner's theorem, the authors derive features from a Gauss-Hermite quadrature estimate of the integral. The result is a kernel estimation scheme that convergences uniformly over the domain quickly. The authors demonstrate theoretically that the improved uniform convergence results in theoretical guarantees on the convergence of both the posterior moments of the Gaussian process and the cumulative regret of Bayesian optimization.  Overall, I quite like the paper. It tackles an issue that really does come up in practice: I have often found random Fourier features to produce unsatisfactory sample paths from the posterior Gaussian process. A method that solves this issue both in practice and provides theoretical convergence guarantees is quite nice. In my opinion, this is good work. I do have a few questions and comments for the authors that I detail below.  Right now, my biggest complaint is that the paper largely ignores the existence of approximate GP methods that aren't based on finite basis expansions. Inducing point methods (e.g. Titsias et al., 2009) can lead to accelerated sampling procedures. When performing stochastic variational inference (Hensman et al., 2013, Wilson et al., 2016), the time complexity required to perform sampling can be as low as O(tm^2) depending on the kernel approximation used. Pleiss et al., 2018 introduces a procedure that achieves O(t) time approximate sampling and applies it to the additive BayesOpt setting. As far as I am aware, theory does not exist demonstrating a cumulative regret bound for Bayesian optimization using sparse GP methods, but theory does exist demonstrating convergence of the posterior moments.   I assume QFF is compatible with other acquisition functions that can be computed using sample paths from the posterior Gaussian process (e.g., max value entropy search (Wang et al., 2017), predictive entropy search (Hernández-Lobato et al., 2014), knowledge gradient (Scott et al., 2009), ...). Have you run experiments with these? The max value entropy search paper in particular discusses another (albeit fairly coarse approximate) approach to sampling directly from the maximum value distribution, p(y*|D).  Finally, can the authors comment a bit on their focus on the high dimensional Bayesian optimization setting? In my mind, the primary contribution of the paper is the provably good kernel approximation that solves a number of issues with the RFF approach. Is this story largely due to a decreased applicability of QFF to high dimensional sampling? Is the regret bound in Theorem 3 substantively different as a result of using QFF? It appears to have the same flavor as has been demonstrated before for additive Bayesian optimization, where cumulative regret depends linearly on the number of additive components and exponentially on the size of the largest additive component. If QFF is limited to usage with additive structure GPs, it's worth noting that this setting also appears in deep kernel learning (DKL, Wilson et al., 2016) where it is standard practice to assume additive structure at the end of the neural network. QFF could therefore be used as a drop-in replacement for kernel approximation schemes in DKL.


Byzantine Stochastic Gradient Descent
This paper studies the problem of distributed stochastic optimization in an adversarial setting where, out of m machines which allegedly compute stochastic gradients every iteration, an ↵-fraction are Byzantine, and may behave adversarially. Our main result is a variant of stochastic gradient descent (SGD) which finds "-approximate minimizers of convex functions in
" 2 iterations. In contrast, traditional mini-batch SGD needs T = O 1 " 2 m iterations, but cannot tolerate Byzantine failures. Further, we provide a lower bound showing that, up to logarithmic factors, our algorithm is information-theoretically optimal both in terms of sample complexity and time complexity.

The paper studies stochastic convex optimization in a distributed master/workers framework, where on each round each machine out of m produces a stochastic gradient and sends it to the master, which aggregates these into a mini-batch. In this paper the authors allow a fraction of alpha of the machines to be Byzantine, i.e., they do not need to report valid stochastic gradients but may produce arbitrary vectors, even in an adversarial manner. The goal is to aggregate the reports of the machines and to converge to an optimal solution of the convex objective despite the malicious Byzantine machines.  The authors present a novel variant of minibatch-SGD which tackles the difficulty the dealing with Byzantine machines. They prove upper-bounds on the convergence and nearly optimal matching lower-bounds on any algorithm working in such framework, and in this sense the results are quite satisfactory. Importantly, the convergence is roughly the same as mini-batch SGD but includes an additional natural term that accounts for the Byzantine machines.   A somewhat similar result appeared in ICML18 [31]. [31] considers a very related problem, yet different, in which each machine is not represented by a stochastic first-order oracle, but is given a batch of m samples for the population distribution. The authors in [31] give a master/workers algorithm not based on SGD but on deterministic GD, where on each round, each machine reports the gradient w.r.t. its entire batch.  Both the current work and [31] can be compared w.r.t to sample complexity, i.e., number of samples each machine uses to guarantee a certain approximation error. While the two works use somewhat different assumptions, still the current work improves a key term in the convergence rate in terms of the dimension of the problem (linear dependence on d in [31] vs. no dependence in current paper). I find this improvement to be of great importance, especially for large-scale problems. Moreover, the algorithmic techniques used in this current paper seems to be quite different than those in [31], e.g., current paper tries to track the set of non-Byzntine machines throughout the run, while no such argument is present in [31]. Hence, also in terms of techniques the current paper seems sufficiently novel.


Alternating Optimization of Decision Trees, with Application to Learning Sparse Oblique Trees
Learning a decision tree from data is a difficult optimization problem. The most widespread algorithm in practice, dating to the 1980s, is based on a greedy growth of the tree structure by recursively splitting nodes, and possibly pruning back the final tree. The parameters (decision function) of an internal node are approximately estimated by minimizing an impurity measure. We give an algorithm that, given an input tree (its structure and the parameter values at its nodes), produces a new tree with the same or smaller structure but new parameter values that provably lower or leave unchanged the misclassification error. This can be applied to both axis-aligned and oblique trees and our experiments show it consistently outperforms various other algorithms while being highly scalable to large datasets and trees. Further, the same algorithm can handle a sparsity penalty, so it can learn sparse oblique trees, having a structure that is a subset of the original tree and few nonzero parameters. This combines the best of axis-aligned and oblique trees: flexibility to model correlated data, low generalization error, fast inference and interpretable nodes that involve only a few features in their decision.

I am satisfied with the authors' response, the new experiments make a good addition and improve the paper. Thus I maintain my score and reommend accepting this work. ----- This paper presents an improvement scheme for (binary) decision trees, which takes a given tree (e.g., output of the CART algorithm) and iteratively refines it using an alternating misclassification error minimization scheme that optimizes over subsets of the tree nodes while keeping the rest fixed. The proposed method can handle both axis-aligned trees (single-feature decision nodes) as well as oblique trees (with decision hyperplanes), and in the latter case, incorporate sparsity-promoting penalties in order to obtain more interpretable oblique trees where each decision node involves a few features only. The TAO (tree alternating optimization) algorithm exploits natural separability of classification paths to define its subproblems, and although no substantial theoretical guarantees are provided for successful improvement, numerical experiments validate the approach and demonstrate that the overall classification accuracy can be improved significantly by running TAO on given trees. Based on my detailed reading (see comments below), I recommend accepting this submission.  The main strength of the paper is the relatively versatile algorithmic scheme to refine decision trees that at least works well in practice, although unfortunately no strong theoretical results have been proven yet. The separability of misclassification errors over tree nodes is not a deep result, but is used cleverly to define error-minimization subproblems involving only parameters of disjoint subsets of the tree nodes that can be (approximately) solved efficiently when the remaining parameters are kept fixed, thus enabling the use of alternating optimization to improve the overall tree. All in all, the paper is well-written and the problems under consideration are well-motivated.  A weak spot of the present work is the lack of theoretical understanding of the proposed method; however, as the tackled problems are NP-hard and inherently discrete (w.r.t. the tree structure/design itself), it is difficult to analyze something like approximation properties of alternating minimization schemes that are more commonly employed in continuous problems, and in light of the convincing numerical results, I do not think this is a big problem (but it should certainly be on the agenda for future research). Moving on, I would like to see some aspects clarified:  1. At some points (particularly, in the abstract), it sounds like TAO is a stand-alone algorithm to learn decision trees that „outperforms various other algorithms“ – this is misleading, as TAO requires a tree as input and cannot start from scratch by itself. Thus, please rephrase to more accurately emphasize that TAO can (significantly) improve the classification accuracy for given decision trees, and particularly given its short runtime, suggests itself as a kind of postprocessing routine for whatever method may be used to obtain a tree in the first place.  2. I would have liked to see experiments combining TAO with the MIP approaches (OCT for axis-aligned, and OCT-H for oblique trees) by Bertsimas & Dunn (cf. suppl. mat.). Specifically, what are the improvements achievable by running TAO on the trees produced by these methods, and vice versa, what may be gained by providing TAO-trees (e.g., obtained by refining CART trees) to the MIP solver as initial solutions? However, I am not sure if the OCT code is accessible, so the authors may not be able to conduct such experiments in the short time until their response is due; but perhaps this is something to consider in the future.  3. There is some lack of (notational) clarity or imprecise wording that I would like to see resolved: a) top of p. 2, in „Finding an optimal decision tree is NP-complete [...]“ – only decision problems can be contained in NP (and thus, be NP-complete if they are at the same time NP-hard), not optimization problems. So, here, it should say „... is NP-hard ...“. b) p. 2, 2nd bullet: „...tends to get stuck in poor local optima...“ (I suggest inserting „local“ to be entirely clear here) c) p. 3, 2nd paragraph: „(multilinear) linear programming“ – unclear, what „multilinear“ is to mean here; please clarify. Also, in that paragraph, the papers cited w.r.t. the claim that the „linear program is so large that the procedure is only practical for very small trees“ are over 20 years old – are you sure that this statement still holds true, given the immense progress in LP-solvers over the past two decades? It would be nice to see a more recent reference for this claim, or else a remark on own computational experience in that regard. Finally, I don't quite see why the Norouzi et al. method should not be applicable to axis-aligned trees – are they not special cases of oblique trees (with unit vectors as decision hyperplanes)? Could you please clarify why it is supposedly not applicable? (Also, starting here, you talk about „induced trees“ several times, but never clearly say what „induced“ means, so maybe define that explicitly at first occurrence.) d) p. 3, 3rd par. and 2nd bullet: Early on (on p. 1), you state the node decision question in oblique trees as strict hyperplane-inequalities, and on p. 3 write that Bertsimas & Dunn „...smooth the strict inequalities ... by adding a small constant value to one side, which can alter the solution“ – does that really matter/is this even necessary? After all, as you yourself do later on p. 3, the decision question could simply be „flipped“ so as to work with non-strict inequalities. So, please, clarify if this makes any differences and if not, make the exposition of these aspects a bit clearer and more consistent. e) In Eq. (2) should be a minimization problem (and is indeed referred to as an optimization problem later on), but the „min“ is missing. f) Thm. 3.1 and its proof: The functional $\mathcal{L}_i(\theta_i)$ should be explicitly given somewhere, preferably in the theorem or else in its proof. (Gathering what it should look like from the present description only, I believe it would be $\mathcal{L}_i(\theta_i) = \sum_{n\in\mathcal{S}_i} L(y_n, T(x_n,\{ \theta_i : i\in\mathcal{S}_i \} )$ – is that correct?) Also, the word „constant“ is a bit misleading, although it becomes clear what is meant; if possible, perhaps clarify further. g) p. 4, 4th par.: Could you please elaborate a bit what you mean by „... a smaller tree that … probably generalizes better“ ? h) p. 4, „Optimizing (2) over a leaf“: Perhaps clarify the „majority vote“ (I take this to mean you pick the most-occurring label from training points routed to this leaf?). Also, what is „otherwise“ here – are the classifiers at leaves not defined to be labels...? i) p. 4, last par.: „...misclassification loss for a subset $\mathcal{C}_i$ ...“ – should it not be $\mathcal{S}_i$ here? Also, „...optimizing (2) where the misclassification error is summed over the whole training set is equivalent to optimizing it over the subset of training points $\mathcal{S}_i that reach node $i$.“ – and then summing the subset-errors over all $i$ ? (Else, I don't see this.) Please elaborate/be a bit clearer here. j) p. 5, 1st par.: So, is $\mathcal{C}_i$ a subset of $\mathcal{S}_i$ ? Perhaps include this relation at first occurrence of $\mathcal{C}_i$ to make it clear. k) p. 5, regarding „Sparse oblique trees“ – What about working with L0 directly rather than resorting to L1 as a sparsity proxy? (Naturally, this yields NP-hard problems again, but there are efficient iterative schemes that may work well in practice.) I don't expect the authors to work out the whole thing based on exact (L0-) sparsity measures, but I would like to see a brief discussion of this aspect at this point. l) In the suppl. mat., Fig. 2: The legend is sticking out – perhaps it could just be moved into the „empty“ part of Fig. 2 at the bottom right?  Finally, there are several typos, so please proof-read again and fix what you find (out of space to list all I found, but e.g.: "vs"/"vs.","eq."/"Eq.","fig./Fig.","inputed").


Hyperbolic Neural Networks
Hyperbolic spaces have recently gained momentum in the context of machine learning due to their high capacity and tree-likeliness properties. However, the representational power of hyperbolic geometry is not yet on par with Euclidean geometry, mostly because of the absence of corresponding hyperbolic neural network layers. This makes it hard to use hyperbolic embeddings in downstream tasks. Here, we bridge this gap in a principled manner by combining the formalism of Möbius gyrovector spaces with the Riemannian geometry of the Poincaré model of hyperbolic spaces. As a result, we derive hyperbolic versions of important deep learning tools: multinomial logistic regression, feed-forward and recurrent neural networks such as gated recurrent units. This allows to embed sequential data and perform classification in the hyperbolic space. Empirically, we show that, even if hyperbolic optimization tools are limited, hyperbolic sentence embeddings either outperform or are on par with their Euclidean variants on textual entailment and noisy-prefix recognition tasks.

A number of recent works in deep learning have been using Hyperbolic Embeddings to represent features in Neural Networks. One main advantage of these representations is that they use a more natural geometry to represent hierarchical data such as trees. This work seeks a generic theoretical grounding as well as practical technique for the use of Hyperbolic Geometry at various layers of a neural network.  Pros: * Overall well written and clear; * A lot of the ideas are well motivated, and the authors took care to give intuitive justifications for many of their choices; * Hyperbolic geometry has been used in multiple machine learning papers recently, and this new paper is a good attempt at proposing a general framework; * I found the use of parallel transport to define biases in hyperbolic networks, particularly nice and clever.  Cons: * experiments are very limited in details (in the main part of the paper), since this is mostly a theory paper, I see this as a minor point; * the introduction points to a number of recent works using hyperbolic geometry. It would be good to have a bit more details about differences between these approaches, and the one proposed in the paper.  Minor/easily fixed comments: * A number of things about the definition of a geodesic on lines 70-71 are not quite right. Firstly, there are many paths of minimal lengths between points on a Riemannian manifold that are not geodesics (in the Riemannian sense). Indeed, the geodesic paths are only the ones that are parameterised proportionally to arc length. For ex, on the real line R, the path $\gamma(t) = t^2$ is of minimal length between $\gamma(0)$ and $\gamma(1)$, but it is not a geodesic. Secondly, on a general Riemannian manifold, there can be paths between two points that are not of minimal length, but are geodesics nonetheless. For example, on a sphere, for two points close to each other, there are two geodesics along the great circle through these two points. What is true, is that geodesics are locally of minimal length. On the Poincaré ball, there is always only one geodesic that joins any two points. The author should make it clear that they use this very particular property, and that it is not clear in general. * On line 72, parallel transport is introduced as a transformation that depends only on the starting and end point $P_{x\rightarrowy}$, but parallel transport is a transformation that does depend on the whole path. In this paper, the authors implicitly always choose the unique geodesic between two points on the Poincaré ball. This should be made clear. * Definition of exponential map on line 76 is unclear: the paragraph makes it look like it is defined only for tangent vectors of norm 1. Which is incorrect. * On line 106, the sum of x and 0 should be x, not 0. * Around line 113, I would suggest adding that $(-1)\bigotimes_c x = -x$. * In Theorem 4, here as well the authors should make it clear that this is for parallel transport along a geodesic. * In the 'Reformulating Euclidean MLR' section, from line 177, the first proposed parameterisation of hyperplanes as $H_{a,b}$ is not actually used. Only the second one (which is a very natural parameterisation), $\tilde{H}_{a, p}$, is used. Introducing both parameterisations is confusing. Removing the first one would also save space.  More important comments: * All hyperbolic balls of the same dimension are isomorphic. Indeed the map $x \mapsto \frac{\sqrt{c}}{\sqrt{c'}} x$ is an isometric diffeomorphism that transforms $\bigoplus_c$ into $\bigoplus_{c'}$, and so on. This is an important point. And using this point would help declutter many of the formulas: the $c$ subscript could be dropped in most places, since the equivalent formula for a general $c$ can easily be deduced from the case $c=1$. The only time that $c$ is useful in the paper is when the authors consider what happens when $c$ goes to $0$. Note also that in the experiments, only $c=1$ is actually used. * Regarding the section on hyperparameters, at line 552, could the authors clarify what they mean by "the hyperbolic geometry is more pronounced at the ball border"? The geometry is constant across the whole Poincaré ball. In particular, all points are equal and for any point, there exists an isometry of the ball that brings this point to 0. The only particular thing for the border is due to the particular parameterisation of the Poincaré ball: as we approach the boundary (in the sense of Euclidean metric, since in terms of hyperbolic metric, the border is always infinitely far), we lose precision in our compute.  =====================  Thanks to the author for their replies. I have read them, and indeed think this is very good submission. Please, do point out the isomorphisms between the balls with various 'c'. This is the main confusing point in the paper.


Active Learning for Non-Parametric Regression Using Purely Random Trees
Active learning is the task of using labelled data to select additional points to label, with the goal of fitting the most accurate model with a fixed budget of labelled points. In binary classification active learning is known to produce faster rates than passive learning for a broad range of settings. However in regression restrictive structure and tailored methods were previously needed to obtain theoretically superior performance. In this paper we propose an intuitive tree based active learning algorithm for non-parametric regression with provable improvement over random sampling. When implemented with Mondrian Trees our algorithm is tuning parameter free, consistent and minimax optimal for Lipschitz functions.

•Post review comments: Assuming the authors will indeed address the unclear parts that I indicated, and in particular the proof of cor. 4.6 I am updating my score. Yet, I still think that their results (including those appearing in their response) are not showing significant gains as seen in other Active Learning settings.   The paper addresses the problem of active learning for regression with random trees. The authors introduce a simple ‘oracle’ algorithm and a risk variance is provided for this criterion. Authors provide a risk minimizing query criteria and claim it is better than the random selection criterion An estimation algorithm is presented that provides the necessary statistics at its first stage to allow an approximation to the ‘oracle’ algorithm. Numerical results are presented in which improved performance over random and uncertainty sample is provided for simulated data, and no improvement for real UCI data.  Strengths:  • The paper provides both an algorithm and theoretical analysis with empirical results. Theoretical analysis for active learning algorithms is a challenging task, lacking in many of the state-of-the-art heuristic algorithms for active learning • The organization of the paper’s sections is good Weaknesses: 1. There are many unclear parts in the paper, lacking formalism in some places which leaves the reader confused. I will give some examples for that in the order in which they appear a. Introduction, 2nd paragraph – the example requires the authors to make some analogy between the chemical quantities of the systems and concepts in active learning. It is completely unclear to me how this example makes a case for active learning. b. The authors make first reference to a leaf in the setting and background in page 2 line 54. However, no definition of a tree has been made yet, or even a simple reference to such.  c. Authors refer to n and d in line 93 and in algorithm 1 however it was never made clear what they are.  ‘n’, for example: Is ‘n’ the total size of data? Is it the test set size? Pool set size? d. Preposition 4.1 is not clear are we looking at the post selected leaf-set X after q_k proportion was selected? e. Most important: Key Corollary 4.6 proof is unclear, short, informal and confusing: what minimum and average are your referring to? This is a key result of the paper and it should be very clearly explained and proved! 2. Author should explain and motivate their decision to let $a_k=\frac{p_k}{\sgima^2_{Y,k}} in line 156. Why should the multiplicative factor have such a form? 3. Experimental results: Results are not convincing in demonstrating that the method is advantageous for real data. Authors should provide such examples or improve their algorithm. 4. In addition: a. Authors should provide basic data sizes for the datasets they experiment with in order for the reader to evaluate the efficiency of their method by, for example, comparing the size of the training set with the size of the full data set. b. Please provide reference for Breiman trees 5. Minor: many typos exist, authors should proof read their paper.


Model-based targeted dimensionality reduction for neuronal population data
Summarizing high-dimensional data using a small number of parameters is a ubiquitous first step in the analysis of neuronal population activity. Recently developed methods use "targeted" approaches that work by identifying multiple, distinct low-dimensional subspaces of activity that capture the population response to individual experimental task variables, such as the value of a presented stimulus or the behavior of the animal. These methods have gained attention because they decompose total neural activity into what are ostensibly different parts of a neuronal computation. However, existing targeted methods have been developed outside of the confines of probabilistic modeling, making some aspects of the procedures ad hoc, or limited in flexibility or interpretability. Here we propose a new model-based method for targeted dimensionality reduction based on a probabilistic generative model of the population response data. The low-dimensional structure of our model is expressed as a low-rank factorization of a linear regression model. We perform efficient inference using a combination of expectation maximization and direct maximization of the marginal likelihood. We also develop an efficient method for estimating the dimensionality of each subspace. We show that our approach outperforms alternative methods in both mean squared error of the parameter estimates, and in identifying the correct dimensionality of encoding using simulated data. We also show that our method provides more accurate inference of lowdimensional subspaces of activity than a competing algorithm, demixed PCA.

The paper introduces an extension of an existing model, targeted dimensionality reduction (TDR), developed by Mante and colleagues in 2013. TDR is a regression based method to perform dimensionality reduction of neural data sets. Unlike PCA, which finds a basis with components sorted by the amount of total variance captured in the data, TDR identifies dimensions that specifically capture task-related variance, regardless of how much of the total variability they explain. This method assigns a single dimension per task variable, which as the authors explain, presents a clear limitation. There is no reason to believe that a given external covariate would elicit only one mode of activation in the neural population. Furthermore, this restriction implies that an important amount of variance in the data is potentially being missed. Therefore, by considering multi-dimensional task-related subspaces, and by identifying a low-rank factorization of the matrices defining such subspaces, the author's method MBTDR provides a means to discover additional structure in the population, which is of key importance in neuroscience.  The features that MBTDR incorporates and are worth being highlighted are  1. They model is a probabilistic extension of TDR that defines a generative model of the data. This allows to incorporate a wide variety of assumptions, such as specifying different noise models or pre-defining structure using different prior settings.  2. The model can be fit to single trial data, even when there are unequal amounts of trials or missing observations. This is often the case in many data sets, as neurons are typically not simultaneously recorded, so the models are fit to the PSTHs, which are averaged responses. This averaging can destroy valuable information about the correlational structure in the population.  3. The authors provide a means of estimating the rank of the task subspaces. This is not straightforward as it involves searching over many possible rank configurations. The greedy algorithm proposed seems to be a reasonable approach to obtain optimal estimates for dimensionality. Similar methods like dPCA do not provide such a feature, although this can be accounted for post-hoc by keeping the number of dimensions needed to explain a certain percentage of the variance. This procedure, however, does not assess optimality.  As the authors acknowledge, dPCA offers similar modelling advantages over TDR as MBTDR does. In particular dPCA is also able to identify multi-dimensional task subspaces. The authors go on by stating the limitations of dPCA with respect to their own method and compare the performance of the two models. They do so by testing how well the models recover the different parameter subspaces on generated data. However, given that the data was generated from the model itself -and in spite of the noise corruption-, I am not sure whether it is indeed surprising that their model can recover the subspaces better than dPCA. Similar arguments apply to figure 2A (although I guess this is partially addressed with the estimation under Poisson noise). Therefore, from the simulation studies, it was not clear to me whether MBTDR provides an advantage over dPCA at the task both dPCA and TDR were designed for: that of demixing variance from different task parameters. An example showing this would have been more illustrative, or at least a clearer reference to this in the text.   I also could not really interpret this couple of sentences: "It is notable that the principle angle is an extremely sensitive measure of errors between subspaces and that both methods provide reasonable results when checked by eye. It is also notable that any differences are observable at all, which give us confidence that these results are quite strong."  Finally, it is perhaps worth mentioning one limitation of these type of modelling approaches, which is that, no matter how much we increase the dimensionality, we are still dealing with a linear method, so non-linear relationships with respect to the external covariates cannot be captured.    Regarding:  -quality The model setup and fitting procedure seems rigorous. The authors used standard methods both for parameter estimation and rank estimation. However, I was not entirely familiar with some of the procedures and I did not check the math in detail, so I would not say I am confident about the methods in general. In any case, the authors seem to provide enough information to reproduce the study (provided that the code is released).  -clarity  The paper is well written, clear and easy to read. The figures are also good. I think most readers will appreciate figure 1A, nice visual illustration of the decomposition!  -originality and significance The model is an extension of an existing method, so the study is not meant to be particularly original... but all the additional features that the model incorporates makes it an important contribution.   minor clarification:  line 219, SNR (in log10 units) is negative. These are high levels of noise, right? Any criteria for how this noise was set? how did parameter recovery depend on this?  typos: line 169: TDA --> TDR line 192: decent --> descent line 300: a using (superfluous), dimension reduction —> dimensionality reduction figure 1B/2A: legend, EM --> ECME (if you wish...)   In conclusion, I believe that the study is an important contribution to both neuroscience and machine learning. I did not identify any methodological flaws and the analysis seemed accurate. My only objection is whether the paper makes a strong enough neuroscientific point. First, regarding its superiority to identify neural structure and second, demonstrating that it is able to trade-off variance from different parameters -this last point if one wish to compare it to dPCA and TDR-. Except from this issue, which hopefully the authors can address, I would accept the submission with confidence.  --------------------------------------------- Comments on authors' rebuttal   The authors have satisfactorily addressed all the issues we raised. If the authors include additional analysis on one of the publicly available datasets from the dPCA eLife paper, that would resolve our biggest concern. Finally, making this method available to the neuroscience community will definitely lead to new insights when applied to different data sets and can guide the development of additional methods to analyse neural data.


A flexible model for training action localization with varying levels of supervision
Spatio-temporal action detection in videos is typically addressed in a fullysupervised setup with manual annotation of training videos required at every frame. Since such annotation is extremely tedious and prohibits scalability, there is a clear need to minimize the amount of manual supervision. In this work we propose a unifying framework that can handle and combine varying types of less-demanding weak supervision. Our model is based on discriminative clustering and integrates different types of supervision as constraints on the optimization. We investigate applications of such a model to training setups with alternative supervisory signals ranging from video-level class labels to the full per-frame annotation of action bounding boxes. Experiments on the challenging UCF101-24 and DALY datasets demonstrate competitive performance of our method at a fraction of supervision used by previous methods. The flexibility of our model enables joint learning from data with different levels of annotation. Experimental results demonstrate a significant gain by adding a few fully supervised examples to otherwise weakly labeled videos.

Paper Summary: The paper describes a method for spatio-temporal human action localization in temporally untrimmed videos based on discriminative clustering [3, 47]. The main contribution of this paper is a new action detection approach which is flexible in the sense that it can be trained with various levels and amounts of supervision. For example, the model can be trained with very weak level of supervision, i.e., train the model for action detection only using ground truth video-level action labels; and also it can be trained with full supervision i.e. with dense per frame bounding box and their class labels.  Experimental results demonstrate the strengths and weaknesses for a wide range of supervisory signals such as, video level action labels, single temporal point, one GT bounding box, temporal bounds etc. The method is experimentally evaluated on the UCF-101-24 and DALY action detection datasets.  Paper Strengths: - The paper is clear and easy to understand. - The problem formulation is interesting and described with enough details. - The experimental results are interesting and promising, clearly demonstrate the significance of varying level of supervision on the detection performance - Table 1. - On DALY dataset, as expected, the detection performance increases with access to more supervision. - The proposed approach outperforms the SOA [46] by a large margin of 18%  (video mAP) on DALY dataset at all levels of supervision. - On UCF-101-24, the proposed approach outperforms the SOA [46] when bounding box annotations are available at any level, i.e., Temp.+1 BB, Temp. + 3 BBs, Fully Spervised (cf. Table 1).  - The visuals are helpful, support well the paper, and the qualitative experiments (in supplementary material) are interesting and convincing.   Paper Weaknesses: I haven't noticed any major weakness in this paper, however would like to mention that - on UCF-101-24, the proposed method has drop in performance as compared to the SOA [46]  when supervision level is "Temporal + spatial points".  This work addresses one of the major problems associated with action detection approaches based on fully supervised learning, i.e., these methods require dense frame level GT bounding box annotations and their labels, which is impractical for large scale video datasets and also highly expensive. The proposed unified action detection framework provides a way to train a ML model with weak supervision at various levels, contributing significantly to address the aforementioned problem. Thus, I vote for a clear accept.


Objective and efficient inference for couplings in neuronal networks
Inferring directional couplings from the spike data of networks is desired in various scientific fields such as neuroscience. Here, we apply a recently proposed objective procedure to the spike data obtained from the Hodgkin-Huxley type models and in vitro neuronal networks cultured in a circular structure. As a result, we succeed in reconstructing synaptic connections accurately from the evoked activity as well as the spontaneous one. To obtain the results, we invent an analytic formula approximately implementing a method of screening relevant couplings. This significantly reduces the computational cost of the screening method employed in the proposed objective procedure, making it possible to treat large-size systems as in this study.

SUMMARY OF THE PAPER  Dealing with a relevant research problem -reconstruction of synaptic connections-; The authors put forth a new approximation to estimate neural couplings from spike data with implications for the analysis of functional connectivity. It took me a bit to understand what was the intended goal (aim or hypothesis never stated explicitly and contributions are declared at the end), but as soon as the intention was clear, the rest of the paper reads quite well. In their approximation, the authors make some assumptions (e.g. time invariance) are strict but otherwise acceptable and reasoned, and if I may, I would proceed just like the authors. The simulations on synthetic data are very informative and the results from the cultured cells gives a good idea of some of the practical limitations of the approach. I’m not sure I share the idea that some coactivity threshold should be based on a significance test, but then, it is not that I have a better recommendation or harder facts, so this latter it is only my humble opinion.    STRONG POINTS • Good and well explained experiments, both on synthetic data and cultured cells covering both spontaneous and evoked activity • Acceptable and well reasons decisions whenever there has been one • The contributed approximation for analysing neural couplings is an important contribution that (I understand) have low computational burden.  WEAK POINTS • Test on single topology • At times, the author seems to favour mathematical decisions over biological dictum (e.g. why should the optimal binning be based on hypothesis testing and not on some physiological guidance?). • Criticism of models such as the Ising models seem more concerned on justifying the authors choice than actually about pointing out a genuine limitation for the model and analysis   SUGGESTIONS TO (PERHAPS) IMPROVE THE PAPER Major • Well, mostly the first two afore stated as weak points above, and yet none is a critical impediment for acceptance; testing and reporting on more topologies will make the paper too dense, and my concerns on favouring some mathematical commodities over phenomenological construct are understandable within the specific frame of the proposal. I’m happy leaving the attendance of this two to the authors’ criterium. Minor • Avoid adjectives (e.g. difficult, naïve, etc) and whenever possible rely on objective (preferably quantitative) measures. • Line 230: Any patterns that the authors may appreciate on Fig 4? For instance, is there any links corresponding to neurons “seeing” the spike train at a particular lag? Any preferred anisotropy direction in the transfer of information across neurons worth mentioning? Does the distribution of error/noise follows any known distribution? • If not critical for the authors purposes and writing styles, I would suggest moving the contributions the introduction and perhaps declared explicitly the goal (providing a new approximation affording some specific benefit) from the abstract.


On Oracle-Efficient PAC RL with Rich Observations
We study the computational tractability of PAC reinforcement learning with rich observations. We present new provably sample-efficient algorithms for environments with deterministic hidden state dynamics and stochastic rich observations. These methods operate in an oracle model of computation-accessing policy and value function classes exclusively through standard optimization primitives-and therefore represent computationally efficient alternatives to prior algorithms that require enumeration. With stochastic hidden state dynamics, we prove that the only known sample-efficient algorithm, OLIVE [1], cannot be implemented in the oracle model. We also present several examples that illustrate fundamental challenges of tractable PAC reinforcement learning in such general settings.

This paper studies episodic reactive reinforcement learning with deterministic transitions where we have access to computational oracles (the method proposed uses a linear programming solver and a cost-sensitive classifier).  Under the assumptions that the value function and optimal policy are within provided function approximation classes, and transitions are deterministic, it shows that an optimal policy is PAC-learnable using a number of oracle calls and number of episodes that is polynomial in the horizon, action space size, and number of states per level.  The number of episodes also has a log dependence on the size of the value function approximation and policy approximation classes (so in the worst case this is polynomial in the number of states per level and the size of the action space).  It improves over previous work by considering computational complexity rather than just statistical complexity.  1. Role of the size of the function classes: Could the authors please comment on the role of the size of the function classes G and Pi in Theorem 2? My understanding is that we assume access to an oracle LP solver that can optimize over G in a single oracle call. Solving an LP is polynomial in the input size, but if the class G is exponentially large can I still solve the LP in polynomial time? This is relevant of course because (1) the paper assumes that the actual value function lies in G; (2) typically one does not know the form of the value function, and so one would like to include all functions; (3) the set of all functions from X onto [0,1] is equivalent to the set of real vectors with dimension |X|, which is exponential in |X|; Adding to the pain, if I use a history dependent state (because I have a POMDP and want to use the method in this paper), then |X| is exponential in H.  I agree that the example with a linear value function with a feature vector of not-too-large-dimension is appealing, but in this case we are unlikely to meet the assumption that the true value function is in the class of linear functions.  2. POMDP vs. MDP: Although this paper positions itself as studying POMDPs in the introduction, it assumes on lines 99-100 that the observation process (x_t) is Markov.  Moreover, the reward depends only on x_t and the action, not the state S_t.  They then correctly state (again, lines 99-100) that this makes the problem an MDP over X.  It argues "The hidden states serve to introduce structure to the MDP and enable tractable learning."  I don't understand why this is the case.  Suppose I simply set S=X, s_t=x_t, and defined Gamma(s,a) = Gamma(x,a).  Then my rewards, the observable stochastic process, the value function over X, and the set of optimal policies would all be unchanged.  Moreover, the paper would be easier to understand, no less general, and I believe would better represent what the method described in the paper is capable of.  Can the authors please explain more precisely why the hidden states enable tractable learning?   3. Somehow full POMDPs are very different from the MDP considered in this paper.  It was very useful for me to consider the following example:  Suppose we have binary actions and that x_h = a_1:h records the history of these actions. Also suppose that R(x,a) = 0 when x has length less than H.  When x has length H, it is equal to 1 if x is equal to a particular binary sequence b, and 0 otherwise. b is fixed but unknown. The set G that I use is the set of value functions that I can have in this problem, enumerating over binary sequences b of length H.  It has size 2^H. In particular, the g in G are of the form g(x) = 1(length(x)=H and x=b') for some b' in {0,1}^H.  My intuition says that the best algorithm for this problem (both from a computational efficiency and oracle efficiency perspective) is to sample paths at random until I find the one that gives me a final reward of 1. The number of paths I need to sample will have expectation 2^H / 2.  When I look at Theorem 2 and 3, at first glance they seem to be in contradiction.  However, my computation is very simple, and the size of my state space at each level M is exponential in H.  Thus, we actually have exponential dependence on H in both theorems.    Moreover, if we did not have the reactivity assumption, I could have had a much smaller state space. I could have S = {0,1} \times [H], where the second component tracks time, and the first component tracks whether my actions have specified all of the bits in b correctly thus far.  Then my M would be polynomial in H.  Could the authors please discuss this in the paper?  I believe it's an important aspect of the significance of the work.  4. Assumptions: The authors point this out, but it bears repeating that the assumptions are extremely strong.  As described above, the paper is really about MDPs in my view.  Moreover, the assumptions that the value function and policy are in some parametric class we are optimizing over is very strong.  This combined with the lack of numerical experiments make this a paper that has a very good theoretical contribution, but not yet relevant to practice.   Quality: high, though I would like more clarification along the lines of the comments above Originality: the paper is original, though it borrows many ideas from [1] Clarity: I found the paper hard to read.  In part, this is because the material is dense, but partly it is because the authors did not elucidate along the lines of comments #1-3 above. Signifance: I think this paper is a significant theoretical result.  #### UPDATE AFTER AUTHOR RESPONSE ##### Thanks for the responses to my major comments.   I continue to feel that this paper deserves to be accepted.