Latent Attention For If-Then Program Synthesis
Automatic translation from natural language descriptions into programs is a longstanding challenging problem. In this work, we consider a simple yet important sub-problem: translation from textual descriptions to If-Then programs. We devise a novel neural network architecture for this task which we train end-toend. Specifically, we introduce Latent Attention, which computes multiplicative weights for the words in the description in a two-stage process with the goal of better leveraging the natural language structures that indicate the relevant parts for predicting program elements. Our architecture reduces the error rate by 28.57% compared to prior art [3]. We also propose a one-shot learning scenario of If-Then program synthesis and simulate it with our existing dataset. We demonstrate a variation on the training procedure for this scenario that outperforms the original procedure, significantly closing the gap to the model trained with all data.

In this paper, the authors applied deep learning models for automatic translation of natural language descriptions into corresponding IFTTT recipes.  They investigated bi-directional LSTM and attention models.  They used one-shot learning for new channels and functions which have a very few recipes.
The paper is not organized well, especially the last sections. There is no conclusion or discussion section. It doesn’t provide any required explanation to emphasize the contribution. Presentation of the paper is bad; model architecture and formula descriptions are not clearly presented.   The paper was not written clearly, especially from the sections 4 to 7.  There are grammatical mistakes, and it is very hard to understand. The explanation of figures and tables are not correctly represented, and the data provided in the tables is not accurately represented in the respective sections. Architecture diagram of the model is not provided for easy understanding.  Symbols and notions are not used consistently. Symbols and notions are mixed up.  Same symbols are used for different variables/parameters. Authors used non-standard terms (E.g., Depending layer, Dependent layer, etc.).   It is difficult to evaluate the originality of this work. The authors claim that there is only one existing work [14] on this problem. Novelty in the proposed models is not clearly explained.  Formulization for HA is not clear in section 4.2.2.  Moreover, it has only a small practical impact since this problem is more specific.  The experimental settings differ for each model; hence comparisons among them are not valid.  The data used for the base line model [14] and the proposed models are different. The authors have mentioned that many recipes used in [14] are no longer downloaded from IFTTT.com (section 5.1).  Moreover, the experimental settings differ for the models HA and BDLSTM.   Model evaluation methods are not described in detail. LSTM description differs from [19].  All the symbols and notations used are not explained in section 4.1. The formulations are not clear. Justifications are not provided for choosing BDLSTM.    1) In the abstract, it is mentioned as HA outperforms by 10 % but in the introduction it is mentioned as 11 points. (line no. 8 and 55) 2) Section - Introduction: Please justify why BDLSTM is chosen for comparison than the other models.  3) Please justify why 10 is chosen as number of models for average. 4) In Line no. 38, it is mentioned as BDLSTM improves over [14] by 15%. But Table 2. shows BDLSTM (average 10 models) improves over [14]. Please also check the same in the line nos. 48 to 51. 5) Table 2.  shows HA (average 10 models) improves over [14] but in line no. 52 to 55, it is mentioned just HA.  6) The purpose of one-shot learning is not mentioned. Which setting it handles well? (line no. 56-58) 7) Section 4 - Model architecture:  how variable-length descriptions are converted into a fixed-size vectors? Please explain. 8) Please check the equation of ht. The dimension of Whh ht-1 is not same as the other terms Wxh xt and bh. 9) The dimension of ht is not mentioned. 10)  All the notations (symbols) used in the equations are not described. For e.g., i, f, o, g, etc. 11)  Please use different symbol for i in wi  (line no. 139) since i is used already.   12)  Please check the equation of ct  : dot product is wrong. It should be element-wise multiplication.  13)  The symbols t and T are mixed up in the equations and throughout the section 4. Please use different symbols for Transpose, Affine transform and upper bound. 14)  Please change pointwise production to element-wise multiplication (line no. : 131) 15)  In line no. 136, order of g index is not correct. 16)  Please explain the proposed model with the architecture diagram. How the models are built is not explained in detail. 17)  How data is embedded to vector is not described?  18)  In line no. 153, please cite and specify other works. 19)  How importance of each word is measured and how weights are computed? (line no. 154). Please explain in detail. 20)  The proposed Hierarchical Attention model is not explained clearly. Please explain the novelty of the proposed model ? 21)  What are all the other “extra subtleties” does HA capture and how? (line no. 165) The authors provided only one example on “to”. 22)  Please cite and specify “the usual attention mechanism”. (line no. 166) 23)  Symbols are not consistently used for tokens: wi  (line no. 139) and wT  (line no. 173). 24)  Depending layer, Dependent layer and Output representation are not standard terminologies. 25)  The formulation in the sub-sub-section 4.2.2 is not clear. 26)  Section 5 : In line no. 186, dependent -> hierarchical 27)  Development set and Gold Test Set are not standard terminologies. 28)  Table 2: Comparison of the model [14] with HA, HA (average 10 models), BDLSTM, BDLSTM (average 10 models) is not valid since the same dataset used by [14] is not used for this work. The authors say they are not able to download many recipes (line no. 191) as used in [14].  29)  The non-frequent tokens are mapped using < UNK >. How typos are handled; which are also not in the top 4000 most frequent tokens? (line no. 202) 30)  Why the scaling factor is different for HA and BDLSTM? (line nos. 208 and 218) 31)  Why the embedding vector sizes are different? (line nos. 215 and 221) 32)  It is not clearly explained how over fitting is avoided. (line no. 209) 33)  Error in representing the data presented in Table 2. (line nos. 230-231)  34)  How will BDLSTM (average 10 models) be 10% better than any single trained model? 35)  Line no. 236 : Performance of HA is not very much greater than BDLSTM 36)  Evaluation methodologies are not described clearly. Which metrics are used for the evaluations of the models? 37)  Which mistakes do HA and BDLSTM make? Please specify (line nos. 238-239) 38)  It is mentioned the error rate is reduced by half. But it is not shown how. 39)  Section 6 : Used same symbol for top-20 set and non-top-20 set 40)  All the models mentioned in Table 4. are trained differently. Hence the comparison is not valid. 41)  Please explain why it is difficult to have balance data for BDLSTM. 42)  Table 4: Why for BDLSTM - Minor (NonTop20) for SkewTop20 is very low (4.01%)? 43)  Section 7 is incomplete. 44)  How HA matches the author’s expectations? (line no. 304)   Minor comments: 1) Long sentence line no. 44-46 2) Please change points to percentage in lines 50, 55 3) Please change percentage points to percentage 4) Typo fo - of in line nos. 72 and  137 5) article ‘the’ is repeated : line nos. 168-169 6) English is not correct (line no. 175-178) – conjunctions are missed 7) Line no. 200 : such as characters - > characters such as 8) Present tense and past tense are mixed up. (e.g., line nos. 227, 228)


Stochastic Variance Reduction Methods for Saddle-Point Problems
We consider convex-concave saddle-point problems where the objective functions may be split in many components, and extend recent stochastic variance reduction methods (such as SVRG or SAGA) to provide the first large-scale linearly convergent algorithms for this class of problems which are common in machine learning. While the algorithmic extension is straightforward, it comes with challenges and opportunities: (a) the convex minimization analysis does not apply and we use the notion of monotone operators to prove convergence, showing in particular that the same algorithm applies to a larger class of problems, such as variational inequalities, (b) there are two notions of splits, in terms of functions, or in terms of partial derivatives, (c) the split does need to be done with convex-concave terms, (d) non-uniform sampling is key to an efficient algorithm, both in theory and practice, and (e) these incremental algorithms can be easily accelerated using a simple extension of the "catalyst" framework, leading to an algorithm which is always superior to accelerated batch algorithms.

This paper considers saddle-point problems with a convex-concave property. The authors show how this problem can be addressed using existing stochastic variance reduced techniques (such as SVRG and SAGA) coupled with a proximal operator. They study these algorithms in the monotone operator setting showing linear convergence. They also propose an accelerated variant as well as analyze a non-uniform sampling scheme.
Novelty/originality: The contributions of this paper are significant. The theoretical analysis is thorough and reveals some new insights for saddle-point problems. The authors propose many extensions in this paper, the most significant one being that their method applies to non-separable functions.  Technical quality: Although I really like the theoretical contributions of this paper, I was nevertheless quite disappointed in the way it is presented. I think the authors could have made the paper more accessible by making the connection to monotone operators more clear. This appears as an extension in Section 6 but the analysis provided by the authors is based on monotone operators. On the experimental side, I would have liked to see experimental results on more than two datasets. In the case of separable functions, the authors should also have compared to accelerated methods such as this stochastic variant of Chambolle-Pock algorithm: http://www.jmlr.org/proceedings/papers/v37/zhanga15.pdf Please add this reference to your submission.  There are a few points that I would like the authors should clarify: 1. How strong are the assumptions A-C, particularly assumption (A). For example, I wonder whether their assumptions hold for solving the saddle-point problem induced by SVM. 2. I can not fully understand the differences between stochastic primal-dual method and their suggested method. Can the authors elaborate on this point? 3. In theorem 2, I am not sure about constant \mu. Is it the monotonicity constant mentioned in appendix? How can we compare this result with the result of theorem 1?  4. The transition from assumption (A)-(C) to strong monotonicity (in appendix) is confusing. Do assumptions (A)-(C) imply monotonicity?   Minor issues: Some mix-up with the constant L used as the condition number and the Lipschitz constant: “The quantity L represents the condition number of the problem” and “we need the Lipschitz constant L” Forward-Backward algorithm: please add a reference


A Non-parametric Learning Method for Confidently Estimating Patient's Clinical State and Dynamics
Estimating patient's clinical state from multiple concurrent physiological streams plays an important role in determining if a therapeutic intervention is necessary and for triaging patients in the hospital. In this paper we construct a non-parametric learning algorithm to estimate the clinical state of a patient. The algorithm addresses several known challenges with clinical state estimation such as eliminating the bias introduced by therapeutic intervention censoring, increasing the timeliness of state estimation while ensuring a sufficient accuracy, and the ability to detect anomalous clinical states. These benefits are obtained by combining the tools of non-parametric Bayesian inference, permutation testing, and generalizations of the empirical Bernstein inequality. The algorithm is validated using real-world data from a cancer ward in a large academic hospital.

NA
Thank you for the submission. There are a few points of clinical clarity.  (1) The use of the term clinical state lacks specificity in the scope of this paper. The more targeted goal that is in line with other models in this area is to predict clinical deterioration (a shift in patient condition that requires a higher level of care). The goal of "clinical state" is less about understanding what it is but how shifts in the state drive an intervention (i.e. ICU transfer) (2) The authors should clarify the use case they describe. Though care practices may vary, in general, patients of these types receive treatment on an outpatient basis (i.e. they would come to the hospital daily to receive their treatment and then go home). Please confirm with your data providers whether or not this is the case. If so, I would modify the x axis of Figure 2a and 2b to be days (rather than hours). Hours seems to imply real time telemetry like data or an inpatient stay where vitals are checked routinely throughout the day and night. The authors do refer to admission time from 4 days to 85 days in some cases, but I'm not sure why these patients were admitted to begin with. It seems that there is some other inciting event that put them in the hospital (aside from just getting chemotherapy treatment). (3) If the authors confirm that the data is outpatient data, then a note should be made that the comparison to MEWS and Rothman index are somewhat apples to oranges in that both are designed for inpatient settings. However the comparison is fine as they are both aggregate scores of clinical conditions. (4) The authors hint at the models doing prediction with data 12 hours before an ICU admission or discharge. Is this in is a cross-validation framework? It seems like the same dataset is used to train and test the models so it is unclear what the model is missing. (5) Did the patient actually go to the ICU or are these estimates for what the labeling clinician thought as a set of vitals that actually required an ICU admission (i.e. the physicians were not given any clinical details and simply given the 17 temporal physiologic streams)? (6) line 252, I think you are referring to the wrong figure. (7) I'm a bit confused as to the defined priors. In the context of this study, the outcome is known (ICU vs discharge). In a practical application, there will be long runs when the next 12 or 24 hours will not result in any change. It would seem the prior distribution of the ICU conversion would be much smaller than indicated by the test data.


Learning Infinite RBMs with Frank-Wolfe
In this work, we propose an infinite restricted Boltzmann machine (RBM), whose maximum likelihood estimation (MLE) corresponds to a constrained convex optimization. We consider the Frank-Wolfe algorithm to solve the program, which provides a sparse solution that can be interpreted as inserting a hidden unit at each iteration, so that the optimization process takes the form of a sequence of finite models of increasing complexity. As a side benefit, this can be used to easily and efficiently identify an appropriate number of hidden units during the optimization. The resulting model can also be used as an initialization for typical state-of-the-art RBM training algorithms such as contrastive divergence, leading to models with consistently higher test likelihood than random initialization.

This paper describes a greedy method for adding hidden nodes to a restricted Boltzmann machine (RBM).  The method is derived by taking the marginal likelihood formula for the RBM, which involves a discrete sum of softplus functions using the weight vectors of the hidden units, and relaxing it to take an expectation with respect to an arbitrary distribution over weight vectors.  Optimizing this distribution using a method similar to functional gradient boosting yields the greedy algorithm for adding nodes.  The experimental results show that the resulting RBM benefits from further tuning using a more standard learning algorithm such as contrastive divergence, but the proposed algorithm is useful for finding the optimal number of hidden units, and provides a better-than-random initialization point for the RBM weights.
This paper is well-written and addresses the important practical issue of choosing the number of hidden nodes in an RBM.  The application of the Frank-Wolfe algorithm is not particularly novel, given the large set of related work that the paper cites.  But the paper makes the key technical move of replacing the discrete sum over hidden nodes in the marginal likelihood function with an arbitrary distribution over weight vectors.  This opens the path to applying Frank-Wolfe in a functional gradient setting.  I found this move interesting and novel (although there may be related work I don't know about); it might suggest similar approaches for other undirected models besides RBMs.  My main question is whether there are other techniques or heuristics that could be included for comparison in the experiments, besides the basic technique of training with various numbers of hidden nodes and various random initializations.  For instance, would it be feasible to compare to the ordered RBM approach of Cote and Larochelle?  Minor comments: * p. 1, line 37: "a efficient" should be "an efficient". * p. 4, line 140: "the second two terms in the gradient enforces".  Should be "enforce". * p. 5, line 169: "one can simply initialize...and continues".  Should be "continue". * p. 7, line 209: "CD-10".  Would be helpful to say what the 10 means here. * p. 7, line 218: "We randomly initialize the CD algorithm 5 times and select the best one".  The best one on the validation set, not the test set, right?  Would be good to make this explicit. * p. 7, line 227: "We evaluate the test likelihood of the model in FW after adding every 20 hidden units, and perform early stopping...".  The rest of this sentence suggests that the early stopping criterion is based just on the training and validation sets, not the test set, but the beginning of the sentence suggests that the test set is somehow involved.  It would be clearer to make these two separate sentences. * p. 8, Figure 2: I'm slightly color-blind and find it very hard to distinguish between the magenta line and the red line in these graphs.  Maybe use cyan instead of magenta, and use different marker shapes for the three series. * p. 8, line 254: "to higher test likelihood".  A verb is missing here; maybe "to achieve higher test likelihood".  == Update After Author Feedback == I appreciate the authors' responses to my comments.  It's interesting to hear that the iRBM tends to learn many more hidden units than the proposed method.  How does the accuracy compare between the two approaches?  I hope it will be feasible to include some comparison in the final version.


Robust Spectral Detection of Global Structures in the Data by Learning a Regularization
Spectral methods are popular in detecting global structures in the given data that can be represented as a matrix. However when the data matrix is sparse or noisy, classic spectral methods usually fail to work, due to localization of eigenvectors (or singular vectors) induced by the sparsity or noise. In this work, we propose a general method to solve the localization problem by learning a regularization matrix from the localized eigenvectors. Using matrix perturbation analysis, we demonstrate that the learned regularizations suppress down the eigenvalues associated with localized eigenvectors and enable us to recover the informative eigenvectors representing the global structure. We show applications of our method in several inference problems: community detection in networks, clustering from pairwise similarities, rank estimation and matrix completion problems. Using extensive experiments, we illustrate that our method solves the localization problem and works down to the theoretical detectability limits in different kinds of synthetic data. This is in contrast with existing spectral algorithms based on data matrix, non-backtracking matrix, Laplacians and those with rank-one regularizations, which perform poorly in the sparse case with noise.

This paper proposes, in spectral detection of global structures in data matrix, a means to resolve the localization problem, which is known to cause several spectral detection methods to fail in sparse and/or noisy cases. The proposal is based on learning a proper regularizer on the basis of localized eigenvectors of the data matrix, where the algorithm is such that one iteratively updates the regularizer so that the regularized matrix has no significant eigenvalues, with the associated eigenvectors being localized in terms of the inverse participation ratio (IPR).
Although the use of IPR in quantifying localization would work well in practice, as demonstrated in the section on numerical evaluations, it seems heuristic, since there would be several different quantities which would have qualitatively the same property as the IPR. Why it is a good quantifier compared with other possibilities in terms of detection of global structures should be discussed.     The author claims that the approach of trimming throws away part of the information, but one can see in the algorithm proposed in [12] that while trimming is used in the first stage the whole dataset is used in the final stage.     I was not convinced by the argument in page 3, which claims that spectral algorithms using the non-backtracking matrix can also be interpreted as an example of equation (1). In the interpretation the regularizer is dependent on the eigenvalue, which obscures the interpretation of these spectral algorithms as instances of the regularized spectral method (equation (1)). Specifically, I do not see any description of the claimed direct relation between the vector v appearing in Line 90 and the eigenvectors of the non-backtracking matrix.     Minor points:     - Line 32: Weigner's -> Wigner's     - Line 35: the wigner's -> Wigner's     - Line 37: which (is -> are) roughly     - Line 68: by learn(ing) a proper     - Line 75: that correlate(d)     - Line 103: that (are) specific     - Line 108: have (a large eigenvalue -> large eigenvalues) / which represent(s)     - Line 110: one who reveal -> ones which reveal     - Line 127: semi-cycle -> semi-circle (But in this case the bulk of the spectrum is not exactly a semi-circle.)     - Line 129: The spectra(l) density     - Line 130: the continu(es -> ous) part     - Line 152: the inverse participat(e -> ion) ratio     - Lines 179, 293: state-of-art -> state-of-the-art     - Line 220: between (duplicate)     - Line 227: assign items a hidden clusters -> assigns to items hidden clusters     - Line 228: then generate(s) similarity between (a) randomly sampled pairs of items     - Line 236: Here the term overlap is briefly explained, but it has already appeared several times before. The brief explanation should be given at its first appearance.     - Line 246: each selected node(s)     - Line 249: fail(s) to work     - Line 255: that (is) closely related     - Line 276: where (one) can see     - Line 286: a(n) accurate completion


) CIFAR Senior Fellow. (*) Indicates equal contribution
We introduce a method to train Binarized Neural Networks (BNNs) -neural networks with binary weights and activations at run-time. At train-time the binary weights and activations are used for computing the parameter gradients. During the forward pass, BNNs drastically reduce memory size and accesses, and replace most arithmetic operations with bit-wise operations, which is expected to substantially improve power-efficiency. To validate the effectiveness of BNNs, we conducted two sets of experiments on the Torch7 and Theano frameworks. On both, BNNs achieved nearly state-of-the-art results over the MNIST, CIFAR-10 and SVHN datasets. We also report our preliminary results on the challenging ImageNet dataset. Last but not least, we wrote a binary matrix multiplication GPU kernel with which it is possible to run our MNIST BNN 7 times faster than with an unoptimized GPU kernel, without suffering any loss in classification accuracy. The code for training and running our BNNs is available on-line.

The paper presents a method for training neural networks with binary weights and activations. It is essentially based on back-propagation where weights and activations are binarized during the forward pass, but the backward pass uses real-valued gradients and weight updates. The networks use batch normalisation and are optimised with Adam or AdaMax, which are also speeded-up by using binary shifts instead of multiplications. The framework is evaluated on MNIST, SVHN, CIFAR-10, and ImageNet where it performs worse than real-valued nets, but still demonstrates reasonable results. The authors also present the benchmarks of the GPU (CUDA) implementation of the binary matrix multiplication, which achieves a substantial speed-up over real-valued matrix multiplication.
Positives:  1) The topic of speeding-up training and evaluation of deep neural nets is very relevant, and the approach of moving the computationally intensive operations from real values to binary looks promising.  2) The method is evaluated on several datasets (including large-scale ImageNet).  Negatives:  1) The contribution is rather incremental, the main difference with BinaryConnect is that the activations are also binarized, which requires a rather minor change in the algorithm.  2) Experimental evaluation is carried out using two different frameworks (Torch and Theano), with different networks, optimisers, binarization schemes, etc. As a result, the experimental section does not look cohesive and is hard to analyse or refer to in future work. Instead of providing the recipes for training BNNs, it confuses the reader.  3) The real-valued baselines for the same networks and optimisers are missing (apart from the ImageNet experiments). This is probably the most important question the experimental evaluation should answer -- how big is the performance gap between real-valued and binary nets, all other things being equal. Such baselines are only presented for ImageNet, where the gap is rather large, e.g. 71%->47.1% for GoogLeNet. It would be useful to see similar comparisons for other datasets.  Questions:  1) For back-propagation through binarized activations, it is proposed to clip the gradients (eq. 5), but this approach is not used for back-propagation w.r.t. the binarized weights. Instead, it is effectively assumed that the gradient w.r.t. real-valued weights is the same as the gradient w.r.t. the binary weights g_{W_k^b}, but then there is clipping after the weight update. Why is back-propagation through binarized weights and activations done differently?  Summary:  Binary networks are a promising direction, and in principle the paper is interesting enough to be presented at NIPS, but the experimental evaluation does require improvement/re-writing. I would recommend the paper for borderline acceptance (conditioned on the authors improving the experimental section).


Online Pricing with Strategic and Patient Buyers
We consider a seller with an unlimited supply of a single good, who is faced with a stream of T buyers. Each buyer has a window of time in which she would like to purchase, and would buy at the lowest price in that window, provided that this price is lower than her private value (and otherwise, would not buy at all). In this setting, we give an algorithm that attains O(T 2/3 ) regret over any sequence of T buyers with respect to the best fixed price in hindsight, and prove that no algorithm can perform better in the worst case.

This paper considers an online pricing problem when the seller faces a steam of incoming strategic and patient buyers. The patience of buyers is modeled by a time window, beyond which buyer will leave. The arrival of buyers types are assumed to be adversarial. A EXP3 based learning algorithm is proposed. The basic idea of the algorithm is to bundle time steps are into epochs, and a fixed price (given by EXP3) will be announced for each epoch. Then the algorithm uses the feedback from the entire epoch to update EXP3. Upper bound on learner’s regret can thus be easily established via using the results from EXP3. Then the problem is shown to be reducible from a MAB problem with switching cost; a matching lower bound is developed nicely following the reduction.   This paper considered a different worker patience model for online pricing. The techniques for proving the lower bound (relate the strategic buyer model to a MAB with switching cost) are interesting, and may find applications in other problem settings. The results are presented clearly.   I read the authors' rebuttal.
While the problem is well formulated, and both upper and matching lower bound are nicely presented, the reviewer has the following concerns.  On modeling: The algorithm requires knowing workers’ patience. It is mentioned in the paper that such information can be gathered through buyers’ reports; but when such information is elicited from workers, would strategic manipulation and reporting complicate the algorithm design?   On the lower bound: Based on the reviewer’s reading, the lower bound is established over a learning space that the learning algorithm always posts price forehand for tau+1 time window sizes. So if this is the case, the proved lower bound is not quite the true lower bound, as the learner can be strategic about price posting, and so can the workers respond correspondingly. Also a relevant note: before presenting Algorithm 1, it is mentioned that the intuition is to minimize the number of times it changes prices, as the buyer would select the lower of the two prices for example when the window size is 2. But this argument is again based on the fact prices are pre-posted? If the learner can adjust the prices accordingly, forgoing the current price the buyer may risk missing a lower price in hindsight.   Other comments:  -The authors mention that modeling patient workers using time window is an important aspect of this work, while considering a discounted buyer will make the problem less trackable. But it looks like the discount model has already been considered in existing results, e.g., in [1],   -Just want to clarify the window size = tau (patience) + 1? so the claim that [11] is a special case of the current paper is not precise: [11] has tau = 0, while the set of results developed in this paper are for tau >= 1; as otherwise a sort{T} lower bound can immediately be established based on the results in [11].


Dual Decomposed Learning with Factorwise Oracles for Structural SVMs of Large Output Domain
Many applications of machine learning involve structured outputs with large domains, where learning of a structured predictor is prohibitive due to repetitive calls to an expensive inference oracle. In this work, we show that by decomposing training of a Structural Support Vector Machine (SVM) into a series of multiclass SVM problems connected through messages, one can replace an expensive structured oracle with Factorwise Maximization Oracles (FMOs) that allow efficient implementation of complexity sublinear to the factor domain. A Greedy Direction Method of Multiplier (GDMM) algorithm is then proposed to exploit the sparsity of messages while guarantees convergence to sub-optimality after O(log(1/ )) passes of FMOs over every factor. We conduct experiments on chain-structured and fully-connected problems of large output domains, where the proposed approach is orders-of-magnitude faster than current state-of-the-art algorithms for training Structural SVMs.

This paper introduces an approach to learning of structural SVMs based on a direction method of multipliers. The starting point is to bring the primal form of the learning objective into a dual-decomposed representation (eq. 9), based on the LP relaxation of the inference problem. This representation is frequently used for inference in intractable graphical models, and has meanwhile also been exploited successfully for structured learning (e.g. in [13,14]). Basically, the problem is broken down into maximization of individual factors, as well as minimization with respect to the model parameters as well as Lagrange variables coupling the factors.  Approaches then differ in how the coupling constraint are handled, and in how the minimization with respect to the model parameters is conducted.  In the present work, the objective in (eq. 9) is again dualized, yielding an objective that is basically a sum of dual SVM objectives - which needs to optimized over variables subject to simplex constraints (as usual in the dual SVM representation), as well as additional coupling constraints resulting from the individual SVMs also being coupled in the primal representation.  The coupling constraints are again handled by forming the Lagrangian; and updates with respect to the Lagrange multipliers are then performed by gradient descent. For minimizing the augmented Lagrangian function, the authors propose two different strategies: 1) Fully-corrective block-coordinate Frank-Wolfe (for factors/sub-SVMs of large cardinality) 2) Block-greedy coordinate descent (for a large number of factors/sub-SVMs)  For factors of large cardinality, it is sometimes possible to find the maximizing configuration in sub-linear time. The authors give a few examples of where this is the case. However, these are not novel insights to my knowledge but have been exploited before. (See, e.g. Martins et al.: "AD3: Alternating Directions Dual Decomposition for MAP Inference in Graphical Models.", JMLR 16 (2015) 495-545). This maximization in sub-linear time can be exploited in the suggested learning algorithms.  The authors perform an empirical comparison against several recent approaches, which suggest that their approach is highly competitive.
This manuscript definitely introduces some new ideas, and the empirical performance of the proposed algorithms seem strong. A lot of work was also put into the convergence proofs. These are non-trivial technical contributions that I find very valuable.  However, I feel that the presentation lacks clarity and polish, and that there is a lack of rigor in the experimental evaluation:  1) There is no conclusion. It is not clear to me why the authors omitted it, as it seems there would have been some space for this still. The conclusion is very helpful for the reader in order to put things into perspective, so I strongly suggest to include one.  2) The description of the experiments is very brief and lacks important information. For instance, it is not clear to me how the \rho parameter of Soft-BCFW was chosen, or how the other competing algorithms were parameterized. I also do not see any mention of the step size parameter \gamma required by the authors' own algorithm. There needs to be a clean protocol for choosing hyper parameters.  3) I am not certain that the experimental comparison was fair. For the comparison in terms of computation time, the authors only plot their "GDMM-subFMO" approach, which exploits sub-linear maximization complexity of certain factor types. The competing algorithms, on the other hand, do not seem to exploit this, even though in my understanding, Soft-BCFW also only requires a factor maximization oracle, and hence could be made to benefit from exactly the same strategy. To better put things into perspective, I would like to see a comparison of "naive" GDMM (without subFMO) with the other algorithms, in terms of computation time.  The claim of the authors is quite strong ("The proposed approach is orders-of-magnitude faster than the state-of-the-art training algorithms for Structural SVM"), so I think there must be a bullet-proof experimental validation that can also be reproduced by others. In the current state of the manuscript, I do not think that this is the case.


Optimal Binary Classifier Aggregation for General Losses
We address the problem of aggregating an ensemble of predictors with known loss bounds in a semi-supervised binary classification setting, to minimize prediction loss incurred on the unlabeled data. We find the minimax optimal predictions for a very general class of loss functions including all convex and many non-convex losses, extending a recent analysis of the problem for misclassification error. The result is a family of semi-supervised ensemble aggregation algorithms which are as efficient as linear learning by convex optimization, but are minimax optimal without any relaxations. Their decision rules take a form familiar in decision theory -applying sigmoid functions to a notion of ensemble margin -without the assumptions typically made in margin-based learning.

In this paper a worst-case optimal aggregation algorithm for binary classifiers under general losses is proposed. This work builds upon a line of work on semi-supervised and transductive learning for classification problems, most notably ref. [1]. The main contribution of the paper is a minimax formulation of the problem exploiting classifier correlation constraints. It is proved that this formulation leads to a convex program, giving a closed-form minimax optimal predictor under rather general conditions on the loss function (and even the type of side constraints used, but this is left in the Appendix). This result is based on a simple application of Sion minimax Theorem, together with a characterization of convexity of the minimax objective, the so-called the Potential Well.  As said before, this paper builds upon existing work, but it extends the applicability of these ideas very broadly. Technically, the paper does not introduce heavy machinery, but the obtained insights lead to new understanding of the problem which seems to be absent in the literature so far. On top of this, it is a very well written paper. For these reasons, I recommend this paper for publication at NIPS.
I believe this paper has great potential impact. First, it tackles a central question in learning theory (semi-supervised classification), it generalizes the literature considerably, and the obtained tools are scalable and practical. The technical content of the paper is good, although not extremely novel; despite this, it is able to provide insights that were not previously obtained in the literature.  A natural high-level question is whether the minimax approach is too conservative in a more benign setting. A possible way to formalize this question is whether the minimax results obtained can be translated to generalization bounds in the distributional setting, possibly by some additional tools such as regularization.  Next I would like to make some minor comments:  1. I couldn't find the justification of the "perfectly tight" minimax bound (Line 97). However, this is evident from Sion minimax Theorem (this is a zero sum game, which is quasiconcave on one variable, and convex --in fact, linear-- in the other), and I suggest to add this as a reference.  2. The discussion on lines 229-233 is confusing. I quickly checked [1], and I couldn't understand where the data distribution is used. Please clarify.  3. Finally, the equation following line 288 needs some further justification.


Structured Sparse Regression via Greedy Hard-thresholding
Several learning applications require solving high-dimensional regression problems where the relevant features belong to a small number of (overlapping) groups. For very large datasets and under standard sparsity constraints, hard thresholding methods have proven to be extremely efficient, but such methods require NP hard projections when dealing with overlapping groups. In this paper, we show that such NP-hard projections can not only be avoided by appealing to submodular optimization, but such methods come with strong theoretical guarantees even in the presence of poorly conditioned data (i.e. say when two features have correlation 0.99), which existing analyses cannot handle. These methods exhibit an interesting computation-accuracy trade-off and can be extended to significantly harder problems such as sparse overlapping groups. Experiments on both real and synthetic data validate our claims and demonstrate that the proposed methods are orders of magnitude faster than other greedy and convex relaxation techniques for learning with group-structured sparsity.

The authors consider an \ell_0 penalized overlapping group regularization problem, and a corresponding greedy iterative hard thresholding method. The main challenge is in the projection step encountered in the latter method. For this, the authors proposed to use submodular optimization to produce an approximate projection. Statistical analysis is provided for the resulting estimator.
I generally like the proposal and find the use of submodular optimization for the approximate projection to be interesting. But at the same time, I'm really not convinced that the proposal is actually useful, either theoretically, or practically. Various claims made throughout the paper are quite confusing to me, and this certainly doesn't help.  You criticize other works at the bottom of page 2 for assuming RIP, and also earlier in lines 26-28, but then you go and assume RSC above Definition 1. Aren't these essentially very similar assumptions ...? And furthermore, all in all, it seems to me that your results are pretty comparable to those from convex methods, if not a little weaker. So I'm left wondering: what is the point, theoretically?  From a practical perspective, I could see Algorithms 1 and 2 be appealing for their simplicity. I could also imagine them being efficient. But the experiments given in the paper are pretty limited. The same can be said of the one experiment in the supplement. I think a much more thorough analysis and comparison should be conducted (especially if the theory is about on par with existing theory).   What are FW, CoGEnT, and GOMP? It's not helpful to have the reader guess, but rather it would be worth spending the time explaining what these are down, even just briefly. I can guess was GOMP is, but FW = Frank-Wolfe is just a generic algorithm that can be applied to a wide range of optimization problems so that is not helpful, and I have no idea what CoGEnT is. Are any of these (perhaps FW) being applied to the same problem you are considering in (4) (or (5))? Are any of these considering the convex analog of (4) (or (5))? I don't think so, and convex analogs should really be compared, especially given your high-level discussion throughout the paper comparing your results/approach to convex ones.  Other specific comments:  - Lines 23-25: "In practice, IHT methods tend to be significantly more scalable than the (group-)lasso style methods that solve a convex program. But, these methods require a certain projection operator which in general is NP-hard to compute..." I know what you mean to convey here, but it reads a little funny. You mean that IHT methods are scalable in practice, becuase they only approximately solve the problems that they are designed for. Exact solutions are NP-hard.  - The notation in (1) is a little crazy. It is almost universal to use \beta as the regression parameter. You use it as the noise variable. And you use \lambda as the noise variance, which is also confusing, because this is often reserved as a regularization parameter (say, penalty parameter for the group lasso).  Again it is almost universal to use \sigma.   - I'm confused about the discussion at the top of page 2. First of all, in the 2nd paragraph, you say that the improvements in sample complexity from the group lasso over the lasso are only logarithmic, which means that \log{p} becomes \log{M}, with M being the number groups and p being the number of features. (An aside, I don't see this as a bad thing; the feature dependence is already logarithmic to begin with, so what more could you ask for?) But isn't this precisely analogous to what you are claiming in the 4th paragraph, with your IHT group sparsity results, over IHT standard sparsity results?  Furthermore, I don't understand the point of paragraph 3 and how it relates to your own work that you describe in paragraph 4. You say in paragraph 3 that "Greedy, Iterative Hard Thresholding (IHT) methods have been considered for group sparse regression 46 problems, but ... the guarantees they provide are along the same lines as the ones that exist for convex methods."  Then in paragraph 4, you say that "In this paper, we show that IHT schemes with approximate projections for the group sparsity problem yield much stronger guarantees." This doesn't really make sense as written; are you using different approximation schemes, is that the point? And also, the guarantees that you describe in paragraph 4 for your method still do sounds quite similar to those from convex programming...


Eliciting Categorical Data for Optimal Aggregation
Models for collecting and aggregating categorical data on crowdsourcing platforms typically fall into two broad categories: those assuming agents honest and consistent but with heterogeneous error rates, and those assuming agents strategic and seek to maximize their expected reward. The former often leads to tractable aggregation of elicited data, while the latter usually focuses on optimal elicitation and does not consider aggregation. In this paper, we develop a Bayesian model, wherein agents have differing quality of information, but also respond to incentives. Our model generalizes both categories and enables the joint exploration of optimal elicitation and aggregation. This model enables our exploration, both analytically and experimentally, of optimal aggregation of categorical data and optimal multiple-choice interface design.

The paper describes a special setting for crowdsourcing labelling with multiclass labels. Unlike previous studies, where an agent (worker) choose the most likely class label, in the present study, an agent is asked a number of questions about his/her posterior probabilistic distribution over class labels. The problem setting assumes that the agent is incentivized to provide true answers. Unlike previous papers, the goal of the current study is to optimize the interface of the questionnaire. Some theoretical analysis is provided for simple cases of the general setting and some experiments are provided to compare a proposed heuristic approach with simple baselines.
**Novelty** The idea of optimizing a labelling interface, which elicit confidence, is interesting and novel. The problem setting would be a good contribution to the literature on crowdsourcing. However, I am not sure that paper is ready for publication for the following reasons:  1) the theoretical part looks not solid, 2) the proposed algorithm (HA) looks not grounded, 3) the results of experiments are not significant.  These points are supported below.  **Theoretical part (Section 4):** Though Theorem 2 is interesting, it is very simple. Lemmas 3,4 are reasonable, however, they cover only very special cases. Specifically, Lemma 3 considers only one agent and Lemma 4 assumes that all agents have the same amount of information (they observed exactly $n$ samples). Both lemmas consider only binary case, while it would be interesting if they could be generalized to multiclass setting. In particular, the following statement seems to be provable in a similar way: **Statement.** In the general setting of $k$ classes, among the shadow partitions (and among all $k$-cell partitions), the optimal one is the partition is $D_r = \{p\mid r\in \argmax_x p(x)-p^*(x)\}$, where $p^*(x)=1/k$ is the uniform distribution over classes.  By the way, the proofs are not perfectly compact and sometimes can be simplified. For example, the first two outline equations in Section F are redundant, are never used further, so their appearance confuses the idea of the proof.  **Proposed algorithm (Section 5):**  No approach to solve the optimization problem (1) is proposed in a general case. There is only a heuristic aggregation (HA) method (line 265), which is applicable only in the case when agents have equal accuracy (equal number of observations $n$). Moreover, HA seems to be biased, while I would suggest that some MC-like method would better estimate the posterior probability $P(\teta\mid D^1,\ldots, D^m)$ (see lines 188-189).  **Experiments:**  I don’t appreciate Section 5.2, because only one simplest baseline is considered (Majority voting) and this baseline does not take into account priors on classes. A more appropriate baseline would be one of the consensus models used in crowdsourcing.  In section 5.3, the budget of the two treatments are different and the probabilities of bonus payment are also different. Are the values fitted in a way to guarantee the equality of the expected costs of the two treatments? What were the actual costs in the experiment? If the costs are different in expectation, this makes the results less interpretable. When choosing the threshold and the payment scheme in section 5.3, it is assumed that the principal knows the proportion of examples in categories, however, in practice it is usually unknown, also the proportions of categories for golden set samples is biased towards easy categories. How to choose the threshold in such practical case? What would be if the assumed proportions for categories were wrong?    **Other comments:**  Lines 162-164 tells nothing new.  **Questions:** 49: “If the principal can elicit agents’ entire belief distributions, our framework can achieve optimal aggregation, in the sense that the principal can make predictions as if she has observed the private information of all agents”. Can you reformulate this sentence so that one could understand it before reading the theoretical part?   148 “This assumption can be satisfied by, for example, allowing the principal to ask for an expert opinion outside of the m agents”. Outside expert opinion sounds like a correct answer, which is not noisy (otherwise, what is the difference between the expert and an agent?). Then the distribution $p(x\mid \teta^*)$ of the expert opinion should be deterministic: $p(x\mid \teta^*)=Id(x=\teta^*)$.  252 Does simple majority voting take into account priors?  Figure 2. What are the values of $\epsilon$ and $n$ in the experiment reported?  292 I did not get the sentence.  312. “Performing HA for Baseline and Maj for ProbBased both led to higher aggregation errors” What are the quantitative results? It would be interesting to examine.   **Misspellings:**  23 “Moreover, disregard of whether full or partial information about agents’ beliefs is elicited, aggregating the information into a single belief or answer is often done …” I believe, there should be “disregarding whether”.  163 ”a agent” -> “an agent”  165 “what are all the scores which are truthful for it”. I believe, “scoring rules” is meant instead “scores”?  194 “The expectation is taken over which partition D^i…” I believe, there should be “which cell”, not “partition”.  236 “if the number of agent is one” -> “if the number of agents is one”    In Supplementary file:  387 ”a agent” -> “an agent”  389 “can can”  Figure 3: “theta=0” -> “$\theta=0$”.   ------  **UPDATE on Author Rebuttal**  After reading rebuttal, I still have some additional arguments on some points.  - A: While Theorem 2 might seem simple, it resolves an open question in previous work as it was considered impossible to obtain the global PPD from the PPDs of all agents.  Though Theorem 2 is reasonable, I would avoid to claim that an open question was resolved. In fact, I don’t think Theorem 2 was previously conjectured in the literature, since it resolves a rather particular and quite simple question (unlike general questions considered in [1]). Moreover, I would rather name it “Statement”, not “Theorem”, since the proof is even easier than to find the right statement.  - A: We also share the reviewer's conjecture on the multi-signal case (indeed, Fig 2 provides some evidence), though given the difficulty of proving Lemma 4, we do not think such a result would be easy to establish without new techniques.  I claim that I can generalize Lemma 3 to the case of multiple classes (as I stated). I believe, it is a proved statement (in mi mind), not a conjecture. I also believe that the paper would be more thorough if this statement was included.   - A: It is true that we do not give a fast algorithm to solve the optimization problem. Ffor small n (as in our experiments), it is feasible to just search over the discretized parameter spaces. Ideally, we could give fast optimization algorithms by MC-like methods. In practice, however, we felt that a heuristic grounded in intuition but supported by the theory would likely be more valuable for practitioners; fortunately, such a heuristic performs very well empirically. Moreover, these heuristics give qualitative insights about how to place thresholds and aggregate.  I have two different issues here, which ate not connected: 1. No approach to solve the optimization problem (1) in the general case of workers *with different accuracies*. 2. In the case of equal accuracies, HA seems to be based on *biased* estimates of posterior probability $P(\teta\mid D^1,\ldots, D^m)$, therefore, some sampling methods would be more appropriate.  - A: We assumed the prior is known. In practice, there are often historical data that provide good estimate of the prior. It's a very interesting and important future direction to study the robustness of our approach to incorrect priors.  It is not clear how one can use historical data of workers’ labels to set the prior, since these labels are usually biased.


Threshold Learning for Optimal Decision Making
Decision making under uncertainty is commonly modelled as a process of competitive stochastic evidence accumulation to threshold (the drift-diffusion model). However, it is unknown how animals learn these decision thresholds. We examine threshold learning by constructing a reward function that averages over many trials to Wald's cost function that defines decision optimality. These rewards are highly stochastic and hence challenging to optimize, which we address in two ways: first, a simple two-factor reward-modulated learning rule derived from Williams' RE-INFORCE method for neural networks; and second, Bayesian optimization of the reward function with a Gaussian process. Bayesian optimization converges in fewer trials than REINFORCE but is slower computationally with greater variance. The REINFORCE method is also a better model of acquisition behaviour in animals and a similar learning rule has been proposed for modelling basal ganglia function.

This paper represents an interesting application of two existing learning algorithm, Williams' reinforce algorithm and Bayesian optimization, on estimating the decision threshold in 2-AFC task. The threshold is estimated by minimizing the difference between the reward sum of the optimal decision policy and the sum of collected reward. The author also compares the performance of the two algorithms and maps the SPRT decision performance over main parameter space.   This paper looks technically sound. I like the main the idea of this paper because estimating decision threshold is an important problem in decision science.
Technical quality: This paper focuses on estimating a decision threshold - the problem which is not sufficiently studied in this community. Two existing algorithms are used to estimate the threshold and their performances are compared against each other. The paper is interesting and technically sound.  Novelty: The idea of estimating the threshold has some novel contribution to the computational psychology society. However, the algorithms used in this paper are not novel because the author uses these algorithm without much modifications based on the previous work [5],[6] & [7]. Moreover, in this paper,  other parameters need to be fixed when the optimal threshold is being estimated. Since other parameters are fixed, a naive approach can be searching the parameter space of the threshold to minimize the cost function, which can be computed by averaging trials simulated with SPRT method. The complexity of this problem is similar to other problems aiming at estimating parameters with fixed threshold.I think the main novel contribution of this paper is trying to estimate the decision threshold which has not been sufficiently studied.  Potential impact: This paper estimates a constant decision threshold in 2-AFC task with infinite horizon. I think this paper will attract attention from the researchers in psychology and cognitive science society. However, in most psychology experiment, the 2-AFC task needs to be finished within a certain deadline, thus making the threshold change over time within each trial. I would like to see a more sophisticated estimation algorithm to estimate the time-dependent threshold under finite horizon. Moreover, this paper would have more impact if the algorithm can jointly learn the threshold with other parameters.   Clarity and presentation: The paper is not very clearly written. The method section misses a lot of important information. It would be hard for some readers who are trying to replicate the work by reading through this paper. For example, in section 3.2 REINFORCE METHOD, the author should provide an illustrative network figure so that the reader can have a clearer idea about what your inputs are and how your network is trained. The information in Sec 3.3 is also vague. I understand that the author lists corresponding references including more details. However, the main framework of those algorithms should be clearly explained in this paper. I would recommend the author includes more information about the algorithms and condense the long discussion section (cut by half). Also, please check and correct a few typo, e.g.,  1.For equation (6), I think you mean "-NC_risk/c" 2.Line 114 "work work well"  --------------- After discussion, I realized that optimal threshold has already been studied by other researchers with different approaches, so I lowered the impact score as the the advantage of the methodology used in this paper is not clear when compared to the others. I also increased the clarity score after I read the paper again.


Learning and Forecasting Opinion Dynamics in Social Networks
Social media and social networking sites have become a global pinboard for exposition and discussion of news, topics, and ideas, where social media users often update their opinions about a particular topic by learning from the opinions shared by their friends. In this context, can we learn a data-driven model of opinion dynamics that is able to accurately forecast users' opinions? In this paper, we introduce SLANT, a probabilistic modeling framework of opinion dynamics, which represents users' opinions over time by means of marked jump diffusion stochastic differential equations, and allows for efficient model simulation and parameter estimation from historical fine grained event data. We then leverage our framework to derive a set of efficient predictive formulas for opinion forecasting and identify conditions under which opinions converge to a steady state. Experiments on data gathered from Twitter show that our model provides a good fit to the data and our formulas achieve more accurate forecasting than alternatives.

In a nutshell, the paper proposes a new model for opinion-dynamics in social networks (where users can publish             messages/posts/tweets). The authors discuss how their model can be efficiently trained, used to predict opinion             dynamics. The experiments demonstrate that the model has more predictive power compared to existing alternatives.              In my view, the following aspects of the model make it stand out among the classical models for opinion dynamics:             1) The opinions of individuals are updated asynchronously. That is, only the opinion of those who observe a new             post/message gets updated.             2) The model captures the intentisity of the posted messages which highly impacts how the opinions will develop. On the             other hand, most classical models assume that opinions get updated at a constant rate.             3) The model assumes that the observed signals (posted messages) are simply noisy reflections of the underlying opinions.  This makes the model more robust and applicable when dealing with real-world datasets.              The technical aspects of the paper is mostly concerned with how the proposed model can be viewed as Markov process and how it can be expressed using SDEs. Furthermore, the paper discusses how the model can be trained using existing methods             and how it can be simulated. A simulation, in this case, is essentially equivalent to sampling from the proposed generative             model which is not trivial, but the authors point out how it can be done using existing algorithms.              Finally, the experiments highlight that (a) the model is complex to produce exteremly different outcomes (i.e, consensus vs. polarization), and (b) the model has a higher predictive power compared to existing opinion-dyanmic models. The real-world             experiments are performed on datasets obtained from twitter.
          The technical aspects of the paper (i.e., the genarative model, how it can be learned, and how it can be simulated) are quite             interesting. The proposed model is a complex data-oriented model (compared to its alternatives) which I expected to be too difficult to study analytically. Nevertheless, the paper talks about both simulation-based and analytical forcasting using the proposed model.  As I mentioned earlier, the proposed model has some unique properties that makes it interesting compared to classical models for opinion-dynamics.              While I really enjoyed the technical contributions of the paper, I feel the experiments don't do justice to the paper. For instance,             1) the predictive power of the model is only tested on twitter.             2) two (decay) parameters of the model are determined through cross-validation. Is this search easy? What is the appropriate range             and granularity for these parameters? Is there a smart way to search this space?             3) It is not really clear how much data is used for training. This is only reported in percentages, but I'm curious to know             how many tweets were available when an opinion is forecasted (say in Figure 4).              Overall, I think the paper is well-written with significant technical contributions. To me, the experiments can be enriched to further improve the quality of the paper.


Balancing Suspense and Surprise: Timely Decision Making with Endogenous Information Acquisition
We develop a Bayesian model for decision-making under time pressure with endogenous information acquisition. In our model, the decision-maker decides when to observe (costly) information by sampling an underlying continuoustime stochastic process (time series) that conveys information about the potential occurrence/non-occurrence of an adverse event which will terminate the decisionmaking process. In her attempt to predict the occurrence of the adverse event, the decision-maker follows a policy that determines when to acquire information from the time series (continuation), and when to stop acquiring information and make a final prediction (stopping). We show that the optimal policy has a "rendezvous" structure, i.e. a structure in which whenever a new information sample is gathered from the time series, the optimal "date" for acquiring the next sample becomes computable. The optimal interval between two information samples balances a trade-off between the decision maker's "surprise", i.e. the drift in her posterior belief after observing new information, and "suspense", i.e. the probability that the adverse event occurs in the time interval between two information samples. Moreover, we characterize the continuation and stopping regions in the decisionmaker's state-space, and show that they depend not only on the decision-maker's beliefs, but also on the "context", i.e. the current realization of the time series.

This paper proposes an interesting model of decision making in a situation where the sensory observation depends on the occurrence of the adverse event, i.e., the deadline. The subject needs to infer the deadline, i.e., occurrence of adverse event, based on the history of sensory information.  The model demonstrates how to compute the optimal “date” for gathering the next information by balancing the Bayesian surprise and suspense. The optimal policy, the rendezvous policy, can be obtained using Bellman optimality condition. The papers also shows that the continuation and stopping regions depend not only on subject’s belief, but also the context of the sensory information.  This paper is interesting and  technically sound, though a practical application of this model is not clear.  I expect that this paper should be able to draw certain attention from a small group of researchers in this community. However, I personally think this paper is more suitable for publication in a journal like SICON.
Technical quality: This paper proposes an interesting model and derives three main theorems. The paper demonstrates when the decision maker acquires new information (continuation) and when makes a final prediction (stop) in a scenario where the deadline depends on the sensory information. It also shows that the subject's belief state is a supermartingle, and that the optimal policy has a "rendezvous" structure. The continuation and stopping region in this model depend both on the subject's belief state and "context" of the sensory observations. The paper is interesting and technically sound.   Novelty: Most decision making models for 2AFC and stop signal task assume that the  deadline is independent of the sensory information. The  main reason is because  those cognitive tasks are designed in a particular way such that the sensory information does not convey the information about the occurrence of the deadline. For example, in 2AFC, the subject needs to decide the moving direction of the stimulus within a fixed or random deadline, which does not include any information about the stimulus. This paper proposes an interesting model of decision making in a situation where the sensory observation depends on the occurrence of the adverse event, i.e., the deadline. The model is interesting and has certain novelty.  Potential impact : Although the model is conceptually novel, the structure of cost function is similar to previous Bayesian decision models in 2AFC. The dependence between observation and deadline can also be included in previous Bayesian decision models. From computational point of view, the impact of this paper is limited. It is not clear how we can build a practical algorithm and run simulations based on the result of this paper. Again, I think this paper is more suitable for publication in a theoretical journal because the proof of the theorem is critical and should at least be sketched in the paper.   Clarity and presentation: This paper is well written and easy to follow. The author makes effort in explaining each theorem beside the equations. I would recommend the following improvement: 1. Provide the expected cost function to be minimized to give readers a clearer idea about the dependency between the deadline and observation context. 2. Use different notations for the sigma-algebra generated by the process' survival  and by the stopping event. 3. Check a few typos existing in this paper.  ----------------------- I have read the rebuttal. Like I said in my previous review, I like the main idea of this paper which tries to solve an important problem in decision making. My main concern is still the same. It is not clear how we can build a practical algorithm and run simulations based on the result of this paper. The whole paper is written in the language of probability measure theory and makes it very hard to be understood by the majority of the audience in the NIPS community. I definitely appreciate the theoretical contribution of this paper. However, I highly recommend the author submitting this paper to a more theoretically oriented journal in control theory or probability theory.


Interaction Screening: Efficient and Sample-Optimal Learning of Ising Models
We consider the problem of learning the underlying graph of an unknown Ising model on p spins from a collection of i.i.d. samples generated from the model. We suggest a new estimator that is computationally efficient and requires a number of samples that is near-optimal with respect to previously established informationtheoretic lower-bound. Our statistical estimator has a physical interpretation in terms of "interaction screening". The estimator is consistent and is efficiently implemented using convex optimization. We prove that with appropriate regularization, the estimator recovers the underlying graph using a number of samples that is logarithmic in the system size p and exponential in the maximum coupling-intensity and maximum node-degree.

This paper deals with reconstructing the graph of an Ising model. The paper is well-written. The main effort is to improve upon the  recent results provided by Bresler, showing that the complexity of identifying the structure of max degree d Ising model is polynomial  in p and independent of d.   Strong Points:   1) The timeliness of the topic in this paper is good, meaning that there is currently ongoing interest and work on Ising model reconstruction. 2) The paper contains more or less all the necessary theoretical results (mainly in terms of sample complexity) that one would expect to see in such a draft.  3) The approach to the solution, i.e., the use of the ISO and the l_1-penalization is quite intuitive in the way that is described in the paper.  Weak points:   1) The whole approach is based on the introduction of the ISO. This is the main trick in the proposed approach. Other than that, the rest  of the method and its analysis are usual and well studied (l_1-penalization and connection with the tutorial by Negahban et. al. on M-estimators), i.e., the rest of the proposed approach is nothing but very usual and well-studied approaches and ideas.    2) Due to the previous point, the analysis is based on well-established proof techniques or even more crucially similar mathematical statements  that have been extensively derived and proposed in the related literature. 3) The authors need to highlight better the discussion in lines 166-170. 4) I would like to see more simulations demonstrating the performance of the proposed method.  Overall, I think that the paper is nice, nevertheless with the aforementioned weaknesses under consideration.
The timeliness of the topic and the derived results are more important according to my opinion than the aforementioned weaknesses. Although normally I would not suggest publication of papers with l_1-penalization, since I think that these approaches have been extensively studied so far in literature, in this particular case I think that an exemption can be made.


An Architecture for Deep, Hierarchical Generative Models
We present an architecture which lets us train deep, directed generative models with many layers of latent variables. We include deterministic paths between all latent variables and the generated output, and provide a richer set of connections between computations for inference and generation, which enables more effective communication of information throughout the model during training. To improve performance on natural images, we incorporate a lightweight autoregressive model in the reconstruction distribution. These techniques permit end-to-end training of models with 10+ layers of latent variables. Experiments show that our approach achieves state-of-the-art performance on standard image modelling benchmarks, can expose latent class structure in the absence of label information, and can provide convincing imputations of occluded regions in natural images.

Arguing that the success of DRAW-like models is due their incremental approach to generating data combined with deterministic connections present between each latent variable and the observation, the authors apply the same design principles to hierarchical latent variable models. The resulting models, called Matryoshka Networks (MatNets), consist of a bottom-up (BU) network, top-down network (TD), and merge modules that directs information flow between the two. The MatNet architecture is reminiscent of Probabilistic Ladder Networks (PLNs) [28], with a deterministic pathway from the observation to a latent layers in the inference model and a top-down factorization of the variational posterior. The primary difference are deterministic connections between the latents and the observation in the Matryoshka Network, which are not present in PLNs. MatNets are evaluated at density modeling on MNIST, Omniglot, and CIFAR achieving either highly competitive or state-of-the-art results.
This is an interesting and fairly well executed paper.  The main contribution of the paper is a hierarchical VAE architecture with deterministic connections in the inference and generative models, implementing incremental generation of observations. A similar approach has been pioneered by DRAW, but in DRAW-like models the layers of latent variables don't form a hierarchy (and are all at essentially the same level). Prob Ladder Nets have all the proposed features except for the deterministic connections in the generative model. Thus the main novel contribution here is the introduction of deterministic connections in a generative hierarchical model. Surprisingly, the experimental section has no experiments demonstrating the importance of deterministic connections in the inference model and/or the generative model. Instead, the experiments show that the proposed architecture performs very well on several dataset, which though interesting but does not provide direct evidence for the importance of the paper's central claim.  The paper is fairly well written, but the presentation of the proposed architecture is not clear enough to be easily accessible. It would be much easier to read if the high-level (probabilistic) description of the model came before the specification of the exact computations involved (i.e. Procedural Description in Sec. 2.1).  In a few places the lack of citations can make some statements appear to be claims of novelty. For example, the discussion of using convolutional GRUs modules fails to mention the prior use of these in generative models in [24]. Given the many similarities with DRAW and PLNs, it is surprising that the related work section mentions neither. While the relation of PLNs to  DRAW is well explained in the introduction, PLNs are not mentioned in the paper at all, which is something the authors should correct.  Was the inference regularization technique from Section 2.3 used for any of the experiments?  Typos: Equations 11 and 12 both should have log in front for p(x|z).


Learning from Small Sample Sets by Combining Unsupervised Meta-Training with CNNs
This work explores CNNs for the recognition of novel categories from few examples. Inspired by the transferability properties of CNNs, we introduce an additional unsupervised meta-training stage that exposes multiple top layer units to a large amount of unlabeled real-world images. By encouraging these units to learn diverse sets of low-density separators across the unlabeled data, we capture a more generic, richer description of the visual world, which decouples these units from ties to a specific set of categories. We propose an unsupervised margin maximization that jointly estimates compact high-density regions and infers low-density separators. The low-density separator (LDS) modules can be plugged into any or all of the top layers of a standard CNN architecture. The resulting CNNs significantly improve the performance in scene classification, fine-grained recognition, and action recognition with small training samples.

The paper describes an method for improving the generalizability of features from pre-trained convolutional neural networks. It introduces a new low-density separator layer that is trained in an unsupervised manner to discriminate between quasi-classes on a very large dataset of unlabeled images. The quasi-classes are selected from the data by first selecting a small number of examples using the k-means++ heuristic, and then expanding these iteratively using linear support vector machines. Given the quasi-classes, which are assumed to correspond to high density regions in the feature space, low density separators are then fit by optimizing an objective designed to produce separators that separate regions with small intra-class distances and large inter-class distances. The experiments show that the resulting features give impressive results on several datasets when there are few labeled training examples, outperforming features from a vanilla pre-trained CNN and other state-of-the-art (DAG-CNN).
The problem of exploiting large datasets of unlabeled images to improve representations is widely acknowledged as being extremely important, and this work makes an important step in this direction. The approach in the paper still requires a CNN that has been pre-trained on a large set of labeled images, but shows that more generalizable representations can be extracted from such a network using a post-hoc unsupervised training process on a larger sets of unlabeled images.  The idea of automatically identifying and using pseudo-classes for unsupervised training on a very large dataset comes from the authors previous ECCV submission [17] given in the supplementary material, with the main delta being that the method presented here is more general, and it allows for fine tuning the full network with backprop when sufficient examples are available. Personally, I think the delta is sufficient, and the results on standard datasets when there are few labeled examples clearly demonstrate the effectiveness of the approach.  In general, I think that the problem being addressed is important, that the idea is novel, the paper is well written and clear, and the results are impressive. The supplementary material is a valuable addition.  Issues:  It wasn't clear to me from the paper how exactly recursive feature elimination was done: are the most active features selected using the YFCC100M dataset or the target dataset? Also, I would expect that PCA would work better than random projections: was this attempted too?  The paper uses the term "unsupervised pre-training" to refer to the process of identifying the low dimensional separators. I found this to be a little confusing, as the term is often used to refer to methods like stacked autoencoders and RBMs for initializing weights when training deep nets. Here, the pre-training actually happens after the network has undergone supervised training on a labeled dataset. Furthermore, the "pre-training" step is not always followed by fine-tuning (i.e. when only a very small number of examples are available, and the features are used off the shelf), which makes is even more misleading.  Including some information (perhaps in the supplementary material) on how long the training took would be useful.  In Figure 4 you give the classification accuracies for a single scale CNN.  Since the single scale LDS+CNN replaces the last CNN layer, it would be good to see the performance of the single scale CNN for features from both the last and penultimate layers in these figures (i.e. fc6 and fc7).  Corrections: - Line 100: "An unit" -> "A unit" - Line 207: "SVM's" -> "SVMs" - Line 286: "significant" -> "significantly" - Figure 3: "categary" -> "category"


Integrated Perception with Recurrent Multi-Task Neural Networks
Modern discriminative predictors have been shown to match natural intelligences in specific perceptual tasks in image classification, object and part detection, boundary extraction, etc. However, a major advantage that natural intelligences still have is that they work well for all perceptual problems together, solving them efficiently and coherently in an integrated manner. In order to capture some of these advantages in machine perception, we ask two questions: whether deep neural networks can learn universal image representations, useful not only for a single task but for all of them, and how the solutions to the different tasks can be integrated in this framework. We answer by proposing a new architecture, which we call multinet, in which not only deep image features are shared between tasks, but where tasks can interact in a recurrent manner by encoding the results of their analysis in a common shared representation of the data. In this manner, we show that the performance of individual tasks in standard benchmarks can be improved first by sharing features between them and then, more significantly, by integrating their solutions in the common representation.

This paper proposes multinet, a new integrated network in which not only deep image features are shared between tasks (representation sharing), but where tasks can interact in a recurrent manner by encoding the results of their analysis in a common shared representation of the data (output correlation). The proposed deep models are novel and seem technically sound, but the experimental evaluation may leave some room for improvement.
This paper is crystal clear and the main points are easily accessible. The key idea of integrated learning of representation sharing and output correlation is sound and well executed in the new architecture comprising CNNs, R-CNNs, RNNs and autoencoders.  My main concern is regarding the experimental evaluation. There is clear room for improvement: (1) the authors are encouraged to use the standard VOC 2012 dataset instead of the more obsolete VOC 2010/2007 datasets--this makes direct comparison of different methods possible; (2) the baseline methods (Independent and Multi-task in Table 1) are too simple to justify the effectiveness of the proposed method, and more recent work on multi-task deep learning should be compared. Note that, although this paper contrasts itself clearly from the literature, it does not mean that it is enough to evaluate the proposed method only against simple baselines.  The heterogeneous multi-task learning scenario is exactly the same with the state of the art work that won the MS COCO competition [8]. The multi-task network cascade (MNC) [8] is a sequential model that exploits the perception behavior of humans, i.e. from coarse-grained perception to fine-grained perception. I think MNC [8] is the most closely related method deserving empirical comparison (although MNC is sequential while the method of this paper is parallel).  Another baseline method could be MT-CNN [31], which addresses the different convergence rates of different networks in the multi-task learning setting. Extending the heterogeneous multi-task learning scenario [31] to the classification, localization, and part detection as this work is straight-forward (MT-CNN is also a parallel learning method as this work).  The final concern is on the model trainability. As multinet is a complex model comprising CNNs, R-CNNs, RNNs and autoencoders, I thus worry about the high difficulty in training the hybrid architecture. In particular, the autoencoders seem rather difficult: the back-propagation process (encoder) resorts to broadcast the outputs to the previous layers, which is quite ad hoc and requires more theoretical and empirical justification. I would expect to see what the authors can clarify in the rebuttal.  -- Update after rebuttal: The authors have provided some responses on why the relevant baselines I mentioned were not empirically evaluated. I think the responses are reasonable and thus keep my original rating unchanged.


Privacy Odometers and Filters: Pay-as-you-Go Composition
In this paper we initiate the study of adaptive composition in differential privacy when the length of the composition, and the privacy parameters themselves can be chosen adaptively, as a function of the outcome of previously run analyses. This case is much more delicate than the setting covered by existing composition theorems, in which the algorithms themselves can be chosen adaptively, but the privacy parameters must be fixed up front. Indeed, it isn't even clear how to define differential privacy in the adaptive parameter setting. We proceed by defining two objects which cover the two main use cases of composition theorems. A privacy filter is a stopping time rule that allows an analyst to halt a computation before his pre-specified privacy budget is exceeded. A privacy odometer allows the analyst to track realized privacy loss as he goes, without needing to pre-specify a privacy budget. We show that unlike the case in which privacy parameters are fixed, in the adaptive parameter setting, these two use cases are distinct. We show that there exist privacy filters with bounds comparable (up to constants) with existing privacy composition theorems. We also give a privacy odometer that nearly matches non-adaptive private composition theorems, but is sometimes worse by a small asymptotic factor. Moreover, we show that this is inherent, and that any valid privacy odometer in the adaptive parameter setting must lose this factor, which shows a formal separation between the filter and odometer use-cases.

This paper mainly considers adaptive composition theorem for differential privacy. The paper allows privacy parameters and number of rounds to be adaptive, and to solve the corresponding problem (for example, differential privacy is not defined suitably), authors propose two natural concepts, the privacy odometer and privacy filter. The privacy odometer can give an upper bound of privacy loss on every round, and the privacy filter can determine whether to continue or to halt the process, in the guarantee of global privacy budget. What’s more, authors prove the privacy filter has the same asymptotic order of bound as advanced composition theorem (C. Dwork 2010), while this cannot be achieved by the privacy odometer, except with an addition O(\sqrt{\log \log(n)}) factor.
In my personal view, the situation considered in this paper is much more general than almost all the existing work, and is more appropriate to use if differential privacy is applied in real word. Besides it, authors also give a deep understanding for the proposed primitives. These are main reasons for me to accept or to rate high scores for this paper, and I think it may open another door for differential privacy related research.  Intuitively speaking, giving an upper bound for privacy loss every round is more difficult than determining whether to continue or to halt the process, as we can easily obtain a privacy filter from a privacy odometer. Thus, I think conclusions in this paper are theoretical sound. A question I am concerned is whether we can obtain a privacy odometer from a privacy filter, for example, once we can solve equation (7) to put \epsilon_g in only one side, could we say the corresponding term in the other side can be used as a privacy odometer?   Some minor comments: 1. For the last word in line 86, I think authors want to say “Continue” not “Halt”; 2. In definition 2.6, there is a clerical error with the third case in randomized response; 3. In line 278 – 279, it will be better to explain more for the content in the brackets, as I am not clear why the left side of the inequality is an upper bound of statistical distance, and why this can conclude \epsilon_i has to be larger than \frac{1}{10n} (though I know this is a basic fact for many DP algorithms); 4. The format of paper’s references and supplementary is different with NIPS standard format.


Automated scalable segmentation of neurons from multispectral images
Reconstruction of neuroanatomy is a fundamental problem in neuroscience. Stochastic expression of colors in individual cells is a promising tool, although its use in the nervous system has been limited due to various sources of variability in expression. Moreover, the intermingled anatomy of neuronal trees is challenging for existing segmentation algorithms. Here, we propose a method to automate the segmentation of neurons in such (potentially pseudo-colored) images. The method uses spatio-color relations between the voxels, generates supervoxels to reduce the problem size by four orders of magnitude before the final segmentation, and is parallelizable over the supervoxels. To quantify performance and gain insight, we generate simulated images, where the noise level and characteristics, the density of expression, and the number of fluorophore types are variable. We also present segmentations of real Brainbow images of the mouse hippocampus, which reveal many of the dendritic segments.

The authors proposed a methdo to automatically segment neurons from multispectral images. The algorithm is based on a 5 steps pipeline which makes us of standard techniques from image processing (voxelization, watershed) and from machine learning (spectral clustering) in order to reconstruct the neurons shape.
After reading the author's rebuttal I have increased the technical quality to 2 and after reading the the other reviews I increased the potential impact to 3.  The authors replied to many questions but not to all, in particular the answer was not satisfactory to the question about the parameter K which is one of the crucial parameter in any segmentation algorithm. Why they did not provide the results using the suggested automatic method in Fig4 instead of cyclying on possible (wrong) number of clusters ? this will asses the generality of the method. I would have expected to see in the results the performances with at least one auto-tuning heuristic to asses its generality (at least the one suggested by the authors).  --------  The paper, despite being written nicely, suffer of some major lacks that makes it not acceptable for NIPS.   In the following the issues found in the paper:  1) In Eq(2) when constructing the adjecency matrix, the ranges of the distances d(...) and \delta(...) are the same ? otherwise is not possible to merge them into the same function otherwise undesired effects in the final affinity may occur.   2) In line 125, d(u,v) is the same as the one in Eq(2) and line 114 ? In the line 114 d(s) is a measure of heterogeneity, in line 125 of  distance and in Eq(2) of color distance.  3) Is not clear why the authors used clustering method that requires an input parameter K when other clustering algorithm exists and that do not require such parameter (i.e. dbscan, affinity propagation)  4) Since the pipeline uses the spectral clusterig algorithm, why the authors did not use the Spectral Gap heuristic to determine the K ?  5) In which sense the parameter K (line 143) has been manually estimated ? did you run the algorithm several time and use the K that produces the better results ?  6) Is not written which function has been used to compute d(s) (line 114). Authors just wrote an example of a possible function to be used but not the actual one.  7) In line 114, d(s) is the same as the one used in Eq(2) ?  8) Is completely arbitrary the number of "reliable" and "unreliable" neighbors because is not explained why the authors decided to use 0.5 and 0.05 (line 120 and 122).  9) The two threshold t_s and t_d how has been set ? is not written in the text.  10) In line 132, I suppose, that the equation e^-{\alpha d^2_{ij}}... \times (0.1 + 0.9 ...) is referred to the Eq.(2). If one expand that equation it results in 0.1e^{-\alpha d_{ij} +0.9e^-{\alpha d_{ij}}e^{\beta \delta{ij}} which is actually not consistent with Eq(2). Please explain.  Qualitatively the performances (in particular in Fig 3) looks very nice but there are too many details that are not clear.  Grammatically speaking, the paper is written nicely withour major nor minor typos.  In the following the motivations for the scores:  Technical quality 1: there are too many unclear things and math formulations that are ambiguous or wrong  Novelty/originality 2: because the work combine known methods from image processing and machine learning  Potential impact or usefulness 2: this work can have an important impact in the society to understand the shape of neurons but due to the issues in the methodological and technical parts in am not sure that it can be applied on different dataset than the one used in this paper. It seems to be higly customized.  Clarity and presentation 2: because the usage of the math notation is not correct and ambiguous.


The Multiscale Laplacian Graph Kernel
Many real world graphs, such as the graphs of molecules, exhibit structure at multiple different scales, but most existing kernels between graphs are either purely local or purely global in character. In contrast, by building a hierarchy of nested subgraphs, the Multiscale Laplacian Graph kernels (MLG kernels) that we define in this paper can account for structure at a range of different scales. At the heart of the MLG construction is another new graph kernel, called the Feature Space Laplacian Graph kernel (FLG kernel), which has the property that it can lift a base kernel defined on the vertices of two graphs to a kernel between the graphs. The MLG kernel applies such FLG kernels to subgraphs recursively. To make the MLG kernel computationally feasible, we also introduce a randomized projection procedure, similar to the Nyström method, but for RKHS operators.

The paper proposes a new graph kernel called the Multiscale Laplacian Graph (MLG) Kernel, which is itself based on the Feature space Laplacian Graph (FLG) kernel, still presented by the authors. The MLP graph kernel relies on the graph spectral decomposition, having the advantage to account for learning similarities at multiple different scales, considering both the overall and local structures. The mathematical formulations and intuitive steps leading to the novel approach are well presented in the paper. Experiments are also conducted for comparison with the state of the art. The MLG outperforms existing methods in 4 out of 6 datasets analysed, and it is competitive in the remaining 2.
---------------- Overall Opinion ----------------  The paper is overall well written, and the methods are technically well presented. My main concerns are relative to the missing intuition and unclear explanations in some parts of the paper, as well as to provide more details in the experimental section. More specific information are given in the comments below.     ------- Novelty -------  The authors propose a new graph kernel, which appears to be competitive with the state of the art. I would overall consider the paper as based on a novel idea.     ----------------- Minor comments -----------------  Style and language issues:  - Line 20: "have been appeared" is formally incorrect, the authors could use (for instance) "have appeared" or "have been proposed" - Line 46: the meaning of "it can take structure into account at multiple different scales" is not fully clear, possibly the authors could correct with "it can take into account structures at multiple different scales" - Line 218: the neighbourhoods ... "have" to be defined, rather than "are".   References:  - Reference [10] is poorly written: rather than "jmlr" a clear statement as in reference [5] would be preferable.     ----------------- Major comments -----------------  Include unclear definitions, notation, technical incongruences and poor intuitive explanations.    Section 2:  - It would be nice to better clarify the notation, in particular regarding graphs, before describing the method. For instance: are the words node's attributes, features and labels used with the same meaning? How are these exactly defined?  - Lines 81, 84 and 95: the "low eigenvalue eigenvectors" is unclear, one could just say the "low eigenvalues" or "the low eigenvalues with corresponding eigenvectors". This might have been considered as minor concern, but since is often repeated it makes its meaning unclear; I therefore consider it as a major comment.   - Line 149-152: the sentence is not clear. Though the mathematics formulation well defines the differences between the two kernels, some more effort should be done here to better explain the intuition behind.    Section 3:  - It would be nice to give more intuitive explanation on what the "level L" represents and how this affects the algorithm. Possibly, also the similarity with the WL kernel should be highlighted.    Section 4:  - The pseudocode algorithm is provided in the Supplementary Materials. I would suggest to include it in the main paper, as it provides the reader with an intuitive and detailed description of the method proposed.    Section 5:  - Are the authors using a stratified cross validation? If this is the case, it should be explicitly stated in the paper, otherwise it is advised to repeat the experiments with a stratified approach to avoid overfitting.  - It would be nice to include the FLG kernel, defined by the author themselves, as a further baseline comparison. This would help to strengthen the real advantages of the proposed MLG kernel.     ----------- Next Steps -----------  I would suggest to revise the paper according to the comments and modifications that I stated above. After that, I would consider the paper to be at an appropriate level for a NIPS publication.