Online Influence Maximization under Independent Cascade Model with Semi-Bandit Feedback
We study the online influence maximization problem in social networks under the independent cascade model. Specifically, we aim to learn the set of "best influencers" in a social network online while repeatedly interacting with it. We address the challenges of (i) combinatorial action space, since the number of feasible influencer sets grows exponentially with the maximum number of influencers, and (ii) limited feedback, since only the influenced portion of the network is observed. Under a stochastic semi-bandit feedback, we propose and analyze IMLinUCB, a computationally efficient UCB-based algorithm. Our bounds on the cumulative regret are polynomial in all quantities of interest, achieve near-optimal dependence on the number of interactions and reflect the topology of the network and the activation probabilities of its edges, thereby giving insights on the problem complexity. To the best of our knowledge, these are the first such results. Our experiments show that in several representative graph topologies, the regret of IMLinUCB scales as suggested by our upper bounds. IMLinUCB permits linear generalization and thus is both statistically and computationally suitable for large-scale problems. Our experiments also show that IMLinUCB with linear generalization can lead to low regret in real-world online influence maximization.

The paper studies the following bandit model: actions are nodes in a known directed graph and at each time step t the following happens: (1) hidden from the player, each edge (i,j) is independently declared "open" or "closed" with fixed but unknown probability w(i,j); (2) the player chooses a subset S_t of nodes of cardinality at most K; (3) all directed paths originated from nodes in S_t that go through open edges are revealed to the player; (4) the player's reward is the number of nodes in these paths. The regret of the player is measured with respect to the expected reward obtained by consistently playing the K-sized subset S of nodes that maximizes the expected reward. Since this set is NP-hard to compute, but easy to approximate, it is assumed that the player has access to an oracle that, given G and estimates of w(i,j) for each edge (i,j), returns the K-sized subset S_t that maximizes the expected reward according to the estimates. The probabilities w(i,j) are assumed to follow a linear model, w(i,j) = x(i,j)*theta, where x(i,j) is the known d-dimensional feature vector associated with edge (i,j) and theta is an unknown parameter vector. A special case is the "tabular model" where all the feature vectors are orthogonal. The paper proposes a natural variant of LinUCB to learn the vector theta using the oracle to compute the maximizing subset. The regret scales with dC_*\sqrt{|E|T} where d is the dimension of theta, E is the edge set, and C_* is a parameter that depends on the interaction between G and w. The experimental section is properly done.
The paper builds on a series of works that study influence maximization in the bandit model. The main new ideas are the linear model for probabilities, which provides a key to scalability, and the introduction of C_* as a natural scaling parameter for the regret. The results are not extremely strong, but they are nevertheless interesting and technically solid.
The parameter sigma plays no role and could perhaps be omitted from the analysis.


Dualing GANs
Generative adversarial nets (GANs) are a promising technique for modeling a distribution from samples. It is however well known that GAN training suffers from instability due to the nature of its saddle point formulation. In this paper, we explore ways to tackle the instability problem by dualizing the discriminator. We start from linear discriminators in which case conjugate duality provides a mechanism to reformulate the saddle point objective into a maximization problem, such that both the generator and the discriminator of this 'dualing GAN' act in concert. We then demonstrate how to extend this intuition to non-linear formulations. For GANs with linear discriminators our approach is able to remove the instability in training, while for GANs with nonlinear discriminators our approach provides an alternative to the commonly used GAN training algorithm.

This submission proposes an alternative view on GAN training by replacing the inner learning of the discriminator by its dual program. In the case of linear discriminators this leads to a stable and simpler maximization problem. The authors propose strategies to cope with non-linear discriminators, by either taking linear approximations to the cost function, or to the scoring function (typically the CNN in the case of image GANs).
PROS:
- The paper proposes an alternative and simple view on GAN training.
- The paper reads well and the formalism is well chosen.
- The proposed algorithm (with scoring function linearization) leads to interesting initial results on CIFAR and MNIST.
CONS:
- The proposed approach requires defining a dual variable for each sample z_i. Therefore, the dual program needs to be solved to satisfactory precision at each step.
- If I am not mistaken, Claim 1 is simply the dual of a binary logistic linear classification problem. Then, it might be good to present it that way, and avoid calling that a "claim" as this is a classical result.
- From a purely formatting point of view, the graphical presentation of results could be improved, as it is hard to read. Moreover, only several runs, and the corresponding learning curves are given. Would it be possible to give other quantitative measures? How often does GAN training diverge? Is the proposed algorithm helping that metric?
Overall, I think that this paper gives a simple view on GAN training using the dual of the inner problem. I enjoyed reading the paper and both the formal and experimental results seem correct. Because of this I lean towards acceptance - however, since I am not a pure GAN expert, I may have misevaluated the importance of this work.


Causal Effect Inference with Deep Latent-Variable Models
Learning individual-level causal effects from observational data, such as inferring the most effective medication for a specific patient, is a problem of growing importance for policy makers. The most important aspect of inferring causal effects from observational data is the handling of confounders, factors that affect both an intervention and its outcome. A carefully designed observational study attempts to measure all important confounders. However, even if one does not have direct access to all confounders, there may exist noisy and uncertain measurement of proxies for confounders. We build on recent advances in latent variable modeling to simultaneously estimate the unknown latent space summarizing the confounders and the causal effect. Our method is based on Variational Autoencoders (VAE) which follow the causal structure of inference with proxies. We show our method is significantly more robust than existing methods, and matches the state-of-the-art on previous benchmarks focused on individual treatment effects.

-	The authors utilized Variational Autoencoders (VAE) in order to estimate the individual treatment effect (ITE) and the average treatment effect (ATE) for the causal model in Figure 1. However, five other causal diagrams can also be considered with confounder proxies (see Figure 1 in [Miao et al., 2016]). It would be interesting to check whether the proposed method can be applied to all these causal diagrams or there are some limitations.
-	Please give some justifications why parametrizing the probability distributions in the form of Equations (2-4) would be a good choice for estimating ITE and ATE. For instance, why covariates of x_i are independent given z_i? Is it restricting the expressive power of this method?
-	In your method, do we need to know the dimension of latent variable z? For instance, in Experimental Results section, it is assumed that z is a 20-dimensional variable. Does the performance degrade if D_z is not adjusted correctly?
-	In Table 1, is there any good reason that BNN has a better performance than CEVAE?
-	When z is categorical, it seems that we can just estimate p(x_i|z_i) by your method and then estimate ITE and ATE from the results in [Kuroki and Pearl, 2011]. It would be interesting to compare the performance of this method with CAVAE?
-	Figure 3 shows that model misspecification may increase the sample complexity. It would be interesting to characterize the sample complexity overhead from model misspecification.
-	Under some conditions, can we have a theoretical guarantee that CEVAE gives a good approximation of ITE or ATE?
-	Some typos:
o	Line 126: p(Z,Xt,y)->p(Z,X,t,y)
o	In equation (2), please define D_x and D_z.


Neural Networks for Efficient Bayesian Decoding of Natural Images from Retinal Neurons
Decoding sensory stimuli from neural signals can be used to reveal how we sense our physical environment, and is valuable for the design of brain-machine interfaces. However, existing linear techniques for neural decoding may not fully reveal or exploit the fidelity of the neural signal. Here we develop a new approximate Bayesian method for decoding natural images from the spiking activity of populations of retinal ganglion cells (RGCs). We sidestep known computational challenges with Bayesian inference by exploiting artificial neural networks developed for computer vision, enabling fast nonlinear decoding that incorporates natural scene statistics implicitly. We use a decoder architecture that first linearly reconstructs an image from RGC spikes, then applies a convolutional autoencoder to enhance the image. The resulting decoder, trained on natural images and simulated neural responses, significantly outperforms linear decoding, as well as simple point-wise nonlinear decoding. These results provide a tool for the assessment and optimization of retinal prosthesis technologies, and reveal that the retina may provide a more accurate representation of the visual scene than previously appreciated.

The paper describes a method to decode natural images from retinal-like activities, using convolutional neural networks. The retinal-like activities are generated by a constructed lattice of linear-nonlinear-Poisson models (separately fitted to RGC responses to natural scenes in a macaque retina preparation) in response to natural static images. After a simple linear decoding of the images from the retinal-like activities, a convolutional neural network further improves on the reconstruction of the original natural images. The paper is clearly written and the results seem sound. A few comments to clarify the motivation, assumptions and impact of the work:
-the method proposed is compared to a linear decoder and shown to perform substantially better. However, the performance of the decoding stage will most likely depend on the performance of the encoding stage. A more explicit discussion on the fact that the results of the CAE will ultimately depend on the encoding would clarify the scope and limitations of the study;
-on a related note, the authors should be explicit about the reasons for choosing a synthetic encoding stage: why didn’t the authors directly fit their method to the data without the synthetic encoding stage? Too few cells recorded? Too few natural images for fitting the CAE? An attempt at excluding the temporal component of the cells responses, so that the problem reduces to decoding static natural images instead of movies (which would be a more complex task)? This should be more explicitly stated;
-in Macintosh et al. 2017 (https://arxiv.org/abs/1702.01825), a convolutional neural network is shown to be a good encoding model of the RGCs in response to natural movies. The authors should discuss their method in the context of such study;
-a plot illustrating how the performance of the CAE decoder depends on the amount of data used for training the network would be instrumental for assessing how powerful and practical the approach is;
-the authors show that the CAE decoder is able to capture phase structure, unlike the linear decoder. Is the CAE better than the linear decoder mostly because it captures the phase structure? How does the performance between CAE and linear decoder compare in phase scrambled images? It would be helpful to comment on this, and perhaps explicitly plot such comparison.
Some typos/minor comments:
-lines 83-84, the authors write “resulting response of each neuron…displayed in Figure 3”, however no neural responses are displayed in Figure 3;
-line 104, the authors write “all tuned through an exhaustive grid-search”. Providing more details about this tuning would improve the clarity of the paper, and allow for a better assessment of the usability of the method presented;
-line 112, the authors write “i = {1,2,…,n}”. However, ’n’ is the number of neurons. Is ’n’ also the number of training epochs?
-- After the rebuttal --
The rebuttal has addressed satisfactorily all my concerns/comments, and therefore I would like to increase my score to a clear accept (score: 8).


Online Reinforcement Learning in Stochastic Games
We study online reinforcement learning in average-reward stochastic games (SGs). An SG models a two-player zero-sum game in a Markov environment, where state transitions and one-step payoffs are determined simultaneously by a learner and an adversary. We propose the UCSG algorithm that achieves a sublinear regret compared to the game value when competing with an arbitrary opponent. This result improves previous ones under the same setting. The regret bound has a dependency on the diameter, which is an intrinsic value related to the mixing property of SGs. If we let the opponent play an optimistic best response to the learner, UCSG finds an ε-maximin stationary policy with a sample complexity of O (poly(1/ε)), where ε is the gap to the best policy.

The paper considers the problem of online learning in two-player zero-sum stochastic games. The main result is constructing a strategy for player 1 that guarantees that the cumulative rewards will never go below the maximin value of the game by more than a certain bound, no matter what strategy the other player follows. The bound is shown to grow sublinearly in the number of rounds T of the game, and polynomially on other problem parameters such as the diameter, the size of the state and action spaces. The results imply that the proposed algorithm can be used in self-play to compute near-maximin strategies for both players. The algorithm and the analysis are largely based on the UCRL algorithm of Auer and Ortner (2007) and the analysis thereof.
I have reviewed a previous version of the paper for ICML'17, and have to say that the quality of the writing has improved quite a bit; the authors obviously invested a lot of effort into improving the readability of the paper. The paper is of course still very technical with most of the analysis details deferred to the appendix. Being already familiar with the analysis, I cannot objectively judge how easy it is to follow for first-time readers (I believe that it still looks rather intimidating), but I am now entirely convinced that the proofs are correct. Since the previous round of reviews, the authors fixed all the technical issues I've pointed out, so at this point, I have no technical objections.
The contribution itself is novel and interesting: the paper proposes a new (but very natural) regret definition suitable for stochastic games and proposes an efficient algorithm that comes with strong guarantees on the proposed regret notion. The fact that such strong results are actually possible was quite surprising to me.
All in all, I think that this paper is ready to be published.
Detailed comments
=================
-definition of \rho^* after line 83: at this point, the existence of the limit is not ensured. Please at least say that this requires some mild assumptions that will be specified later.
-112: Assumptions 1 and 2: Maybe it would be useful to use different letters for the two diameter notions to avoid confusion.
-154: "Player 1 optimistically picks one model"---Well, not entirely optimistically, as the choice of the opponent's policy is done conservatively. Please rephrase for clarity.
-186: This Lemma 4 is very informal; please state explicitly that this is not the full statement of the lemma. Also, the statement itself is written a bit strangely.
-189: "analyzing techniques" -> "analysis techniques"
-195: "our algorithm makes [...]" -> "our algorithm uses policy [...]"
-261: Please highlight the fact that \pi_i^2 is a non-stationary policy, and in general all policies discussed in this section can be non-stationary.


Thinking Fast and Slow with Deep Learning and Tree Search
Sequential decision making problems, such as structured prediction, robotic control, and game playing, require a combination of planning policies and generalisation of those plans. In this paper, we present Expert Iteration (EXIT), a novel reinforcement learning algorithm which decomposes the problem into separate planning and generalisation tasks. Planning new policies is performed by tree search, while a deep neural network generalises those plans. Subsequently, tree search is improved by using the neural network policy to guide search, increasing the strength of new plans. In contrast, standard deep Reinforcement Learning algorithms rely on a neural network not only to generalise plans, but to discover them too. We show that EXIT outperforms REINFORCE for training a neural network to play the board game Hex, and our final tree search agent, trained tabula rasa, defeats MOHEX 1.0, the most recent Olympiad Champion player to be publicly released.

SUMMARY:
The paper proposes an algorithm that combines imitation learning with tree search, which results in an apprentice learning from an ever-improving expert.
A DQN is trained to learn a policy derived from an MCTS agent, with the DQN providing generalisation to unseen states. It is also then used as feedback to improve the expert, which can then be used to retrain the DQN, and so on.
The paper makes two contributions: (1) a new target for imitation learning, which is empirically shown to outperform a previously-suggested target and which results in the apprentice learning a policy of equal strength to the expert. (2) using the apprentice to improve the expert after each imitation learning step is shown to result in strong play against a policy gradient algorithm for the game Hex.
COMMENTS:
I found the paper generally well-written, clear and easy to follow, barring Section 6. The idea of combining reactive-style approaches with explicit planning techniques is an interesting one, and makes sense intuitively (I particularly liked the link to the different systems of human thinking), although the downside here is that we then require a model of the environment, which negates the benefits of using model-free approaches (like DQN) in the first place.
A few questions that I had about the paper:
1. On line 69, some of the drawbacks of RL in general are stated. What guarantees, though, are there that these issues don't apply to ExIt as well?
2. On line 212, why is it advantageous or desirable for the positions in the training set to be similar to those in the test set?
3. I found section 6 lacking in detail, especially for a purely empirical paper. In particular, was the version of ExIt presented here the online version? On lines 122-125, some reasons are provided as to why an online variant may be superior, but there is no experiment validating this. I would therefore expect to see maybe an online version, and a version where the data is discarded after each iteration in the results graph, which could then show that ExIt is better than policy gradients, as well as the advantage (or not) of the online version.
4. I do not follow the results in section 6.2. What were the "reference agents" mentioned in the caption? Which network is being referred to on line 267?  How many games were played? What were the standard errors on those results? What is "NN evaluations"?
Some remarks on the actual writing:
1. For an arbitrary policy, the definition of the advantage function appears incorrect on line 50. It should be defined as A^\pi(s, a) = Q^\pi(s,a ) - V^\pi(s), where V\pi(s) = Q^\pi(s,\pi(s)) which is not necessarily equal to the max of one-step lookahead as is currently written.
2. In the pseudocode on line 80, there is a function called "Targets". Since it is only made clear two pages later what "Targets" actually refers to, it may be better to briefly mention its meaning here.
3. The exploration constant c_b in the UCB1 formula on line 162 is not explained. Also, was this set to zero because RAVE was used?
4. On line 232, should it not read "When a node is TRAVERSED"? The word "expanded" gives the impression that it is in the expansion phase of MCTS where a child is randomly added to the tree, as opposed to the tree phase where UCB1 selects the next child.
There were also a few minor typographical errors:
1. There are missing full stops on lines 101 and 171
2. The footnote on line 167 should appear after the full stop
3. The semicolon on line 187 should be a comma
4. Line 254: "by network trained" => "by a network trained"
5. In the footnote on line 176, "ADAM" has been capitalised, but in the supplementary material it is referred to as "adam"
6. The reference on line 340 has the first names of the authors listed, instead of their surnames.


Efficient Online Linear Optimization with Approximation Algorithms
We revisit the problem of online linear optimization in case the set of feasible actions is accessible through an approximated linear optimization oracle with a factor ↵ multiplicative approximation guarantee. This setting is in particular interesting since it captures natural online extensions of well-studied offline linear optimization problems which are NP-hard, yet admit efficient approximation algorithms. The goal here is to minimize the ↵-regret which is the natural extension of the standard regret in online learning to this setting. We present new algorithms with significantly improved oracle complexity for both the full information and bandit variants of the problem. Mainly, for both variants, we present ↵-regret bounds of
), were T is the number of prediction rounds, using only O(log(T )) calls to the approximation oracle per iteration, on average. These are the first results to obtain both average oracle complexity of O(log(T )) (or even poly-logarithmic in T ) and ↵-regret bound O(T c ) for a constant c > 0, for both variants.

This paper addresses the problem of online linear optimization when the feasible set is such that doing exact projections into it is NP-hard but there is an approximate algorithm for maximizing linear functions in the feasible set with a constant approximation ratio. This paper is an extension to "Playing games with approximation algorithms", by Kakade et al with a similar algorithm with substantially better computational complexity.
The main issue with the previous work (kakade et al)  addressed in this paper is the linear (in the length of the online prediction history) number of calls to the approximation algorithm required in each online prediction round. Needing to essentially remember the entire history and have ever more expensive computation in each iteration is prohibitive and doesn't match most other work in online learning.
This paper proposes a new infesible projection algorithm which given an input either returns its approximate projection into the feasible set or returns an infeasible point which is a convex combination of a known set of feasible points and is not too far from the input. Using this infeasible projection algorithm it's possible to make a stochastic online optimization algorithm which has better complexity (roughly because projecting a point which is near from an existing projection is easy).
It is unclear what is applicability of this algorithm, however, as well as what would be the interest of it to the community. This seems to be of at most theoretical interest, and the paper should provide better justification of why this result is useful if it is to belong in nips.


Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon
How to develop slim and accurate deep neural networks has become crucial for realworld applications, especially for those employed in embedded systems. Though previous work along this research line has shown some promising results, most existing methods either fail to significantly compress a well-trained deep network or require a heavy retraining process for the pruned deep network to re-boost its prediction performance. In this paper, we propose a new layer-wise pruning method for deep neural networks. In our proposed method, parameters of each individual layer are pruned independently based on second order derivatives of a layer-wise error function with respect to the corresponding parameters. We prove that the final prediction performance drop after pruning is bounded by a linear combination of the reconstructed errors caused at each layer. By controlling layer-wise errors properly, one only needs to perform a light retraining process on the pruned network to resume its original prediction performance. We conduct extensive experiments on benchmark datasets to demonstrate the effectiveness of our pruning method compared with several state-of-the-art baseline methods. Codes of our work are released at: https://github.com/csyhhu/L-OBS.

Summary:
This paper adapts Optimal Brain Surgeon (OBS) method to a local version, and modified the objective function to be the target activation per each layer. Similar to OBS, it uses an approximation to compute Hessian inverse by running through the dataset once. Compare to prior methods, it finishes compression with much less retraining iterations. A theoretical bound on the total error based on local reconstruction error is provided.
Pros:
- The paper explores a local version of OBS and shows effectiveness of proposed method in terms of less time cost for retraining the pruned network.
- Leveraging local second order information is a much clever way than solving/retraining a network through brute force.
- The proposed method achieves good compression ratio without losing performance, with much less retraining iterations.
Comments:
- I recommend the authors to include the basics of Optimal Brain Surgeon (OBS) as a background section. This will make the flow of the paper much better. Many of the techniques used later (e.g. Lagrange multiplier of the MSE error function, iterative algorithm for computing the Hessian inverse) should be part of the background.
- The formulation of layer-wise objective is difficult to understand. Z^l is also a function of Theta^l, and since Z^l = Y^(l-1) W^l, then E^l is actually not even a function of Theta^l, since both terms cancel out in Z^l and hat(Z)^l. This is another reason why the - paper should have included OBS in the background, since here it uses Z^l as a local target activation value, and no longer treated as a function of Theta^l. I had read section 3.2 over many times to realize this point.
- Line 129: “For a well-trained network, gradient w.r.t. Parameters is small enough to be ignored as well”. This is not true. For the original OBS, it is true since the error is the total loss function of the network. For a local MSE function, the gradient of E^l wrt. Theta^l is literally zero (not just small enough).
- I recommend the authors move the proof Theorem 3.2 to the Appendix. This bound is good to know (that the total error is bounded by each local errors) but not very useful and hurts the flow of the paper.
- I recommend the authors edit section 3.4 into algorithmic format. It is hard to read an algorithm in a paragraph. Same for section “prune convolutional layer”.
- Line 191: Again, I don’t understand why “well-trained” is part of the clause here. For any function, if you perturb the input a little bit, the output will be quite close. In the original OBS, it is the final output vs. the target, so “well-trained” is necessary. But here, the target is itself.
- The paper gives some asymptotic bound on time complexities, but I am still curious of the actual wall clock timing compared to related work (DNS, Net-Trim), especially the time cost to compute the Hessian inverse.
- The paper claims that Net-Trim has memory limitation. But maybe it is worth trying a mini-batch version of Net-Trim?
- The paper puts ResNet-101 as future work. It would have been better if the authors can give preliminary results to give insights on how expensive it is to compute Hessian inverse on a deeper network like this. Based on the time complexity, I don’t see why running ResNet-101 with L-OBS is an issue.
- The model achieves good results with much less number of retraining iterations. However, in the AlexNet experiment, it is still losing to DNS in terms of compression ratio. Although, the paper claims to have faster compression time, one major use case for network compression/sparsification deployment on embedded systems, in which the compression ratio is more of a concern than time cost to retrain the network. For completeness, I also recommend the authors to implement DNS and report numbers on VGG-16.
Overall, I feel the paper has some good contribution, but the paper clarity needs to be improved by a lot (Background section, local error function formulation). Some experiments need to be clarified (e.g. wall clock time comparison), and completed (DNS numbers on VGG). Based on these points, I recommend marginal accept, conditioned on a major revision.
After rebuttal note: I am glad that DNS numbers on VGG and ResNet is provided. After seeing the rebuttal I decided to upvote my rating to 7. I hope the authors will reorganize the paper as promised in the rebuttal.


Time-dependent spatially varying graphical models, with application to brain fMRI data analysis
In this work, we present an additive model for space-time data that splits the data into a temporally correlated component and a spatially correlated component. We model the spatially correlated portion using a time-varying Gaussian graphical model. Under assumptions on the smoothness of changes in covariance matrices, we derive strong single sample convergence results, confirming our ability to estimate meaningful graphical structures as they evolve over time. We apply our methodology to the discovery of time-varying spatial structures in human brain fMRI signals.

This paper studied the graphical structure in spatiotemporal data through transferring the time series data into an additive model, by assuming stationary temporal correlation structure and time-varying undirected Gaussian graphical model for spatial correlation structure. With the assumption that the spatial correlations change smoothly with time, they proposed estimators for both spatial and temporal structures based on kernel method and GLasso approach. The statistical convergence property of the estimators was provided under certain assumptions. The approach presented good performance in both simulation and fMRI data application studies.
This paper is overall clearly written, with solid theoretical support and interesting application. However, some assumptions, such as a common temporal correlation matrix for different spatial locations, may be not valid in application. It may worth further discussion. There are also some other points may need further improvement.
1.	The authors didn’t provide correct supplementary, but another version of the main text.
2.	In the estimation section, tr(A) is supposed to be known. This is not so obviously satisfied in real data application. It is better to provide more discussions on this issue.
3.	Line 178 is unclear. Which lemma? It should be the inverse of B, instead of B itself, exploiting sparsity.
4.	As m / n increases, the bound in Theorem 2 / Theorem 4 converges. However, the regularization parameter \lambda_m / \lambda_n also converges to 0 at the same time, which is a sacrifice of sparsity. In simulation, the authors used a variety of regularization parameters, instead of the proposed one in the theorem. More discussions are expected on this issue.
5.	In simulation, it is recommended to compare with other methods, such as Qiu (2016). Also the authors mentioned that some previous methods were limited to AR(1) model, while their method in this paper can handle more complex models. But they only provide simulation results for AR(1) case, as shown in Fig. 2 & 3. So it is expected to see the simulation results on more general models.
6.	In simulation, it would be better to use the inverse of B(t) instead of B(t) for evaluation, as the focus is the structure of the underlying undirected graphs instead of the edge weights / strength of the undirected graphs.
7.	The assumption that all spatial points follow the same temporal structure is a bit strong. In application, its validity should be discussed. For example, in the fMRI data, different brain regions may present different developing pattern. The visual cortex may develop fast in childhood, while the dorsal prefrontal cortex related to decision and attention usually has a development peak in a later age. It would be better to check the validity of these assumptions before application.
8.	Assuming a time series follows a joint normal distribution (equation (2)) is also not very common. In the fMRI data application, its validity should also be discussed. Each of the 90 regions of interest generally has hundreds to thousands voxels, which should enable the test for normality.
9.	In Line 118, it assumes that spatial structure is sparse and m much larger than n. However, the brain connectivity is usually not sparse. In the meantime, in this resting-state fMRI dataset, some subjects has less time points (m larger than or equal to 76) than the number of ROIs (n = 90).
10.	In the application, the dataset include both healthy group and ADHD group. Did the authors use them all together for analysis? It might be better to apply the method to each group separately and compare the results. Different developing trends are expected. And if such a result is obtained, it should be more powerful to illustrate the effectiveness of this method.
11.	In consistency statements for ‘signal’ and ‘noise’, in sense of their correspondence with ‘temporal’ and ‘spatial’ structure. In abstract (line 3-4), they mentioned ‘temporally correlated signal’ and ‘spatially correlated noise’. However, in 2.1 estimator section (line 133 – 134), they mentioned ‘spatial signal variance’ and ‘noise variance … corresponding to tr(A)’, while A corresponds to temporal structure as indicated in equation (2).
12.	In the paragraph from line 44 to 51, it is better to add comparison with Qiu (2016) and Monti (2014), in sense of methodology.
13.	Some minor points: (i) It is better to provide the definition of e_i after equation (4), as it is provided after equation (5). (ii) The subscripts in equation (8) are not consistent, where ‘t’ should be ‘i’. (iii) In line 170, B to B(t). (iv) In line196, m to n.
Update: I am satisfied with the authors' replies to my comments. Hence I would like to raise my grade.


A Sample Complexity Measure with Applications to Learning Optimal Auctions
We introduce a new sample complexity measure, which we refer to as split-sample growth rate. For any hypothesis H and for any sample S of size m, the splitsample growth rateτ H (m) counts how many different hypotheses can empirical risk minimization output on any sub-sample of S of size m/2. We show that the expected generalization error is upper bounded by O log(τ H (2m)) m . Our result is enabled by a strengthening of the Rademacher complexity analysis of the expected generalization error. We show that this sample complexity measure, greatly simplifies the analysis of the sample complexity of optimal auction design, for many auction classes studied in the literature. Their sample complexity can be derived solely by noticing that in these auction classes, ERM on any sample or sub-sample will pick parameters that are equal to one of the points in the sample.

Summary:
Rademacher complexity is a powerful tool for producing generalization guarantees. The Rademacher complexity of a class H on a sample S is roughly the expected maximum gap between training and test performance if S were randomly partitioned into training and test sets, and we took the maximum over all h in H (eqn 11).  This paper makes the observation that the core steps in the standard generalization bound proof will still go through if instead of taking the max over all h in H, you only look at the h's that your procedure can possibly output when given half of a double sample (Lemma 1).  While unfortunately the usual final (or initial) high-probability step in this argument does not seem to go through directly, the paper shows (Theorem 2) that one can nonetheless get a useful generation bound from this using other means.  The paper then shows how this generalization bound yields good sample complexity guarantees for a number of natural auction classes.  In several cases, this improves over the best prior guarantees known, and also simplifies their analysis.
Evaluation:
While the core idea is to some extent an observation, I like the paper because it is a nice, cute idea that one could actually teach in a class.  And it allows for a simpler sample complexity analysis for auctions, which is great.  It may also have other uses too.  On the negative side, I wish the paper were able to answer whether the linear dependence on 1/delta is really necessary (see below).  That would make the paper feel more complete to me.  Still, I am overall positive.
Questions:
Do you know if the linear dependence on delta in Theorem 2 is required?  I see why you need it in your proof (due to using Markov) and you explain nicely why the usual use of McDiarmid to get an O(log(1/delta)) bound doesn't go through.  But this leaves open whether (a) there is a different way to get an O(log(1/delta)) bound, or on the other hand (b) there exist D,H for which you indeed have a more heavy-tailed chance of failure.   If this were resolved, the paper would feel more complete.
On a related note, do you know if you can get a bound along the lines of Theorem 2 in terms of the maximum # of hypotheses that can be produced from half of the *actual* sample S? (That is, replacing the sup over S of size m in the definition of \hat{\tau} with the actual training sample S?)
Suggestions:
- If I am not mistaken, The proof of Lemma 1 basically goes through the textbook Rademacher bound proof, changing it where needed to replace H with \hat{H}_{S \union S'}.  That's fine, but you should explicitly tell the reader that that is what you are doing up front, so they know what's new vs what's old.  E.g., you have a nice discussion of this form for Theorem 2, where you talk about where it differs from the usual argument, that I think is quite useful.
- In Section 4 you examine several interesting scenarios (single bidder, multiple iid regular,...) , and for each one you say "In this case, the space of hypotheses H is ...".  I think what you mean to say is more that "In this case, it is known that an *optimal auction belongs to* the hypothesis class H of ...".  In other words, the generalization guarantees apply to any H you want, but now you are saying what the meaningful H is in each scenario, right?  I think this is worth saying explicitly, since otherwise the reader might get confused why H is changing each time.
Also I'll point out that the idea of counting the number of possible outputs of an auction mechanism in order to get a sample complexity bound has been used in the auction literature before. The first I know of is the FOCS 2005 paper "Mechanism design via machine learning" by Balcan, Blum, Hartline, and Mansour, though that paper operates in a transductive setting where an argument of this type is much easier to make.


Poincaré Embeddings for Learning Hierarchical Representations
Representation learning has become an invaluable approach for learning from symbolic data such as text and graphs. However, state-of-the-art embedding methods typically do not account for latent hierarchical structures which are characteristic for many complex symbolic datasets. In this work, we introduce a new approach for learning hierarchical representations of symbolic data by embedding them into hyperbolic space -or more precisely into an n-dimensional Poincaré ball. Due to the underlying hyperbolic geometry, this allows us to learn parsimonious representations of symbolic data by simultaneously capturing hierarchy and similarity. We present an efficient algorithm to learn the embeddings based on Riemannian optimization and show experimentally that Poincaré embeddings can outperform Euclidean embeddings significantly on data with latent hierarchies, both in terms of representation capacity and in terms of generalization ability.

Summary
=======
  The paper proposes a link prediction model that embeds symbols in a
  hyperbolic space using Poincaré embeddings. In this space, tree
  structures can more easily be represented as the distance to points
  increases exponentially w.r.t. Euclidean distance. The paper is
  motivated and written well. Furthermore, the presented method is
  intriguing and I believe it will have a notable impact on link
  prediction research. My concerns are regarding the comparison to
  state-of-the-art link prediction and how the method performs if the
  assumption about a hierarchy in the data is dropped.
Strengths
=========
  - Impressive results for embedding hierarchies in a non-Euclidean
    space using a much smaller dimension.
Weaknesses
==========
  - Maybe I am missing a point, but why is there no comparison with
    state-of-the-art neural link prediction models (e.g. HolE/ComplEx)?
    This would strengthen the experimental evaluation a lot as TransE
    has consistently be outperformed by these methods. Furthermore, how
    does Poincaré compare to TransE in terms of runtime?
  - I would have liked to see more experimental results of the
    limitations of the proposed method. Given that many real-world
    datasets have less pronounced hierarchies than WordNet, it would
    have been nice to show what happens if a hierarchy of symbols cannot
    be assumed for a large portion of the data. For instance, how would
    the method perform on FB15k? Generally, I would like to see a more
    thorough discussion about when Poincaré embeddings can be used in
    practice and when not. For instance, the authors mention word2vec,
    GloVe and FastText in the introduction and I guess that Poincaré
    embeddings cannot serve as drop-in replacement for word vectors in
    these models as we don't know how to perform certain translations in
    hyperbolic space or how to apply backprop? Such limitations should
    be clarified in the paper.
Minor Comments
==============
  - Embedding hierarchies in Euclidean has been attempted in prior work
    that should be mentioned:
    - For textual entailment and images: Ivan Vendrov, Ryan Kiros, Sanja
      Fidler, Raquel Urtasun. Order-Embeddings of Images and
      Language. ICLR 2016
    - For link prediction: Thomas Demeester, Tim Rocktäschel, Sebastian
      Riedel. Lifted Rule Injection for Relation Embeddings. EMNLP 2016
  - Footnote 2: "would be more more difficult" -> "would be more
    difficult"
  - L277: "representations whats allow us to" -> "representations that
    allow us to"


VAE Learning via Stein Variational Gradient Descent
A new method for learning variational autoencoders (VAEs) is developed, based on Stein variational gradient descent. A key advantage of this approach is that one need not make parametric assumptions about the form of the encoder distribution. Performance is further enhanced by integrating the proposed encoder with importance sampling. Excellent performance is demonstrated across multiple unsupervised and semi-supervised problems, including semi-supervised analysis of the ImageNet data, demonstrating the scalability of the model to large datasets.

%%% UPDATE: Thank you for your response, which has been read %%%
%%% The score I gave has increased by one level, after I was assured that this work is of some interest to research on VAE %%%
%%% Please ensure that ALL details needed to reproduce the experiments are included in any revision of the paper %%%
This paper contains a lot of different ideas around variational auto-encoders and Stein's method, which are brought together in a way that appears novel. However I feel that the presentation could be improved and more details about the (complicated) implementation should have been included.
- The background on variational auto-encoders was not clear, to this reviewer who has not worked with VAEs. For instance, first p(x | z_n ; \theta) is called the "encoder", then q(z | x ; \phi) is called the "encoder" - which is it, and why is it given this name?
- \top should be used as transpose in latex.
- What is the domain of \theta? Related, how is \grad_\theta \psi(\theta ; D) defined if both \theta and \psi are vectors?
- Several approximations are composed en route to the final method that is proposed. It is unclear how all these approximations interact - this isn't assessed theoretically - so that all of the assessment must be based on the empirical results.
- Unfortunately several details necessary to reproduce the results seem to have been omitted. e.g. the value of \epsilon (step size for SVGD), the number of SVGD iterations (or the stopping criterion), the initial locations of the particles, etc. In particular, the initial location of the particles would presumably be quite influential to the performance of SVGD.
- I did not understand Theorem 2. The claim seems to be that KL_{q,p}^k is involved in a tighter variational lower bound, but what is the significance of the fact that Lim_{k \rightarrow \infty} KL_{q,p}^k(\Theta ; D) = 0? This is not really explained.
- On line 178, "q_0" has not been defined, I think.
- In general there are a lot of moving parts in the proposed method, and most of these "levers" are unexplored in the experiments. For instance, how does the choice of k impact on performance?
- An MCMC benchmark is used for a couple of the experiments - there is presumably an implicit argument that the proposed method is faster than MCMC. However, this isn't actually tested and computational times against MCMC are not included.


Learning Active Learning from Data
In this paper, we suggest a novel data-driven approach to active learning (AL). The key idea is to train a regressor that predicts the expected error reduction for a candidate sample in a particular learning state. By formulating the query selection procedure as a regression problem we are not restricted to working with existing AL heuristics; instead, we learn strategies based on experience from previous AL outcomes. We show that a strategy can be learnt either from simple synthetic 2D datasets or from a subset of domain-specific data. Our method yields strategies that work well on real data from a wide range of domains.

Key insights/novelty:
* Trains a regression model that predicts improvement in generalisation error
* To the best of my knowledge this is an original approach to this problem (although similar, but sufficiently different, methods do exist)
Questions:
* In the experimental section, I recommend that the authors also include experiments on rotated checkerboards - the discriminating features of checkerboards are aligned to the axes, and using random forest classifiers may yield an overly optimistic performance curve. Rotating at 45 degrees may yield overly pessimistic curve, but it will explain the utility of the proposed methods better.
* Baseline methods are not satisfactory since none of the proposed baselines can learn as much as the topology of the feature space as the proposed method (eg Zhu, Ghahramani, Lafferty ICML2003). At the very least this method should be compared in the discussions.
* I'm not convinced by the results since they don't seem to be significantly better than baselines. Particularly with uncertainty sampling on the real data. This point relates to my suggestion of rotating the checkerboard so that the discriminating features will not be axis-aligned.
* This method will require a relatively substantial proportion of data to be labelled to begin with. While you say that only a couple of hours were spent in labeling for the offline experiments, this time is domain-specific. Are there any techniques that can be used to improve this?
* The conclusions of this paper were very short and underwhelming.
General Statements:
* Very well written, clear motivation. Easy to follow story.
Small typos/recommended corrections:
* L79. Unannotated data?
* Showing the relative accuracy only shows one side of the picture. It would be insightful to show the accuracies of both too. Practitioners will be interested in knowing whether it is normally with learning strategies that have a good initial generalisation that disimprove moving forward.
* Algorithm 1: unclear which datasets the classifiers are trained on. Recommend clarifying. And in line 14, please clarify. Do you mean $\{\delta_m: 1 \leq m \leq M\}$?
* Should a selection function be passed into algorithm 1 to clarify line 7?


On Structured Prediction Theory with Calibrated Convex Surrogate Losses
We provide novel theoretical insights on structured prediction in the context of efficient convex surrogate loss minimization with consistency guarantees. For any task loss, we construct a convex surrogate that can be optimized via stochastic gradient descent and we prove tight bounds on the so-called "calibration function" relating the excess surrogate risk to the actual risk. In contrast to prior related work, we carefully monitor the effect of the exponential number of classes in the learning guarantees as well as on the optimization complexity. As an interesting consequence, we formalize the intuition that some task losses make learning harder than others, and that the classical 0-1 loss is ill-suited for structured prediction.
class loss functions [3,39,48]. A large body of work specifically focuses on the related tasks of ranking [18,9,40] and ordinal regression [37]. Contributions. In this paper, we study consistent convex surrogate losses specifically in the context of an exponential number of classes. We argue that even while being consistent, a convex surrogate might not allow efficient learning. As a concrete example, Ciliberto et al. [10] recently proposed a consistent approach to structured prediction, but the constant in their generalization error bound can be exponentially large as we explain in Section 5. There are two possible sources of difficulties from the optimization perspective: to reach adequate accuracy on the task loss, one might need to optimize a surrogate loss to exponentially small accuracy; or to reach adequate accuracy on the surrogate loss, one might need an exponential number of algorithm steps because of exponentially large constants in the convergence rate. We propose a theoretical framework that jointly tackles these two aspects and allows to judge the feasibility of efficient learning. In particular, we construct a calibration function [43], i.e., a function setting the relationship between accuracy on the surrogate and task losses, and normalize it by the means of convergence rate of an optimization algorithm.
Aiming for the simplest possible application of our framework, we propose a family of convex surrogates that are consistent for any given task loss and can be optimized using stochastic gradient descent. For a special case of our family (quadratic surrogate), we provide a complete analysis including general lower and upper bounds on the calibration function for any task loss, with exact values for the 0-1, block 0-1 and Hamming losses. We observe that to have a tractable learning algorithm, one needs both a structured loss (not the 0-1 loss) and appropriate constraints on the predictor, e.g., in the form of linear constraints for the score vector functions. Our framework also indicates that in some cases it might be beneficial to use non-consistent surrogates. In particular, a non-consistent surrogate might allow optimization only up to specific accuracy, but exponentially faster than a consistent one.
We introduce the structured prediction setting suitable for studying consistency in Sections 2 and 3. We analyze the calibration function for the quadratic surrogate loss in Section 4. We review the related works in Section 5 and conclude in Section 6.

The paper examines consistency of surrogate losses for multiclass prediction. The authors present their results using the formalism of structured prediction. Alas, there is no direct connection or exploitation of the separability of structured prediction losses. The paper is overly notated and variables are frequently overloaded. I got the feeling that the authors are trying to look mathematically fancy at the expense of readability. As for the technical content, I find the contribution in terms of new proof techniques quite marginal. The authors build on results by Toingo(Tong + Ingo) and the theorems seem lackluster to me.
From algorithmic prospective the paper has rather little to offer. The quadratic loss is either intractable to calculate or in the case of separable it amounts to sum of squares over the vector L_j. There's no attempt to experimentally validate the calibrated losses. Also absent is a sincere discussion on the importance of calibration given the empirical success of overly-specified models.
In "Contributions" paragraph the word "exponential" repeats over and over. It would not be clear to a reader who is not familiar with the setting what is the notion of exponential dependency.
The sentence "...  sets of possible predictions and correct outputs always coincide and do not depend on x." left me baffled.
There is excessive overload of notation for L which is used as a function, specific losses, table, etc...
minimal -> smallest / lowest
We denote the y-th component of f with f y
"... for now, we assume that a family of score functions F consists of all vector-valued Borel measurable functions f : X → F where F ⊆ R k ..." and then
what ?
Is it feasible to find inf φ ?
"... equals the smallest excess of the conditional surrogate risk when the excess of the conditional actual risk is larger than ..." left me bewildered again.
I feel that paper would have rather little appeal at NIPS and a natural venue would be COLT or ALT.v


Gradient Descent Can Take Exponential Time to Escape Saddle Points
Although gradient descent (GD) almost always escapes saddle points asymptotically , this paper shows that even with fairly natural random initialization schemes and non-pathological functions, GD can be significantly slowed down by saddle points, taking exponential time to escape. On the other hand, gradient descent with perturbations [Ge et al., 2015 is not slowed down by saddle points-it can find an approximate local minimizer in polynomial time. This result implies that GD is inherently slower than perturbed GD, and justifies the importance of adding perturbations for efficient non-convex optimization. While our focus is theoretical, we also present experiments that illustrate our theoretical findings.

It has recently been shown that, when all the saddle points of a non-convex function are "strict saddle", then gradient descent with a (reasonable) random initialization converges to a local minimizer with probability one. For a randomly perturbed version of gradient descent, the convergence rate can additionally be shown to be polynomial in the parameters. This article proves that such a convergence rate does not hold for the non-perturbed version: there exists reasonably smooth functions with only strict saddle points, and natural initialization schemes such that gradient descent requires a number of steps that is exponential in the dimension to find a local minimum.
I liked this article very much. It answers a very natural question: gradient descent is an extremely classical, and very simple algorithm. Although it is known not to be the fastest one in many situations, it is widely used in practice; we need to understand its convergence rate. The proof is also conceptually simple and elegant, and I found its presentation very clear.
My only criticism is that it seems to me that there are a lot of miscalculations in the proof, notably in the proof of Lemma B.2. I absolutely do not think they are a problem for the correctness of the proof, but I would appreciate them to be corrected before the article is published.
Minor remarks and typos:
- l.91: "postpone in" should be "postpone to"
- l.109: ""strict" defined" should be ""strict", which is defined"?
- l.115: "the approximation to" should be "approximate"?
- Paragraph "initialize far, far away": some condition on h is needed to ensure that x_2 does not suddenly increase when x_1 is between -2 and -1. (I understand that this discussion is informal anyway, but it could be briefly mentioned.)
- l.163: "in previous" should be "as in previous"
- l.166: why [-1/2;1/2] and not [-1;1]?
- l.167: "has" should be "have"
- l.173: "from of" should be "from"
- l.191: no comma before the closing parenthesis
- Corollary 4.3: "The for"? "satisfies" should be "that satisfies"; "same of" should be "same as in"
- l.217: "Guassian"
- l.218, "where ...": a verb is missing.
- Equation (3), second and third line, right hand side: x should be x_1.
- l.241: "still satisfy the assumptions" should be "the assumptions are still satisfied"
- l.250: "constraint" should be "constrained"
- l.251 and 438: "make as" should be "make"?
- l.252: remove "both coordinate".
- l.261: "(14)and" should be "(14) and"
- l.281: "when if" should be "if"
- I know that the conjunction "that" can often be omitted after verbs like "know", "note", "imply" ... However, I would personally find the sentences easier to understand if "that" was not omitted when it introduces a complex proposition (lines 274, 388, 391 ...).
- l.367: I do not understand "We connnect ... in a spline".
- l.376: "where" should be "which"
- Page 12: the number of the figure should be written in the caption.
- l.397: "above lemma" should be "lemma below"
- l.403: remove "for GD"
- l.406: "the iterate" should be "the iterates"
- Page 13: it seems to me that we need to know that the iterates stay in D_0 in order for the first equations (after line 414) to be valid (which is proved after the equations).
- Equation after l.421, second line: I think that the inequality given by Lemma B.2 yields a \geq, not \leq.
- Line after this equation: "where the" should be "where for the".
- Equation after l.426, second line: a "2" is missing.
- I do not think that the definition in Equation (12) is correct (because, with this definition, D_a does not depend on a).
- Equation between (15) and (16): shouldn't a distinction also be made on the sign of x_{i+1}?
- Step 2: why couldn't we simply define f(x_1,\dots,x_d) by f(x_1,\dots,x_d)=f(|x_1|,\dots,|x_d|)? It seems to me that it is more or less what is done, but written in a more complicated way. Maybe I did not understand this part.
- l.460: "agree" should be "agree with"
- l.471: maybe also say that there is no noise added when \tau \leq x_k\leq 2\tau?
- l.472: I think the assumption should be r \leq \tau/20.
- l.478: "same argument" should be "same argument as"
- l.496: the comma should be on the previous line.
- Theorem B.1: the assumption that f'(y_0)\leq 0 is also needed in order to ensure p(y) \leq f(y_0).
- According to my computations, g_1''(\tau) is equal to -4\gamma, while the continuity assumption on the first derivative requires -2\gamma.
- l.521: I do not see why -g_1(2\tau)+4L\tau^2 is equal to (17\gamma/6+11 L/2)\tau^2.
- I do not see why \nabla f_{i,2} is equal to \nabla f_{i+1,1} (for the derivative along x_i, I do not find the same value).
- Theorem B.3: it seems to me that this is an over-simplification of [Chang, 2015, Theorem 1.3] (in particular, but not only, because I think that the notion of C^m functions does not really make sense on generic closed subsets of \R^d).


Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning
Off-policy model-free deep reinforcement learning methods using previously collected data can improve sample efficiency over on-policy policy gradient techniques. On the other hand, on-policy algorithms are often more stable and easier to use. This paper examines, both theoretically and empirically, approaches to merging on-and off-policy updates for deep reinforcement learning. Theoretical results show that off-policy updates with a value function estimator can be interpolated with on-policy policy gradient updates whilst still satisfying performance bounds. Our analysis uses control variate methods to produce a family of policy gradient algorithms, with several recently proposed algorithms being special cases of this family. We then provide an empirical comparison of these techniques with the remaining algorithmic details fixed, and show how different mixing of off-policy gradient estimates with on-policy samples contribute to improvements in empirical performance. The final algorithm provides a generalization and unification of existing deep policy gradient techniques, has theoretical guarantees on the bias introduced by off-policy updates, and improves on the state-of-the-art model-free deep RL methods on a number of OpenAI Gym continuous control benchmarks.

* summary: This paper studies how to merge on-policy and off-policy gradient algorithms. The authors propose a new policy gradient framework that unifies many previous on-policy and off-policy gradient methods. Many previous policy gradient algorithms can not only be re-derived and but also get improved by the introduction of control variate. Even though this framework introduces bias to gradient updates but, they show in theoretical results that this bias can be bounded. Experiments are very well done and provide enough insights to understand the proposed framework.
* In overall, the paper is very well written with excellent related work and discussions. The main contents are well presented as well, the theoretical results are very intuitive and make sense.
* I think this is very strong paper. I have only few comments as following:
- I think that even if one is familiar with TRPO, he might still not be able to know how to transfer the technique of TRPO to approximately implement Algorithm 2. It would be nicer if the authors elaborate on this step.
- As seen in Eq. (6) and Theorem 1, the bias introduced by the difference between \beta and \pi is non-trivial. Therefore, I find that IPG does not really always outperform Q-Prop, sometimes they are comparable (when \beta is not equal to \pi). According to my understanding, v from 0 to 1 means a shifting gradually from on-policy to off-policy, therefore one could compensate the bias by gathering more data/larger batch-size. Therefore, it might be more interesting to see if Table 2 is expanded with more experiment setting: v increases gradually from 0 to 1 (with and without control variate), and also with the change of data-size/batch-size to see if bias can be compensated somehow.
- line 151, should it be $v=0$?


Exploring Generalization in Deep Learning
With a goal of understanding what drives generalization in deep networks, we consider several recently suggested explanations, including norm-based control, sharpness and robustness. We study how these measures can ensure generalization, highlighting the importance of scale normalization, and making a connection between sharpness and PAC-Bayes theory. We then investigate how well the measures explain different observed phenomena.

Update after rebuttal: quote the rebuttal "Even this very simple variability in architecture, proves challenging to study using the complexity measures suggested. We certainly intend to study also other architectural differences."
I hope the authors include the discussions (and hopefully some experiment results) about applying the proposed techniques to the comparison of models with different architectures in the final version if it get accepted. The current results are already useful steps towards understanding deep neural networks by their own, but having this results is really a great add on to this paper.
-------------------------
This paper investigated primarily two (related) questions: what is the complexity measure that is implicitly enforced to allow good generalization performance of large deep neural networks? Given different global minimizers of the empirical risk, can we tell which one generalize better based on the complexity measure.
Although no definite answer is given in this paper, a few options are explored and some of them seem to be positive choice. More specifically, norm based complexity measure and (average) sharpness based complexity measures are empirically investigated. The sharpness based measure are further justified via PAC-Bayes bound on generalization.
There is no discussion on how or why those complexity measures are enforced during training of neural networks, but the explorations are interesting by themselves.
Although a theorem is given to control the sharpness of a network under a number of assumptions (C1-C3), this is primarily an empirical paper. Therefore, the reviewer thinks it would be more useful to include more extensive experiments. More specifically, could the proposed measures be useful when comparing across different architectures? Could it be useful when compare to different global minimizers from different algorithms (variants of SGD, or SGD with small / large batches)? etc.


Detrended Partial Cross Correlation for Brain Connectivity Analysis
Brain connectivity analysis is a critical component of ongoing human connectome projects to decipher the healthy and diseased brain. Recent work has highlighted the power-law (multi-time scale) properties of brain signals; however, there remains a lack of methods to specifically quantify short-vs. long-time range brain connections. In this paper, using detrended partial cross-correlation analysis (DPCCA), we propose a novel functional connectivity measure to delineate brain interactions at multiple time scales, while controlling for covariates. We use a rich simulated fMRI dataset to validate the proposed method, and apply it to a real fMRI dataset in a cocaine dependence prediction task. We show that, compared to extant methods, the DPCCA-based approach not only distinguishes short and long memory functional connectivity but also improves feature extraction and enhances classification accuracy. Together, this paper contributes broadly to new computational methodologies in understanding neural information processing.

The paper proposes a partialized version of DCCA, and applies it to synthetic and real fMRI data. While the contribution is incremental (this is a straight analogy to Whittaker's 1990 estimator of partial correlation), the results are at least on par with existing methods, and offer very interesting additional insights into connectivity across time scales. It also brings methods from statistical physics into neuroimaging, where they have been used (for DFA see e.g. e.g. Shen et al Clin Neurophysiol 2003 (EEG),  Mutch et al PLoS one 2012 (fMRI)), but not often for connectivity analysis. There are a few technical issues to address.
# Section 2.2.1
ROIs are derived from task fMRI on the whole dataset. This is problematic for the cocaine dependence prediction task (section 4.4) because it happens outside the cross-validation fold. The GLM should be re-trained within each fold using only training data, to compute ROIS which would then be applied to constructuct both training and testing graphs. Doing it outside the fold gives an unfair advantage and artificially boosts performance. Given the relatively large (by neuroimaging standards) sample size, this should not be too dramatic but it needs to be acknowledged in the text.
This is of course causes no problem in comparisons between connectivity methods since they all use the same ROIs.
# Section 3.1
Equation 1: in the denominator, should `C_{a,b} be C_{a,a}`, and likewise for `C_{b,b}` ?
Is there a guarantee that \rho in equation 2 is always invertible ?
# Section 3.2.2, 3.2.3
Why use the max over all time scales rather than some other statistic of the absolute value ? Figure 3 illustrates why the authors chose this, but this choice of an extreme statistic probably comes with a bias towards false positives. In this respect it is not clear why the authors say on lines 177-178 that DPCCA has low TPR?
Also, step 17 in algorithm 1 does not correct for multiple comparisons, using an uncorrected p-value while where are many hypothesis tests, which also contributes to false positives. This is not necessarily an issue but, together with the max operator of line 5, it is unclear how to assign proper p-values to each edge after algorithm 1, if that were desireable.
The null model chosen works for multi-subject studies only. It yields a very different (much lower) degree distribution. Is this a concern?
# Section 4.1
DCCA max is actually also located correctly at 10 s (fig 3e)
# Section 4.2
Please move hypothesis test results about DPCCA+CCA > ICOV from conclusion to this section. Given table 1 and large std it is not obvious at all that DPCCA+CCA is better, so having the test result here will be more convincing.
# Section 4.4.
As mentioned in my remarks on 2.2.1, ROIs are computed out-of-fold, which gives a small unfair edge to all methods, but does not affect comparison between methods computing connectivity on the same ROIs.
Using AUC is not recommended for comparing different classifiers (Hand, Machine Learning 77, 2009, but also Hanczar et al., Bioinformatics 26, 2010). Please show balanced accuracy as well.
# Section 5
In table 3 it looks as if all methods (except Corr) do about the same, but the authors state in lines 241-242 that DPCCA+PCA works best. Given large std I don't think that the difference with, say ICOV or ParCorr is significant. Please provide a hypothesis test here.


Improving Regret Bounds for Combinatorial Semi-Bandits with Probabilistically Triggered Arms and Its Applications
We study combinatorial multi-armed bandit with probabilistically triggered arms and semi-bandit feedback (CMAB-T). We resolve a serious issue in the prior CMAB-T studies where the regret bounds contain a possibly exponentially large factor of 1/p * , where p * is the minimum positive probability that an arm is triggered by any action. We address this issue by introducing a triggering probability modulated (TPM) bounded smoothness condition into the general CMAB-T framework, and show that many applications such as influence maximization bandit and combinatorial cascading bandit satisfy this TPM condition. As a result, we completely remove the factor of 1/p * from the regret bounds, achieving significantly better regret bounds for influence maximization and cascading bandits than before. Finally, we provide lower bound results showing that the factor 1/p * is unavoidable for general CMAB-T problems, suggesting that the TPM condition is crucial in removing this factor.

Overview: The paper studies combinatorial multi-armed bandit with probabilistically triggered arms and semi-bandit feedback (CMAB-T). The paper improves the existing regret bounds by introducing a new version of the bounded smoothness condition on the reward functions, called triggering probability modulated bounded smoothness condition. The intuition behind this is that hard-to-trigger arms do not contribute much to the expected regret. With this new condition, the authors are able to remove a potentially exponentially large factor in the existing regret bounds. The paper then shows that applications such as influence maximization bandit and combinatorial cascading bandit both satisfy this condition. Finally, the paper gives a lower bound and shows that for general CMAB-T problems, the exponential factor is unavoidable.
Quality: The paper provides rigorous proofs for all its claims.
Clarity: The paper is well-organized and clearly-written. Everything is well-defined and well-explained. The paper also has a nice flow to it. My only complaint is that a concluding section is missing. The authors may also want to proofread for typos and grammatical errors.
Originality: The paper strengthens an existing condition and gives a tighter regret bound for CMAB-T problems. The paper also applies this new condition to influence maximization bandit and combinatorial cascading bandit. The authors introduce several new techniques in their proofs.
Significance: The paper greatly improves the regret bound for CMAB-T problems with a new condition that has many applications.


Deep Dynamic Poisson Factorization Model
A new model, named as deep dynamic poisson factorization model, is proposed in this paper for analyzing sequential count vectors. The model based on the Poisson Factor Analysis method captures dependence among time steps by neural networks, representing the implicit distributions. Local complicated relationship is obtained from local implicit distribution, and deep latent structure is exploited to get the long-time dependence. Variational inference on latent variables and gradient descent based on the loss functions derived from variational distribution is performed in our inference. Synthetic datasets and real-world datasets are applied to the proposed model and our results show good predicting and fitting performance with interpretable latent structure.

This papers introduces the deep dynamic Poisson factorization model, a model that builds on PF to allow for temporal dependencies. In contrast to previous works on dynamic PF, this paper uses a simplified version of a recurrent neural network to allow for long-term dependencies. Inference is carried out via variational inference, with an extra step to maximize over the neural network parameters. The paper reports experiments on 5 real-world datasets.
Overall, I found the idea potentially interesting, but I still think the paper needs some improvements in its execution before it can be accepted in a conference like NIPS. Please find my comments below.
+The formulation in Eq. 4 doesn't correspond to a RNN structure, because h_t^{(n)} depends only on the top-layer memory vectors, and it does not depend on previous time steps for the same layer, as in the RNN setup. Is there any reason for this simplified structure?
+In Section 2, some of the model equations are missing (e.g., it is not clear how to parameterize the Gamma in Eq. 4, or what's the distribution that generates theta_t in Eq. 3).
+I disagree with the discussion in lines 110-118 regarding "implicit distributions". Eqs. 9 and 10 are actually explicit: they are Gammas with given shapes and rates. Since the distribution of theta_tk (Eq. 9) can be computed in closed form conditioned on h^{(0)}, it cannot be called implicit. The same comment applies to h in Eq. 10.
+Lines 30-31: The sentence "This method may have weak points in analyzing other data with different pattern long-time dependence, such as fanatical data and disaster data" wasn't clear to me; can you elaborate?
+The subsection in lines 133-155 isn't clear to me. What is exactly the "loss function" in Eq. 15?
+MSE/PMSE is typically not a good metric for PF-like models. The results should additionally include at least predictive log-likelihood.
+The value of K in the experiments (line 207) seems too small to me. I suggest the authors use larger values, such as K=100 (at least).
+I couldn't connect the discussion in lines 224-232 with Fig. 3: it is not clear.
+The update equations for the variational inference procedure (Eqs. 11-14) can safely be moved to the supplement.
+The caption in Fig. 4 cannot be read clearly.
+I *strongly* recommend to follow general writing advice about the use of the passive voice (specially for the abstract).
+The writing quality is poor, and the paper contains many typos. Here are some examples:
  -Line 15, "high-dimensional"
  -Line 22, "factorize"
  -Line 24, "property of the"
  -Lines 29, 34, "Dirichlet"
  -Line 34, "nested"
  -Fig. 1, "visual representation"
  -Fig.1, "transmission" (not sure if that's an appropriate word anyways)
  -Incomplete sentences (e.g., line 49)
  -Many missing articles (e.g., "Although Dirichlet distribution is often used as prior distribution")


Rényi Differential Privacy Mechanisms for Posterior Sampling
With the newly proposed privacy definition of Rényi Differential Privacy (RDP) in [14], we re-examine the inherent privacy of releasing a single sample from a posterior distribution. We exploit the impact of the prior distribution in mitigating the influence of individual data points. In particular, we focus on sampling from an exponential family and specific generalized linear models, such as logistic regression. We propose novel RDP mechanisms as well as offering a new RDP analysis for an existing method in order to add value to the RDP framework. Each method is capable of achieving arbitrary RDP privacy guarantees, and we offer experimental results of their efficacy.

This paper analyzes the privacy cost of posterior sampling for exponential family posteriors and generalized linear models by using the recently proposed privacy definition of Renyi-DP. Specifically, they applied the privacy analyses to two domains and presented experimental results for Beta-Bernoulli sampling and Bayesian Logistic Regression.
In my opinion, the privacy analyses that the paper considers can be interesting and valuable for the community. But, the paper does not motivate enough the necessity of the usage of Renyi-DP. It is a relaxed notion of pure DP like CDP and z-CDP but the readers cannot learn from the paper that when they can use it or why should they use it.
In terms of technical correctness of the the paper, the algorithm presented relies on RDP and standard DP definitions, I believe the analyses are correct. But, the problem is the authors did not cite the definitions which are already published in the literature (e.g. Definition 1-2-3-4-…). So, it is hard for me (and I suppose for the other readers) to distinguish which of the definitions/theorems are proposed by the authors and which of them are already published.
Furthermore, the search process to find an “appropriate” value of r and m is not clear. In the experiments section they searched over six values but we do not know whether all of them are valid or not. What do you mean by “appropriate”? Besides, I prefer to see the connection/correlation between the values r/m and accuracy (their intuitive interpretation would be immensely helpful). Maybe I miss it, but the paper does not provide any formal utility guarantee, which is something interesting (at least we can see the privacy/accuracy connection empirically).
Finally, in Figure 2c and 2d and in the real data experiments, there is no baseline such as the non-private case. OPS is not a proper baseline because we cannot compare pure-DP and RDP (maybe you can formalize the connection somewhere in methodology and then we can compare). So, it will be nice to show what is a good result for these experiments when privacy is not a concern.


Is Input Sparsity Time Possible for Kernel Low-Rank Approximation?
Low-rank approximation is a common tool used to accelerate kernel methods: the n ⇥ n kernel matrix K is approximated via a rank-k matrixK which can be stored in much less space and processed more quickly. In this work we study the limits of computationally efficient low-rank kernel approximation. We show that for a broad class of kernels, including the popular Gaussian and polynomial kernels, computing a relative error k-rank approximation to K is at least as difficult as multiplying the input data matrix A 2 R n⇥d by an arbitrary matrix C 2 R d⇥k . Barring a breakthrough in fast matrix multiplication, when k is not too large, this requires ⌦(nnz(A)k) time where nnz(A) is the number of non-zeros in A. This lower bound matches, in many parameter regimes, recent work on subquadratic time algorithms for low-rank approximation of general kernels [MM16, MW17], demonstrating that these algorithms are unlikely to be significantly improved, in particular to O(nnz(A)) input sparsity runtimes. At the same time there is hope: we show for the first time that O(nnz(A)) time approximation is possible for general radial basis function kernels (e.g., the Gaussian kernel) for the closely related problem of low-rank approximation of the kernelized dataset.

The paper presents negative and positive results to the problem of computing low-rank approximations to the kernel matrix of a given dataset. More specifically, for a fixed kernel function and given some input data and a desired rank k, the authors consider the two related problems of (1) computing a rank k matrix which is relatively close to the optimal rank k approximation of the kernel matrix and of (2) computing an orthonormal basis of a k-dimensional space such that when projecting the data from the feature space (defined by the kernel) down to this k-dimensional space it is relatively close to the optimal such projection. In particular, the paper focuses on the question whether it is possible to solve these problems in a time that is independent of the dimensionality of the input data matrix A (say n times d) and instead depends on the number of non-zero entries in A.
Regarding problem (1) the paper provides a negative answer for the linear kernel, all polynomial kernels, and the Gaussian kernel by showing that the problem is as hard as computing (exactly) the product of the input matrix with an arbitrary matrix C (of dim d times k). This implies that a solution cannot be achieved without a significant improvement of the state of the art in matrix multiplication.
Regarding problem (2) the paper provides a positive result for shift-invariant kernels using an algorithm based on random Fourier features. While the practical relevance of the given algorithm is limited due to a poor scaling with the number of data points (n to the 4.5), its existence is theoretically significant because it achieves the desired independence of n times d and, thus, gives hope for further improvement down to a practically relevant dependencies on n.
The given results constitute valuable and technically challenging contributions to a topic of high relevance to the NIPS community. Additionally, the paper is excellently written including a good exposition of the results and high-level outline of the proofs, which makes it easier to follow the detailed steps.


Matching neural paths: transfer from recognition to correspondence search
Many machine learning tasks require finding per-part correspondences between objects. In this work we focus on low-level correspondences -a highly ambiguous matching problem. We propose to use a hierarchical semantic representation of the objects, coming from a convolutional neural network, to solve this ambiguity. Training it for low-level correspondence prediction directly might not be an option in some domains where the ground-truth correspondences are hard to obtain. We show how transfer from recognition can be used to avoid such training. Our idea is to mark parts as "matching" if their features are close to each other at all the levels of convolutional feature hierarchy (neural paths). Although the overall number of such paths is exponential in the number of layers, we propose a polynomial algorithm for aggregating all of them in a single backward pass. The empirical validation is done on the task of stereo correspondence and demonstrates that we achieve competitive results among the methods which do not use labeled target domain data.

This paper presents a method for determining spatial correspondences between two images using a pre-trained convolutional network, without needing to train on the specific correspondence task.  Given two images, the matching similarity at each pixel location is defined as the sum total similarity between all pairs of possible paths through network, starting at each location and its corresponding offset location in the other image.  For a single offset value, this results in a recurrence relation dependent on the next layer up, which enables a dynamic-programming-like algorithm to compute all costs in linear time using a single backwards pass.  In this way, a multi-layer matching is performed for each offset, accounting for a hierarchy of scales and features.
This algorithm is applied to the task of stereo matching, where it achieves performance slightly lower than SOA supervised methods that require retraining, but better than a baseline computing correlation between stacked feature maps.
This idea is an interesting way to use a pretrained network for additional uses.  However, I think it could be further explored and evaluated, and perhaps more simply explained.  In particular, I would like to see whether it can be applied to other dense matching tasks, e.g. video optical flow, and compared to other dense correspondence methods, such as Deep Flow and SIFT Flow (in both speed and accuracy).  In addition, the notation seems a little overly complex and can be hard to follow (though I was able to understand it) --- I think it would be good to try simplifying this if possible.
Overall, I think this is an interesting method, but feel it could be further explored for more image correspondence applications.
Additional detail:  Twice the paper says the derivation of the algorithm makes use of the "associative" property, but this is actually the distributive property (not associative).


Learning Identifiable Gaussian Bayesian Networks in Polynomial Time and Sample Complexity
Learning the directed acyclic graph (DAG) structure of a Bayesian network from observational data is a notoriously difficult problem for which many non-identifiability and hardness results are known. In this paper we propose a provably polynomialtime algorithm for learning sparse Gaussian Bayesian networks with equal noise variance -a class of Bayesian networks for which the DAG structure can be uniquely identified from observational data -under high-dimensional settings. We show that O(k 4 log p) number of samples suffices for our method to recover the true DAG structure with high probability, where p is the number of variables and k is the maximum Markov blanket size. We obtain our theoretical guarantees under a condition called restricted strong adjacency faithfulness (RSAF), which is strictly weaker than strong faithfulness -a condition that other methods based on conditional independence testing need for their success. The sample complexity of our method matches the information-theoretic limits in terms of the dependence on p. We validate our theoretical findings through synthetic experiments.
ordinary least squares problems, where p is the number of nodes and k is the maximum Markov blanket size of a variable. We show that O(( k 4 /↵ 2 ) log( p / )) samples suffice for our algorithm to recover the true DAG structure and to approximate the parameters to at most ↵ additive error, with probability at least 1 , for some > 0. The sample complexity of O(k 4 log p) is close to the information-theoretic limit of ⌦(k log p) for learning sparse GBNs as obtained by [4]. The main assumption under which we obtain our theoretical guarantees is a condition that we refer to as the ↵-restricted strong adjacency faithfulness (RSAF). We show that RSAF is a strictly weaker condition than strong faithfulness, which methods based on independence testing require for their success. In this identifiable regime, given enough samples, our method can recover the exact DAG structure of any Gaussian distribution. However, existing exact algorithms like the PC algorithm [5] can fail to recover the correct skeleton for distributions that are not faithful, and fail to orient a number of edges that are not covered by the Meek orientation rules [6,7]. Of independent interest is our analysis of OLS regression under the random design setting for which we obtain`1 error bounds.
Related Work. In the this section, we first discuss some identifiability results for GBNs known in the literature and then survey relevant algorithms for learning GBNs and Gaussian SEMs.
[3] proved identifiability of distributions drawn from a restricted SEM with additive noise, where in the restricted SEM the functions are assumed to be non-linear and thrice continuously differentiable. It is also known that SEMs with linear functions and strictly non-Gaussian noise are identifiable [8]. Indentifiability of the DAG structure for the linear function and Gaussian noise case was proved by [9] when noise variables are assumed to have equal variance.
Algorithms for learning BNs typically fall into two distinct categories, namely: independence test based methods and score based methods. This dichotomy also extends to the Gaussian case. Score based methods assign a score to a candidate DAG structure based on how well it explains the observed data, and then attempt to find the highest scoring structure. Popular examples for the Gaussian distribution are the log-likelihood based BIC and AIC scores and the`0-penalized log-likelihood score by [10]. However, given that the number of DAGs and sparse DAGs is exponential in the number of variables [4,11], exhaustively searching for the highest scoring DAG in the combinatorial space of all DAGs, which is a feature of existing exact search based algorithms, is prohibitive for all but a few number of variables. [12] propose a score-based method, based on concave penalization of a reparameterized negative log-likelihood function, which can learn a GBN over 1000 variables in an hour. However, the resulting optimization problem is neither convex -therefore is not guaranteed to find a globally optimal solution -nor solvable in polynomial time. In light of these shortcomings, approximation algorithms have been proposed for learning BNs which can be used to learn GBNs in conjunction with a suitable score function; notable methods are Greedy Equivalence Search (GES) proposed by [13] and an LP-relaxation based method proposed by [14].
Among independence test based methods for learning GBNs, [15] extended the PC algorithm, originally proposed by [5], to learn the Markov equivalence class of GBNs from observational data. The computational complexity of the PC algorithm is bounded by O(p k ) with high probability, where k is the maximum neighborhood size of a node, and is only efficient for learning very sparse DAGs. For the non-linear Gaussian SEM case, [3] developed a two-stage algorithm called RESIT, which works by first learning the causal ordering of the variables and then performing regressions to learn the DAG structure. As we formally show in Appendix C.1, RESIT does not work for the linear Gaussian case. Moreover, Peters et al. proved the correctness of RESIT only in the population setting. Lastly, [16] developed an algorithm, which is similar in spirit to our algorithm, for efficiently learning Poisson Bayesian networks. They exploit a property specific to the Poisson distribution called overdispersion to learn the causal ordering of variables.
Finally, the max-min hill climbing (MMHC) algorithm by [17] is a state-of-the-art hybrid algorithm for BNs that combines ideas from constraint-based and score-based learning. While MMHC works well in practice, it is inherently a heuristic algorithm and is not guaranteed to recover the true DAG structure even when it is uniquely identifiable.

Summary:
    The paper considers a Gaussian Bayesian Network which is directed acyclic and is governed by structural equations which are linear with additive gaussian noise of identical variance. This class of bayesian networks have been shown to be identifiable in prior work. There has been no polynomial time algorithm known even for this case.
The authors propose a polynomial time algorithm which involves inverse covariance estimation, ordinary least squares and a simple identification rule for terminal vertices (leaf vertices in the Directed Acyclic Graph) for this problem. Under the assumption of restricted strong adjacency faithfulness (RSAF) (which essentially says that for all induced sub graphs, variables in the Markov Blanket exhibit high enough influence on a variable) the algorithm identifies the correct graph with O(k^4 log p) samples where k is the maximum size of the Markov Blanket.
Strengths:
   a) As far as I am aware this is the first fully polynomial time algorithm for the problem under this setting.
   b) The paper hinges on few important observations:
     1) Learning undirected gaussian graphical models is very very efficient due to large number of papers on inverse covariance estimation.
     2) But usually inverse covariance estimation can only recover the Markov Blanket of every variable (considering the moralized graph) and cannot in general reveal directions.
     3) However, if a topological ordering of the vertices respecting the partial order in the DAG is known, then Ordinary least squares can be repeatedly used to figure out parents of a variable i by considering an induced graph with vertices in the ordering upto i and the inverse covariance structure at every subsequent stage.
     4) The authors exploit the following key fact construct a topological ordering by discovering terminal vertices (lead vertices) recursively: When a terminal vertex i is regressed on the rest of the nodes, then ratio of regression coefficient of vertex j in the graph to (i,j)th entry in the inverse covariance matrix equal to the noise variance in the problem. In fact, for non terminal vertices, the ratio is strictly larger under RSAF condition. This involves only inverse covariance estimation which is efficient.
  c)) Because the authors define effective influence characterizing the markov blanket in terms of the weights in the structural equations, authors can handle cases even when the model is not causally faithful (this is a standard assumption for most causal inference algorithms).
  d) In all its a really good paper. The authors have also compared it to some state of the art algorithms like PC, MMHC and GES and shown that for synthetic gaussian models, they do better in terms of recovery.

 However, I only have some semi-mild to some mild comments. Comment d is the most serious. It would be great if authors can address it.
 Weaknesses:
      a) In page 3 at the end, Markov Blanket of vertex i is defined using effective influence and the information theoretic definition that a set of nodes such that conditioned on them, the rest of the variables are independent to i. Under casual faithfulness, markov blanket is simply - parents, children and co-parents. I think defining it in terms of effective influence lets them deal with pathological cases not covered by causal faithfulness. This lets them work on examples in page 5. I think this must be made clear. Because a lot of readers of BN literature may be used to a graph theoretic notion of Markov Blanket.
     b) In the example in page 5 below Proposition 1, it appears the for i,j which are neighbors if one chooses a pathological set of weights, only then effective influence becomes zero which can still be handled by the algorithm of the authors. I find it to be a corner case although I agree technically its a super set. The reason for this comment is most other algorithms like PC, MMHC are completely non parametric as far as at least skeleton + v-structure discovery is concerned. But this algorithm has very tight parametric assumptions (so it may not be that fair a comparison).
    c) Page 2, before Section 2 "MMHC works well in practice but inherently a heuristic algorithm" - I partially agree with this. MMHC has two parts - one is skeleton recovery (which is provably correct under infinite samples) and the other part used score based optimization heuristic to orient. So it may not be totally correct to say it is heuristic. Another minor point - Like 74 , complexity of PC is said to be p^k. k in the paper is Markov Blanket size. However the PC's complexity exponent is the max degree in the graph. Markov blanket is much larger.

   d) Slightly serious concern:
       Page 7 , Algorithm 1, Line 24: Now the algorithm after identifying the order is considering induced subgraphs of first i vertices in the ordering and using regression of i on the remaining is finding the parents of i. First, to find the markov blanket of i i the induced sub graph , \hat{\Omega}_{z_i,j} is used - It should be z_i,z_j in the subscript (I think). But the major concern is which inverse covariance is it ? Is it the induced subgraph's inverse covariance upto vertex i in the ordering ? or is it the inverse covariance of the entire graph estimate at 6? Please clarify. This seems important because this step of identifying node i's parents from just from the ordering and regression is something crucial. In fact, the explanation I understood is:  the global invariance covariance matrix's markov blanket of i is intersected with the the induced sub graph of vertices till i, now the node i is regressed on its non zero entries. And it will turn out that the parent set of i is correctly identified. This is because in the induced sub graph, markov blanket of i will not include the co-parents. This has been explained in the last but one paragraph in page 14 in Appendix A for proof of theorem 1. But I think this reasoning should be included in the main paper and Line 24 in Algorithm 2 must be clarified even better.
e) In Appendix B.5 authors talk about some ref 31 which can split the graph into connected components. Does it mean that the graph in that table has connected components of size at most 500. So has the algorithm applied after the connected components found out ?? If this is the case 500 is not a large number for MMHC algorithm. In fact in the 2006 paper they claim it can actually process 1000 node graphs. Why did the authors not compares to that in that case ?
f)Line 100, (i,j) seems to be directed edge from j to i. From my experience, this seems like an odd choice. It is usually i->j. But this is just my personal quip.
g) Page 11 . Appendix, Proof of Lemma 1, First line. "forward direction follows from (5)". Should it not be Lemma 5 ??