Gradient Estimators for Implicit Models
Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research. Some examples include data simulators that are widely used in engineering and scientific research, generative adversarial networks (GANs) for image synthesis, and hot-off-the-press approximate inference techniques relying on implicit distributions. The majority of existing approaches to learning implicit models rely on approximating the intractable distribution or optimisation objective for gradient-based optimisation, which is liable to produce inaccurate updates and thus poor models. This paper alleviates the need for such approximations by proposing the emph{Stein gradient estimator}, which directly estimates the score function of the implicitly defined distribution. The efficacy of the proposed estimator is empirically demonstrated by examples that include meta-learning for approximate inference and entropy regularised GANs that provide improved sample diversity.
 In this paper, the authors proposed the Stein gradient estimator, which directly estimates the score function of the implicit distribution. Direct estimation of gradient is crucial in the context of GAN because it could potentially lead to more accurate updates. Motivated by the Stein’s identity, the authors proposed to estimate the gradient term by replacing expectation with the empirical counterpart and then turn the resulting formulation into a regularized regression problem. They also showed that the traditional score matching estimator (Hyvarinen 2005) can be obtained as a special case of their estimator. Moreover, they also showed that their estimator can be obtained by minimizing the kernelized Stein discrepancy (KSD) which has been used in goodness-of-fit test. In the experiments, the proposed method is evaluated on few tasks including Hamiltonian flow with approximate gradients, meta-learning of approximate posterior samplers, and GANs using entropy regularization. The novelty of this work consists of an approach based on score matching and Stein’s identity to estimate the gradient directly and the empirical results of the proposed method on meta-learning for approximate inference and entropy regularized GANs. The proposed method is new and technically sound. The authors also demonstrated through several experiments that the proposed technique can be applied in a wide range of applications. Nevertheless, I suspect that the drawback of this method compared to existing ones is computational cost. If it takes significantly longer to compute the gradient using proposed estimator compared to existing methods, the gain in terms of accuracy is questionable. By spending the same amount of time, we may obtain an equally accurate estimate using other methods. For example, the authors claimed in Section 4.3 that the Stein gradient estimator is faster than other methods, but it is not clear as to why this is the case. Hence, the comparison in terms of computational cost should also be included either in the text or in the experiment section. While the proposed Stein gradient estimator is technically interesting, the experimental results do not seem to evident that it significantly outperforms existing techniques. In Section 4.2, the authors only consider four datasets (out of six UCI datasets). Also, in Section 4.3, it is not clear what the point of this experiment is: whether to show that entropy regularization helps or the Stein gradient estimator outperforms other estimators. Some comments: - Perhaps, it is better to move Section 3.3 before Section 3.2 to emphasize the main contribution of this work, i.e., using Stein’s identity to derive an estimate of the gradient of the score function. - Stein gradient estimator vs KDE: What if the kernel is not translation invariant? - In Section 4.3, why did you consider the entropy regularizer? How does it help answer the main hypothesis of this paper? - The experiments in Section 4.3 seems to be a bit out of context.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Gradient Estimators for Implicit Models
Implicit models, which allow for the generation of samples but not for point-wise evaluation of probabilities, are omnipresent in real-world problems tackled by machine learning and a hot topic of current research. Some examples include data simulators that are widely used in engineering and scientific research, generative adversarial networks (GANs) for image synthesis, and hot-off-the-press approximate inference techniques relying on implicit distributions. The majority of existing approaches to learning implicit models rely on approximating the intractable distribution or optimisation objective for gradient-based optimisation, which is liable to produce inaccurate updates and thus poor models. This paper alleviates the need for such approximations by proposing the emph{Stein gradient estimator}, which directly estimates the score function of the implicitly defined distribution. The efficacy of the proposed estimator is empirically demonstrated by examples that include meta-learning for approximate inference and entropy regularised GANs that provide improved sample diversity.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 In this paper, the authors proposed the Stein gradient estimator, which directly estimates the score function of the implicit distribution. Direct estimation of gradient is crucial in the context of GAN because it could potentially lead to more accurate updates. Motivated by the Stein’s identity, the authors proposed to estimate the gradient term by replacing expectation with the empirical counterpart and then turn the resulting formulation into a regularized regression problem. They also showed that the traditional score matching estimator (Hyvarinen 2005) can be obtained as a special case of their estimator. Moreover, they also showed that their estimator can be obtained by minimizing the kernelized Stein discrepancy (KSD) which has been used in goodness-of-fit test. In the experiments, the proposed method is evaluated on few tasks including Hamiltonian flow with approximate gradients, meta-learning of approximate posterior samplers, and GANs using entropy regularization. The novelty of this work consists of an approach based on score matching and Stein’s identity to estimate the gradient directly and the empirical results of the proposed method on meta-learning for approximate inference and entropy regularized GANs. The proposed method is new and technically sound. The authors also demonstrated through several experiments that the proposed technique can be applied in a wide range of applications. Nevertheless, I suspect that the drawback of this method compared to existing ones is computational cost. If it takes significantly longer to compute the gradient using proposed estimator compared to existing methods, the gain in terms of accuracy is questionable. By spending the same amount of time, we may obtain an equally accurate estimate using other methods. For example, the authors claimed in Section 4.3 that the Stein gradient estimator is faster than other methods, but it is not clear as to why this is the case. Hence, the comparison in terms of computational cost should also be included either in the text or in the experiment section. While the proposed Stein gradient estimator is technically interesting, the experimental results do not seem to evident that it significantly outperforms existing techniques. In Section 4.2, the authors only consider four datasets (out of six UCI datasets). Also, in Section 4.3, it is not clear what the point of this experiment is: whether to show that entropy regularization helps or the Stein gradient estimator outperforms other estimators. Some comments: - Perhaps, it is better to move Section 3.3 before Section 3.2 to emphasize the main contribution of this work, i.e., using Stein’s identity to derive an estimate of the gradient of the score function. - Stein gradient estimator vs KDE: What if the kernel is not translation invariant? - In Section 4.3, why did you consider the entropy regularizer? How does it help answer the main hypothesis of this paper? - The experiments in Section 4.3 seems to be a bit out of context.
assistant
 In this paper, the authors proposed the Stein gradient estimator, which directly estimates the score function of the implicit distribution. Direct estimation of gradient is crucial in the context of GAN because it could potentially lead to more accurate updates. Motivated by the Stein’s identity, the authors proposed to estimate the gradient term by replacing expectation with the empirical counterpart and then turn the resulting formulation into a regularized regression problem. They also showed that the traditional score matching estimator (Hyvarinen 2005) can be obtained as a special case of their estimator. Moreover, they also showed that their estimator can be obtained by minimizing the kernelized Stein discrepancy (KSD) which has been used in goodness-of-fit test. In the experiments, the proposed method is evaluated on few tasks including Hamiltonian flow with approximate gradients, meta-learning of approximate posterior samplers, and GANs using entropy regularization. The novelty of this work consists of an approach based on score matching and Stein’s identity to estimate the gradient directly and the empirical results of the proposed method on meta-learning for approximate inference and entropy regularized GANs. The proposed method is new and technically sound. The authors also demonstrated through several experiments that the proposed technique can be applied in a wide range of applications. Nevertheless, I suspect that the drawback of this method compared to existing ones is computational cost. If it takes significantly longer to compute the gradient using proposed estimator compared to existing methods, the gain in terms of accuracy is questionable. By spending the same amount of time, we may obtain an equally accurate estimate using other methods. For example, the authors claimed in Section 4.3 that the Stein gradient estimator is faster than other methods, but it is not clear as to why this is the case. Hence, the comparison in terms of computational cost should also be included either in the text or in the experiment section. While the proposed Stein gradient estimator is technically interesting, the experimental results do not seem to evident that it significantly outperforms existing techniques. In Section 4.2, the authors only consider four datasets (out of six UCI datasets). Also, in Section 4.3, it is not clear what the point of this experiment is: whether to show that entropy regularization helps or the Stein gradient estimator outperforms other estimators. Some comments: - Perhaps, it is better to move Section 3.3 before Section 3.2 to emphasize the main contribution of this work, i.e., using Stein’s identity to derive an estimate of the gradient of the score function. - Stein gradient estimator vs KDE: What if the kernel is not translation invariant? - In Section 4.3, why did you consider the entropy regularizer? How does it help answer the main hypothesis of this paper? - The experiments in Section 4.3 seems to be a bit out of context.