Answers.Body: <p>In full:</p>
  
  <blockquote>
    <p>The limit, as standard deviation <span class="math-container">$\sigma$</span> tends towards zero, of the gradient with respect to vector <span class="math-container">$\mathbf{x}$</span>, of the expectation - where perturbation <span class="math-container">$\epsilon$</span> follows the normal distribution with mean 0 and variance <span class="math-container">$\sigma^2$</span> times identity vector <span class="math-container">$[1,1,1,1...]$</span> * - of any function <span class="math-container">$f$</span> of <span class="math-container">$\mathbf{x}$</span> plus <span class="math-container">$\epsilon$</span> is equal to the gradient with respect to <span class="math-container">$x$</span> of the same function of <span class="math-container">$\mathbf{x}$</span>.</p>
  </blockquote>
  
  <p>If we break that down:</p>
  
  <p><span class="math-container">$$\lim\limits_{\sigma \rightarrow 0}$$</span></p>
  
  <blockquote>
    <p>The limit, as standard deviation <span class="math-container">$\sigma$</span> tends towards zero of</p>
  </blockquote>
  
  <p><span class="math-container">$$\nabla_\mathbf{x}$$</span></p>
  
  <blockquote>
    <p>the gradient with respect to vector <span class="math-container">$\mathbf{x}$</span> of</p>
  </blockquote>
  
  <p><span class="math-container">$$\mathbb{E}$$</span></p>
  
  <blockquote>
    <p>the expectation ...</p>
  </blockquote>
  
  <p><span class="math-container">$$\mathbb{E}_{\epsilon \sim \mathcal{N}(0, \sigma^2\mathbf{I})}$$</span></p>
  
  <blockquote>
    <p>[the expectation] - where perturbation <span class="math-container">$\epsilon$</span> follows the normal distribution with mean 0 and variance <span class="math-container">$\sigma^2$</span> times identity vector <span class="math-container">$[1,1,1,1...]$</span> * - of</p>
  </blockquote>
  
  <p><span class="math-container">$$f(\mathbf{x} +\epsilon)$$</span></p>
  
  <blockquote>
    <p>any function <span class="math-container">$f$</span> of <span class="math-container">$x$</span> plus <span class="math-container">$\epsilon$</span></p>
  </blockquote>
  
  <p><span class="math-container">$$ = \nabla_x f(\mathbf{x})$$</span></p>
  
  <blockquote>
    <p>is equal to the gradient with respect to <span class="math-container">$\mathbf{x}$</span> of the same function <span class="math-container">$f(\mathbf{x})$</span>.</p>
  </blockquote>
  
  <p>Basically it says that taking small perturbations of a vector input to a function and measuring the gradient at those varied points can be used to give you a valid estimate of the true gradient at the point you are making variations of.</p>
  
  <p>In terms of understanding the equations, read introductory texts to the research area, and if like me your maths has been unused for many years before attempting this, expect to spend time and effort. Re-read the equations, memorise and write out the basic ones from the field, apply them to simple problems that might be presented in text books. Reading maths equations is not much different to reading music or reading another language - it takes concentration, practice, time and effort to become fluent enough to read an equation and comprehend it. The different research fields can be quite different too, some might be as similar enough to get by with what you know already, others may require learning all over again.</p>
  
  <hr>
  
  <p>* I am not 100% certain of the interpretation of <span class="math-container">$\mathbf{I}$</span> as an identity vector - a matrix may be more appropriate, which depends on the form of <span class="math-container">$\mathcal{N}(\mu, \sigma^2)$</span> when handling vector distributions. A matrix form for the second argument would be more general and allow for covariance, although the use of <span class="math-container">$\mathbf{I}$</span> would then explicitly remove covariance and make each component of <span class="math-container">$\epsilon$</span> independent, which is required for this result.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T13:13:27.820Z
Answers.Text: @DuttaA: Actually I am not sure, it depends on the form of second argument $\mathcal{N}$, which I am assuming will just take a vector of variances for independent vector components here, but you are right it could also support some forms of covariance in general, and we'd want an identity matrix instead to make the variables independent. Do you have a reference to the common assumptions of that function?
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T11:13:21.077Z
Answers.LastActivityDate: 2020-04-04T13:18:40.203Z
Answers.LastEditDate: 2020-04-04T13:18:40.203Z
Answers.UserDisplayName: Neil Slater
Body: <p>I was going through the <a href="https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf" rel="nofollow noreferrer">paper</a> on GAN by Ian Goodfellow. Under the related work section, there is an equation. I cannot decipher the equation. Can anyone help me understand the meaning of the equation? </p>
  
  <p><span class="math-container">$$\lim_{\sigma \to 0} \nabla_{\mathbf  x} \mathbb E_{\epsilon \sim \mathcal N(0, \sigma^2 \mathbf I)} f(\mathbf x+\epsilon) = \nabla_x f(\mathbf x)$$</span></p>
  
  <p>Also, any guide to understanding mathematical notation for reading research paper is highly appreciated.</p>
  
Tags: <math><generative-adversarial-networks><papers><notation>
Title: What does equation in the "related work" section of the GAN paper mean?
UserDisplayName: Bhuwan Bhatt
ThreadText: # What does equation in the "related work" section of the GAN paper mean?
  
  (Asked by: username_0 on 2020-04-04)
  
  I was going through the [paper](https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf) on GAN by Ian Goodfellow. Under the related work section, there is an equation. I cannot decipher the equation. Can anyone help me understand the meaning of the equation?
  
  $$\lim\_{\sigma \to 0} \nabla\_{\mathbf x} \mathbb E\_{\epsilon \sim \mathcal N(0, \sigma^2 \mathbf I)} f(\mathbf x+\epsilon) = \nabla\_x f(\mathbf x)$$
  
  Also, any guide to understanding mathematical notation for reading research paper is highly appreciated.
  
  **Tags:** math, generative-adversarial-networks, papers, notation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-04-04. Score: 2 | Accepted answer
  
  In full:
  
  > The limit, as standard deviation $\sigma$ tends towards zero, of the gradient with respect to vector $\mathbf{x}$, of the expectation - where perturbation $\epsilon$ follows the normal distribution with mean 0 and variance $\sigma^2$ times identity vector $[1,1,1,1...]$ \* - of any function $f$ of $\mathbf{x}$ plus $\epsilon$ is equal to the gradient with respect to $x$ of the same function of $\mathbf{x}$.
  
  If we break that down:
  
  $$\lim\limits\_{\sigma \rightarrow 0}$$
  
  > The limit, as standard deviation $\sigma$ tends towards zero of
  
  $$\nabla\_\mathbf{x}$$
  
  > the gradient with respect to vector $\mathbf{x}$ of
  
  $$\mathbb{E}$$
  
  > the expectation ...
  
  $$\mathbb{E}\_{\epsilon \sim \mathcal{N}(0, \sigma^2\mathbf{I})}$$
  
  > [the expectation] - where perturbation $\epsilon$ follows the normal distribution with mean 0 and variance $\sigma^2$ times identity vector $[1,1,1,1...]$ \* - of
  
  $$f(\mathbf{x} +\epsilon)$$
  
  > any function $f$ of $x$ plus $\epsilon$
  
  $$ = \nabla\_x f(\mathbf{x})$$
  
  > is equal to the gradient with respect to $\mathbf{x}$ of the same function $f(\mathbf{x})$.
  
  Basically it says that taking small perturbations of a vector input to a function and measuring the gradient at those varied points can be used to give you a valid estimate of the true gradient at the point you are making variations of.
  
  In terms of understanding the equations, read introductory texts to the research area, and if like me your maths has been unused for many years before attempting this, expect to spend time and effort. Re-read the equations, memorise and write out the basic ones from the field, apply them to simple problems that might be presented in text books. Reading maths equations is not much different to reading music or reading another language - it takes concentration, practice, time and effort to become fluent enough to read an equation and comprehend it. The different research fields can be quite different too, some might be as similar enough to get by with what you know already, others may require learning all over again.
  
  ---
  
  \* I am not 100% certain of the interpretation of $\mathbf{I}$ as an identity vector - a matrix may be more appropriate, which depends on the form of $\mathcal{N}(\mu, \sigma^2)$ when handling vector distributions. A matrix form for the second argument would be more general and allow for covariance, although the use of $\mathbf{I}$ would then explicitly remove covariance and make each component of $\epsilon$ independent, which is required for this result.
  
  > **Comments:**
  > **username_1**: @DuttaA: Actually I am not sure, it depends on the form of second argument $\mathcal{N}$, which I am assuming will just take a vector of variances for independent vector components here, but you are right it could also support some forms of covariance in general, and we'd want an identity matrix instead to make the variables independent. Do you have a reference to the common assumptions of that function?  
  
  ---
  
  