Answers.Body: <p>If you take a look at the Wikipedia page related to the <a href="https://en.wikipedia.org/wiki/Normal_distribution" rel="nofollow noreferrer">normal distribution</a>, you will see the definition of the Gaussian density</p>
  <p><span class="math-container">$$
  {\displaystyle f(x)={\frac {1}{\sigma {\sqrt {2\pi }}}}e^{-{\frac {1}{2}}\left({\frac {x-\mu }{\sigma }}\right)^{2}}} \label{1}\tag{1}
  $$</span></p>
  <p>and you will see that the <span class="math-container">$y$</span> in your formula corresponds to the <span class="math-container">$x$</span> in equation \ref{1}.</p>
  <p>I've seen this notation in the context of computer vision and image processing, where the Gaussian kernel is used to blur images.</p>
  <p>So, as pointed out by someone in a comment, <span class="math-container">$y$</span> should indeed be the point where you evaluate the density.</p>
  <p>Maybe the confusing part is that all parameters are treated equally in terms of their purpose, while <span class="math-container">$\mu$</span> and <span class="math-container">$\sigma$</span> are clearly the parameters that define the specific density, so they are not the inputs to the specific density.</p>
  <p>After having read the relevant section of the paper, I now understand why you're confused. The author refers to <span class="math-container">$y$</span> as the <em>output</em> (not yet sure why: maybe it's the output of another unit that feeds this Gaussian unit?), but I think that this explanation still applies. The output of the Gaussian density <span class="math-container">$g$</span> is not <span class="math-container">$y$</span>, but the density that corresponds to <span class="math-container">$y$</span>. In fact, in appendix <span class="math-container">$B$</span> of the paper, the author says that <span class="math-container">$Y$</span> is the <a href="https://planetmath.org/supportoffunction" rel="nofollow noreferrer"><strong>support</strong></a> of <span class="math-container">$g$</span> and <span class="math-container">$y$</span> is an element of <span class="math-container">$Y$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-11T01:45:53.287Z
Answers.Text: So, in other words, $f(x)$ simply returns the probability of input value $x$ with respect to a Gaussian distribution parameterized by $\mu$ and $\sigma$. Is that correct?
Answers.UserDisplayName: Daniel B.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-11T01:55:05.467Z
Answers.Text: $f(x)$ is not a _probability_ but a [_density_ value](https://en.wikipedia.org/wiki/Probability_density_function), i.e. it's not necessarily a number in the range $[0, 1]$. Remember that, for continuous r.v.s., the probability that it takes a specific value is _zero_. So, with continuous r.v.s, you typically want to calculate the probability that the r.v. lies in some range. Note also that when people use the term "probability distribution", they may refer to the pdf, if the r.v. is continuous. Maybe check [this answer](https://ai.stackexchange.com/a/16842/2444).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-11T02:27:34.227Z
Answers.Text: Ok, thanks a lot! The referred answer helps quite a lot as well!
Answers.UserDisplayName: Daniel B.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-13T02:13:50.927Z
Answers.Text: A crucial bit I just overlooked from the paper (page 14) was "density function $g$ determining the output $y$". So, $y$ is indeed a sample from a Gaussian parameterized by $\mu$ and $\sigma$. I *suspect* that the term $y$ is only shown in that definition of the density function $g$, since $g$'s partial derivative (in this way including $y$ as well) is used to compute the gradient of the policy network's output nodes having produced the parameterization which eventually led to the sampling of $y$ (where those nodes shall be updated also considering the "quality" of the drawn sample $y$).
Answers.UserDisplayName: Daniel B.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-13T08:26:34.377Z
Answers.Text: @DanielB. Yes, I think you're right, $y$ can be a sample from the Gaussian distribution (so, in a way, an "output" from the Gaussian distribution), but it can also denote an input to the Gaussian density. So, the samples from the Gaussian distribution should indeed be values that you can evaluate the Gaussian density at. For example, let's say that you have a Gaussian with $\mu = 0$ and $\sigma = 1$. Then you sample $y=0$ (i.e. a number from the Gaussian, which, in this case, happens to be the mean), then you can evaluate the density of $0$ (it would be the height of this Gaussian at zero).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-13T08:45:06.733Z
Answers.Text: He's probably using the letter $y$ rather than $x$ to emphasize that this is a sample (output) from a Gaussian distribution rather than the value you want to evaluate the Gaussian density at, given that $y$, in the context of neural networks and stuff, is usually used to denote an output.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-13T14:20:51.003Z
Answers.Text: I think important is also to consider that the variable $x$ had already been reserved to denote inputs to NN layers constituting the $\mu$ and $\sigma$ generating NN. So, I think the author really intends to adhere to the standard Machine Learning way of referring to inputs and outputs as $x$ and $y$ respectively. And after all that's also pretty accurate; except for the fact that the NN (i.e. the policy network) is not a vanilla NN, but simply one whose regular output layer is appended by an additional layer of random generators generating outputs given the outputs of the previous layer.
Answers.UserDisplayName: Daniel B.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-10T13:47:08.890Z
Answers.LastActivityDate: 2021-01-10T14:59:34.373Z
Answers.LastEditDate: 2021-01-10T14:59:34.373Z
Answers.UserDisplayName: nbro
Answers.Body: <p>It states that &quot;To simplify notation, we focus on one single unit and omit the usual unit index subscript throughout&quot;</p>
  <p>So they are simply removing the i-th index from the equation for simplicity.  So g is a function of a given instance &quot;y&quot; and the parameters μ and σ.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-10T11:17:06.707Z
Answers.Text: But what, in particular, does the 'given instance "y"' refer to?
Answers.UserDisplayName: Daniel B.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-10T11:38:40.453Z
Answers.Text: "y" is just some measurement or data instance.  So if you have a dataset of temperature measurements in the day, "y" could be one single temperature measurement in your dataset, e.g. 20
Answers.UserDisplayName: John Rothman
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-10T11:06:45.823Z
Answers.LastActivityDate: 2021-01-10T11:06:45.823Z
Answers.UserDisplayName: John Rothman
Body: <p>I am wondering what the parameter <span class="math-container">$y$</span> in the function <span class="math-container">$g(y,\mu,\sigma)=\frac{1}{(2\pi)^{1/2}\sigma}e^{-(y-\mu)^{2/2\sigma^2}}$</span> stands for in Section 6 (page 14) of the <a href="https://link.springer.com/content/pdf/10.1007/BF00992696.pdf" rel="nofollow noreferrer">paper</a> introducing the REINFORCE family of algorithms.</p>
  <p>Drawing an analogy to Equation 4 of the same paper, I would guess that it refers to the outcome (i.e. <em>sample</em>) of sampling from a probability distribution parameterized by the parameters <span class="math-container">$\mu$</span> and <span class="math-container">$\sigma$</span>. However, I am not sure whether that is correct or not.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-10T10:13:54.593Z
Comments.Text: As far as I can tell, it is the point at which you would evaluate the density function of the random variable (in this case the normal distn). In a policy gradient setting I imagine that $y$ here is the action. Does this sound correct?
Comments.UserDisplayName: David
Tags: <reinforcement-learning><papers><reinforce><notation>
Title: What does the parameter $y$ stand for in function $g(y,\mu,\sigma)$ related to REINFORCE algorithm?
UserDisplayName: Daniel B.
ThreadText: # What does the parameter $y$ stand for in function $g(y,\mu,\sigma)$ related to REINFORCE algorithm?
  
  (Asked by: username_0 on 2021-01-10)
  
  I am wondering what the parameter $y$ in the function $g(y,\mu,\sigma)=\frac{1}{(2\pi)^{1/2}\sigma}e^{-(y-\mu)^{2/2\sigma^2}}$ stands for in Section 6 (page 14) of the [paper](https://link.springer.com/content/pdf/10.1007/BF00992696.pdf) introducing the REINFORCE family of algorithms.
  
  Drawing an analogy to Equation 4 of the same paper, I would guess that it refers to the outcome (i.e. *sample*) of sampling from a probability distribution parameterized by the parameters $\mu$ and $\sigma$. However, I am not sure whether that is correct or not.
  
  **Tags:** reinforcement-learning, papers, reinforce, notation
  
  > **Comments:**
  > **username_1**: As far as I can tell, it is the point at which you would evaluate the density function of the random variable (in this case the normal distn). In a policy gradient setting I imagine that $y$ here is the action. Does this sound correct?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2021-01-10. Score: 2 | Accepted answer
  
  If you take a look at the Wikipedia page related to the [normal distribution](https://en.wikipedia.org/wiki/Normal_distribution), you will see the definition of the Gaussian density
  
  $$
  {\displaystyle f(x)={\frac {1}{\sigma {\sqrt {2\pi }}}}e^{-{\frac {1}{2}}\left({\frac {x-\mu }{\sigma }}\right)^{2}}} \label{1}\tag{1}
  $$
  
  and you will see that the $y$ in your formula corresponds to the $x$ in equation \ref{1}.
  
  I've seen this notation in the context of computer vision and image processing, where the Gaussian kernel is used to blur images.
  
  So, as pointed out by someone in a comment, $y$ should indeed be the point where you evaluate the density.
  
  Maybe the confusing part is that all parameters are treated equally in terms of their purpose, while $\mu$ and $\sigma$ are clearly the parameters that define the specific density, so they are not the inputs to the specific density.
  
  After having read the relevant section of the paper, I now understand why you're confused. The author refers to $y$ as the *output* (not yet sure why: maybe it's the output of another unit that feeds this Gaussian unit?), but I think that this explanation still applies. The output of the Gaussian density $g$ is not $y$, but the density that corresponds to $y$. In fact, in appendix $B$ of the paper, the author says that $Y$ is the [**support**](https://planetmath.org/supportoffunction) of $g$ and $y$ is an element of $Y$.
  
  > **Comments:**
  > **username_0**: So, in other words, $f(x)$ simply returns the probability of input value $x$ with respect to a Gaussian distribution parameterized by $\mu$ and $\sigma$. Is that correct?  
  > **username_2**: $f(x)$ is not a _probability_ but a [_density_ value](https://en.wikipedia.org/wiki/Probability_density_function), i.e. it's not necessarily a number in the range $[0, 1]$. Remember that, for continuous r.v.s., the probability that it takes a specific value is _zero_. So, with continuous r.v.s, you typically want to calculate the probability that the r.v. lies in some range. Note also that when people use the term "probability distribution", they may refer to the pdf, if the r.v. is continuous. Maybe check [this answer](https://ai.stackexchange.com/a/16842/2444).  
  > **username_0**: Ok, thanks a lot! The referred answer helps quite a lot as well!  
  > **username_0**: A crucial bit I just overlooked from the paper (page 14) was "density function $g$ determining the output $y$". So, $y$ is indeed a sample from a Gaussian parameterized by $\mu$ and $\sigma$. I *suspect* that the term $y$ is only shown in that definition of the density function $g$, since $g$'s partial derivative (in this way including $y$ as well) is used to compute the gradient of the policy network's output nodes having produced the parameterization which eventually led to the sampling of $y$ (where those nodes shall be updated also considering the "quality" of the drawn sample $y$).  
  > **username_2**: @DanielB. Yes, I think you're right, $y$ can be a sample from the Gaussian distribution (so, in a way, an "output" from the Gaussian distribution), but it can also denote an input to the Gaussian density. So, the samples from the Gaussian distribution should indeed be values that you can evaluate the Gaussian density at. For example, let's say that you have a Gaussian with $\mu = 0$ and $\sigma = 1$. Then you sample $y=0$ (i.e. a number from the Gaussian, which, in this case, happens to be the mean), then you can evaluate the density of $0$ (it would be the height of this Gaussian at zero).  
  > **username_2**: He's probably using the letter $y$ rather than $x$ to emphasize that this is a sample (output) from a Gaussian distribution rather than the value you want to evaluate the Gaussian density at, given that $y$, in the context of neural networks and stuff, is usually used to denote an output.  
  > **username_0**: I think important is also to consider that the variable $x$ had already been reserved to denote inputs to NN layers constituting the $\mu$ and $\sigma$ generating NN. So, I think the author really intends to adhere to the standard Machine Learning way of referring to inputs and outputs as $x$ and $y$ respectively. And after all that's also pretty accurate; except for the fact that the NN (i.e. the policy network) is not a vanilla NN, but simply one whose regular output layer is appended by an additional layer of random generators generating outputs given the outputs of the previous layer.  
  
  ---
  
  ### Answer by username_3 on 2021-01-10. Score: 0
  
  It states that "To simplify notation, we focus on one single unit and omit the usual unit index subscript throughout"
  
  So they are simply removing the i-th index from the equation for simplicity. So g is a function of a given instance "y" and the parameters μ and σ.
  
  > **Comments:**
  > **username_0**: But what, in particular, does the 'given instance "y"' refer to?  
  > **username_3**: "y" is just some measurement or data instance.  So if you have a dataset of temperature measurements in the day, "y" could be one single temperature measurement in your dataset, e.g. 20  
  
  ---
  
  