Answers.Body: <p>Not necessarily. The neural network (or whatever else you use) is a <em>model</em> of what you are trying to do, and usually models are not able to perfectly model reality, as it is too complex. A noise term is generally used to represent that, ie the imperfection of the model's relationship with the actual world.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-31T08:56:39.453Z
Answers.LastActivityDate: 2021-03-31T08:56:39.453Z
Answers.UserDisplayName: Oliver Mason
Answers.Body: <p>That equation is just an assumption that we make about the relationship between a <em>response variable</em> (aka <a href="https://en.wikipedia.org/wiki/Dependent_and_independent_variables#Statistics_synonyms" rel="nofollow noreferrer"><em>dependent variable</em></a>) <span class="math-container">$y$</span> and a <em>predictor</em> (aka <a href="https://en.wikipedia.org/wiki/Dependent_and_independent_variables#Statistics_synonyms" rel="nofollow noreferrer"><em>independent variable</em></a>) <span class="math-container">$x$</span>, i.e. the response variable (target) is an <strong>unknown</strong> function <span class="math-container">$f$</span> of the predictor <span class="math-container">$x$</span> plus some noise <span class="math-container">$\epsilon$</span> due to e.g. measurement errors (caused e.g. by damaged sensors). So, if you have a dataset <span class="math-container">$D = \{(y_i, x_i)\}_{i=1}^N$</span>, you assume that <span class="math-container">$y_i = f(x_i) + \epsilon, \forall i$</span>. The goal (in supervised learning) is then to <strong>estimate</strong> <span class="math-container">$f$</span> with e.g. a neural network <span class="math-container">$\hat{f}_\theta$</span>, so the goal is to find a function <span class="math-container">$\hat{f}_\theta$</span> such that <span class="math-container">$\hat{f}_\theta(x_i) = y_i$</span>, so, in practice, you often ignore <span class="math-container">$\epsilon$</span> because that is associated with irreducible errors.</p>
  <p>You can find that equation on page 16 of the book <a href="https://www.statlearning.com/" rel="nofollow noreferrer">An Introduction to Statistical Learning</a>. There you will also find more info about the goal of (statistical) supervised learning and why <span class="math-container">$\epsilon$</span> is irreducible.</p>
  <p>So, the answer to your question is <strong>no</strong>, given that <span class="math-container">$f$</span> there is not the neural network but an unknown function. If your neural network <span class="math-container">$\hat{f}$</span> was equal to <span class="math-container">$f$</span>, then, yes, but, of course, in practice, this will almost never be the case.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-31T11:15:27.300Z
Answers.Text: thanks for answering! How does this apply to the Kalman Filter where we say that $y = Hx + r$, where r is the measurement noise. If there is no measurement noise, then the observations can be fully derived from the state?
Answers.UserDisplayName: f_3464gh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-31T12:11:32.167Z
Answers.Text: @m_3464gh I'm not currently familiar with the details of the Kalman filter, so I can't answer your question. I was about to suggest you to ask a separate question, but you have already done that [here](https://ai.stackexchange.com/q/27100/2444).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-31T09:51:36.603Z
Answers.LastActivityDate: 2021-03-31T09:57:05.190Z
Answers.LastEditDate: 2021-03-31T09:57:05.190Z
Answers.UserDisplayName: nbro
Body: <p>I wonder if the following equation (you can find it in almost every ML book) refers to a general assumption that we make when using machine learning:</p>
  <p><span class="math-container">$$y = f(x)+\epsilon,$$</span></p>
  <p>where <span class="math-container">$y$</span> is our output, <span class="math-container">$f$</span> is e.g. a neural network and <span class="math-container">$\epsilon$</span> is an independent noise term.</p>
  <p>Does this mean that we assume the <span class="math-container">$y$</span>'s contained in our training data set come from a noised version of our network output?</p>
  
Tags: <machine-learning><deep-learning><statistical-ai>
Title: Is the target assumed to be a noisy version of the output of the model in machine learning?
UserDisplayName: f_3464gh
ThreadText: # Is the target assumed to be a noisy version of the output of the model in machine learning?
  
  (Asked by: username_0 on 2021-03-31)
  
  I wonder if the following equation (you can find it in almost every ML book) refers to a general assumption that we make when using machine learning:
  
  $$y = f(x)+\epsilon,$$
  
  where $y$ is our output, $f$ is e.g. a neural network and $\epsilon$ is an independent noise term.
  
  Does this mean that we assume the $y$'s contained in our training data set come from a noised version of our network output?
  
  **Tags:** machine-learning, deep-learning, statistical-ai
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2021-03-31. Score: 3
  
  Not necessarily. The neural network (or whatever else you use) is a *model* of what you are trying to do, and usually models are not able to perfectly model reality, as it is too complex. A noise term is generally used to represent that, ie the imperfection of the model's relationship with the actual world.
  
  
  ---
  
  ### Answer by username_2 on 2021-03-31. Score: 1
  
  That equation is just an assumption that we make about the relationship between a *response variable* (aka [*dependent variable*](https://en.wikipedia.org/wiki/Dependent_and_independent_variables#Statistics_synonyms)) $y$ and a *predictor* (aka [*independent variable*](https://en.wikipedia.org/wiki/Dependent_and_independent_variables#Statistics_synonyms)) $x$, i.e. the response variable (target) is an **unknown** function $f$ of the predictor $x$ plus some noise $\epsilon$ due to e.g. measurement errors (caused e.g. by damaged sensors). So, if you have a dataset $D = \{(y\_i, x\_i)\}\_{i=1}^N$, you assume that $y\_i = f(x\_i) + \epsilon, \forall i$. The goal (in supervised learning) is then to **estimate** $f$ with e.g. a neural network $\hat{f}\_\theta$, so the goal is to find a function $\hat{f}\_\theta$ such that $\hat{f}\_\theta(x\_i) = y\_i$, so, in practice, you often ignore $\epsilon$ because that is associated with irreducible errors.
  
  You can find that equation on page 16 of the book [An Introduction to Statistical Learning](https://www.statlearning.com/). There you will also find more info about the goal of (statistical) supervised learning and why $\epsilon$ is irreducible.
  
  So, the answer to your question is **no**, given that $f$ there is not the neural network but an unknown function. If your neural network $\hat{f}$ was equal to $f$, then, yes, but, of course, in practice, this will almost never be the case.
  
  > **Comments:**
  > **username_0**: thanks for answering! How does this apply to the Kalman Filter where we say that $y = Hx + r$, where r is the measurement noise. If there is no measurement noise, then the observations can be fully derived from the state?  
  > **username_2**: @m_3464gh I'm not currently familiar with the details of the Kalman filter, so I can't answer your question. I was about to suggest you to ask a separate question, but you have already done that [here](https://ai.stackexchange.com/q/27100/2444).  
  
  ---
  
  