Answers.Body: <p>It is not saying that the <span class="math-container">$t_n$</span> are independent of one another but that <span class="math-container">$t_n|y_n$</span> are independent.</p>
  <p>The only variation in the target values <span class="math-container">$t_n$</span> once you've supplied the <span class="math-container">$y_n$</span> is given by <span class="math-container">$\epsilon_n$</span>. In other words, the target values are independent <em>given</em> the <span class="math-container">$y$</span> values. The <span class="math-container">$y$</span> values are not necessarily independent. Their covariance is determined by <span class="math-container">$\textbf{K}$</span>, the Gram matrix derived from the kernel function. The <span class="math-container">$t$</span> values are also not independent in general. It is only after you've supplied the appropriate <span class="math-container">$y$</span> that a <span class="math-container">$t$</span> becomes independent of its fellows.</p>
  <p>Notes:</p>
  <ul>
  <li><span class="math-container">$\beta$</span> is a <em>scalar</em> hyperparameter representing the shared precision of the noise. <span class="math-container">$\beta^{-1}$</span> is another way of writing <span class="math-container">$\frac{1}{\beta}$</span>, the shared variance of the noise.</li>
  <li>I'm using the 2006 edition of Bishop. This comes from section 6.4.2 (Gaussian processes for regression).</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-06T20:03:36.567Z
Answers.LastActivityDate: 2024-01-06T20:03:36.567Z
Answers.UserDisplayName: Eponymous
Body: <p>From Bishop's Pattern Recognition and Machine Learning:</p>
  <blockquote>
  <p><span class="math-container">$t_n = y_n + \epsilon_n$</span>, where <span class="math-container">$\epsilon_n$</span> is a random noise variable whose value is chosen independently for each observation <span class="math-container">$n$</span>. Consider</p>
  </blockquote>
  <p><span class="math-container">$$p(t_n|y_n)= {\cal N}(t_n|y_n,\beta^{-1})$$</span></p>
  <blockquote>
  <p>Because noise is independent for each data point, we have</p>
  </blockquote>
  <p><span class="math-container">$$p(\textbf t | \textbf{y}) = {\cal N} (\textbf t|\textbf y,\beta^{-1}I_N)$$</span></p>
  <blockquote>
  <p>where <span class="math-container">$\textbf t = (t_1,...,t_N)^T$</span>, and <span class="math-container">$\textbf y =
  &gt; (y_1,....,y_N)^T$</span></p>
  </blockquote>
  <p>Question: Why the covariance matrix is equal to <span class="math-container">$\beta^{-1}I_N$</span>? I know if two random variables are marginal independent then their covariance is zero (identity in this case), but it only gives <span class="math-container">$\epsilon_n$</span> are independent of each other, not <span class="math-container">$t_n$</span> are independent of each other. I hope someone could clarify this question, please.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-05T13:20:40.003Z
Comments.Text: What is $\beta^{-1}$ here? It might also be a good idea to define $t$ and $y$. $\epsilon$ is clear. Thanks.
Comments.UserDisplayName: nbro
Tags: <probability-distribution><random-variable><probability-theory><gaussian-process>
Title: Independence of random variable in Gaussian Process context
UserDisplayName: piero
ThreadText: # Independence of random variable in Gaussian Process context
  
  (Asked by: username_0 on 2024-01-05)
  
  From Bishop's Pattern Recognition and Machine Learning:
  
  > $t\_n = y\_n + \epsilon\_n$, where $\epsilon\_n$ is a random noise variable whose value is chosen independently for each observation $n$. Consider
  
  $$p(t\_n|y\_n)= {\cal N}(t\_n|y\_n,\beta^{-1})$$
  
  > Because noise is independent for each data point, we have
  
  $$p(\textbf t | \textbf{y}) = {\cal N} (\textbf t|\textbf y,\beta^{-1}I\_N)$$
  
  > where $\textbf t = (t\_1,...,t\_N)^T$, and $\textbf y =
  > > (y\_1,....,y\_N)^T$
  
  Question: Why the covariance matrix is equal to $\beta^{-1}I\_N$? I know if two random variables are marginal independent then their covariance is zero (identity in this case), but it only gives $\epsilon\_n$ are independent of each other, not $t\_n$ are independent of each other. I hope someone could clarify this question, please.
  
  **Tags:** probability-distribution, random-variable, probability-theory, gaussian-process
  > **Comments:**
  > **username_1**: What is $\beta^{-1}$ here? It might also be a good idea to define $t$ and $y$. $\epsilon$ is clear. Thanks.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-01-06. Score: 1
  
  It is not saying that the $t\_n$ are independent of one another but that $t\_n|y\_n$ are independent.
  
  The only variation in the target values $t\_n$ once you've supplied the $y\_n$ is given by $\epsilon\_n$. In other words, the target values are independent *given* the $y$ values. The $y$ values are not necessarily independent. Their covariance is determined by $\textbf{K}$, the Gram matrix derived from the kernel function. The $t$ values are also not independent in general. It is only after you've supplied the appropriate $y$ that a $t$ becomes independent of its fellows.
  
  Notes:
  
  * $\beta$ is a *scalar* hyperparameter representing the shared precision of the noise. $\beta^{-1}$ is another way of writing $\frac{1}{\beta}$, the shared variance of the noise.
  * I'm using the 2006 edition of Bishop. This comes from section 6.4.2 (Gaussian processes for regression).
  
  
  ---
  
  