Answers.Body: <p>In generalised Linear models, each output variable <span class="math-container">$y_i$</span> is modelled as a distribution from the exponential family, with the hypothesis function <span class="math-container">$h_{\theta}(x)$</span> for a given <span class="math-container">$\theta$</span> as the expected value of <span class="math-container">$y_i$</span> and maximum likelihood estimation is usually the method used to solve GLM's. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-26T02:49:17.830Z
Answers.LastActivityDate: 2020-05-26T02:49:17.830Z
Answers.UserDisplayName: calveeen
Body: <p>Reading through the <a href="http://cs229.stanford.edu/notes/cs229-notes7b.pdf" rel="nofollow noreferrer">CS229 lecture notes</a> on generalised linear models, I came across the idea that a linear regression problem can be modelled as a Gaussian distribution, which is a form of the exponential family. The notes state that <span class="math-container">$h_{\theta}(x)$</span> is equal to <span class="math-container">$E[y | x; \theta]$</span>. However, how can <span class="math-container">$h_{\theta}(x)$</span> be equal to the expectation of <span class="math-container">$y$</span> given input <span class="math-container">$x$</span> and <span class="math-container">$\theta$</span>, since the expectation would require a sort of an averaging to take place?</p>
  <blockquote>
  <p>Given x, our goal is to predict the expected value of <span class="math-container">$T(y)$</span> given <span class="math-container">$x$</span>. In most of our examples, we will have <span class="math-container">$T(y) = y$</span>, so this means we would like the prediction <span class="math-container">$h(x)$</span> output by our learned hypothesis h to satisfy <span class="math-container">$h(x) = E[y|x]$</span>.</p>
  <p>To show that ordinary least squares is a special case of the GLM family of models, consider the setting where the target variable y (also called the response variable in GLM terminology) is continuous, and we model the conditional distribution of y given x as a Gaussian <span class="math-container">$N(\mu,\sigma^2)$</span>. (Here, <span class="math-container">$\mu$</span> may depend <span class="math-container">$x$</span>.) So, we let the ExponentialFamily(<span class="math-container">$\eta$</span>) distribution above be the Gaussian distribution. As we saw previously, in the formulation of the Gaussian as an exponential family distribution, we had μ = η. So, we have
  <span class="math-container">$$h_{\theta}(x) = E[y|x; \theta] = \mu = \eta = \theta^Tx.$$</span></p>
  </blockquote>
  <p><em><strong>EDIT</strong></em></p>
  <p>Upon reading other sources, <span class="math-container">$y_i \sim N(\mu_i, \sigma^2)$</span> meaning that each individual output has it's own normal distribution with mean <span class="math-container">$\mu_i$</span> and <span class="math-container">$h_{\theta}(x_i)$</span> is set as the mean of the normal distribution for <span class="math-container">$y_i$</span>. In that case, then the hypothesis makes sense to be assigned the expectation.</p>
  
Tags: <models><linear-regression><logistic-regression>
Title: Why is the hypothesis function $h_{\theta}(x)$ equivalent to $E[y | x; \theta]$ in generalised linear models?
UserDisplayName: calveeen
ThreadText: # Why is the hypothesis function $h_{\theta}(x)$ equivalent to $E[y | x; \theta]$ in generalised linear models?
  
  (Asked by: username_0 on 2020-05-25)
  
  Reading through the [CS229 lecture notes](http://cs229.stanford.edu/notes/cs229-notes7b.pdf) on generalised linear models, I came across the idea that a linear regression problem can be modelled as a Gaussian distribution, which is a form of the exponential family. The notes state that $h\_{\theta}(x)$ is equal to $E[y | x; \theta]$. However, how can $h\_{\theta}(x)$ be equal to the expectation of $y$ given input $x$ and $\theta$, since the expectation would require a sort of an averaging to take place?
  
  > Given x, our goal is to predict the expected value of $T(y)$ given $x$. In most of our examples, we will have $T(y) = y$, so this means we would like the prediction $h(x)$ output by our learned hypothesis h to satisfy $h(x) = E[y|x]$.
  >
  > To show that ordinary least squares is a special case of the GLM family of models, consider the setting where the target variable y (also called the response variable in GLM terminology) is continuous, and we model the conditional distribution of y given x as a Gaussian $N(\mu,\sigma^2)$. (Here, $\mu$ may depend $x$.) So, we let the ExponentialFamily($\eta$) distribution above be the Gaussian distribution. As we saw previously, in the formulation of the Gaussian as an exponential family distribution, we had μ = η. So, we have
  > $$h\_{\theta}(x) = E[y|x; \theta] = \mu = \eta = \theta^Tx.$$
  
  ***EDIT***
  
  Upon reading other sources, $y\_i \sim N(\mu\_i, \sigma^2)$ meaning that each individual output has it's own normal distribution with mean $\mu\_i$ and $h\_{\theta}(x\_i)$ is set as the mean of the normal distribution for $y\_i$. In that case, then the hypothesis makes sense to be assigned the expectation.
  
  **Tags:** models, linear-regression, logistic-regression
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-05-26. Score: 1
  
  In generalised Linear models, each output variable $y\_i$ is modelled as a distribution from the exponential family, with the hypothesis function $h\_{\theta}(x)$ for a given $\theta$ as the expected value of $y\_i$ and maximum likelihood estimation is usually the method used to solve GLM's.
  
  
  ---
  
  