<p>A pair of random variables $(Y,X)$ that has a joint bivariate distribution that belongs to the Elliptical Symmetric Family and to the Pearson Family (they overlap), have the property that the associated conditional expectation functions (of $Y$ given $X$ but also of $X$ given $Y$)  are linear (more generally, affine) functions.</p>

<p>Examples include the Normal distribution and Student's $t$-distribution. Other bivariate distributions that have affine conditional expectation functions are the Pareto, Beta, Gamma, F-, Binomial, Poisson, and Negative Binomial.</p>

CC BY-SA 3.0
2016-01-21T17:17:42.363Z
Thanks a lot for your answer. Why is it then, that even when we have absolutely no reason to believe that Y and X have joint normals or marginal normals that we stick to linear regression?
ChinG
CC BY-SA 3.0
2016-01-19T22:44:43.587Z
2016-01-19T22:44:43.587Z
Alecos Papadopoulos
<p>If X and Y are normal, then the distribution of X conditional on Y is:
$$ X | (Y = y) = N(\mu_x + \rho \frac{\sigma_x}{\sigma_y}(y - \mu_y), (1-\rho)^2 \sigma^2_x )$$ </p>

<p>Therefore, $E[X|Y = y] = \mu_x + \rho \frac{\sigma_x}{\sigma_y}(y - \mu_y) = (\mu_x - \mu_y (\rho \frac{\sigma_x}{\sigma_y})) + (\rho \frac{\sigma_x}{\sigma_y})y $ which is linear in $y$.
Symmetrically:
$$E[Y|X = x] = (\mu_y - \mu_x (\rho \frac{\sigma_y}{\sigma_x})) + (\rho \frac{\sigma_y}{\sigma_x})x   $$
which is also linear in $x$</p>

CC BY-SA 3.0
2016-01-19T20:05:38.760Z
Thanks a lot..how did you get the first step? Is it a property of normal distributions?
ChinG
CC BY-SA 3.0
2016-01-19T20:08:47.880Z
In fact from your equation, one can clearly see that the coefficient is simply the cov(x,y)/var(x). Thanks!
ChinG
CC BY-SA 3.0
2016-01-19T20:24:55.997Z
It is a well known expression but I checked it against the Wikipedia page https://en.wikipedia.org/wiki/Multivariate_normal_distribution
BKay
CC BY-SA 3.0
2016-01-19T20:04:45.347Z
2016-01-19T20:04:45.347Z
BKay
<p>The conditional mean of $Y$
  given $X$
  is:$$E[Y|X]=\int yf(y|x)dy$$
  What is the relation to the linear model? I have read somewhere that when X
  and $Y$
  are normal (their marginal distributions), then the conditional mean of $Y$
  becomes a linear function of $X$.
  How can I see this?</p>

CC BY-SA 3.0
2016-01-19T19:57:10.693Z
If $X$ and $Y$ are independent then your claim is clearly false so I am guessing you omitted some conditions.
Giskard
CC BY-SA 3.0
2016-01-19T20:03:54.017Z
If X and Y are independent, Y would become a constant function (simply the mean of Y)...I am thinking of non degenerate cases..but thanks
ChinG
CC BY-SA 3.0
2016-01-19T22:05:10.487Z
I was confused because you denote the variable $Y$ and the conditional mean of $Y$ with the same letter in your question. (I think you meant to write "$E[Y|X]$ becomes a linear") Please consider editing this so that the question is easier to understand for future readers.
Giskard
<regression>
Conditional Mean and Linearity
ChinG
# Conditional Mean and Linearity

(Asked by: username_0 on 2016-01-19)

The conditional mean of $Y$
given $X$
is:$$E[Y|X]=\int yf(y|x)dy$$
What is the relation to the linear model? I have read somewhere that when X
and $Y$
are normal (their marginal distributions), then the conditional mean of $Y$
becomes a linear function of $X$.
How can I see this?

**Tags:** regression

> **Comments:**
> **username_1**: If $X$ and $Y$ are independent then your claim is clearly false so I am guessing you omitted some conditions.  
> **username_0**: If X and Y are independent, Y would become a constant function (simply the mean of Y)...I am thinking of non degenerate cases..but thanks  
> **username_1**: I was confused because you denote the variable $Y$ and the conditional mean of $Y$ with the same letter in your question. (I think you meant to write "$E[Y|X]$ becomes a linear") Please consider editing this so that the question is easier to understand for future readers.  

---

## 2 answers

### Answer by username_3 on 2016-01-19. Score: 2 | Accepted answer

If X and Y are normal, then the distribution of X conditional on Y is:
$$ X | (Y = y) = N(\mu\_x + \rho \frac{\sigma\_x}{\sigma\_y}(y - \mu\_y), (1-\rho)^2 \sigma^2\_x )$$

Therefore, $E[X|Y = y] = \mu\_x + \rho \frac{\sigma\_x}{\sigma\_y}(y - \mu\_y) = (\mu\_x - \mu\_y (\rho \frac{\sigma\_x}{\sigma\_y})) + (\rho \frac{\sigma\_x}{\sigma\_y})y $ which is linear in $y$.
Symmetrically:
$$E[Y|X = x] = (\mu\_y - \mu\_x (\rho \frac{\sigma\_y}{\sigma\_x})) + (\rho \frac{\sigma\_y}{\sigma\_x})x $$
which is also linear in $x$

> **Comments:**
> **username_0**: Thanks a lot..how did you get the first step? Is it a property of normal distributions?  
> **username_0**: In fact from your equation, one can clearly see that the coefficient is simply the cov(x,y)/var(x). Thanks!  
> **username_3**: It is a well known expression but I checked it against the Wikipedia page https://en.wikipedia.org/wiki/Multivariate_normal_distribution  

---

### Answer by username_2 on 2016-01-19. Score: 1

A pair of random variables $(Y,X)$ that has a joint bivariate distribution that belongs to the Elliptical Symmetric Family and to the Pearson Family (they overlap), have the property that the associated conditional expectation functions (of $Y$ given $X$ but also of $X$ given $Y$) are linear (more generally, affine) functions.

Examples include the Normal distribution and Student's $t$-distribution. Other bivariate distributions that have affine conditional expectation functions are the Pareto, Beta, Gamma, F-, Binomial, Poisson, and Negative Binomial.

> **Comments:**
> **username_0**: Thanks a lot for your answer. Why is it then, that even when we have absolutely no reason to believe that Y and X have joint normals or marginal normals that we stick to linear regression?  

---

