<p>I recently saw a paper where they run a regression of the covariate of interest <span class="math-container">$X_{1}$</span> on the outcome <span class="math-container">$Y\in\{0,1\}$</span> without further controls
<span class="math-container">$$ 
Y = \alpha_{1} + \beta_{1} X_{1} + \epsilon_{1} \tag{1}\label{1}
$$</span>
where <span class="math-container">$Y$</span> and <span class="math-container">$X_{1}$</span> are such that one may wonder whether <span class="math-container">$E[\epsilon_{1} |X_{1}]=0 $</span> holds. Then they argue that they decompose <span class="math-container">$E[Y|X_{1}]$</span> into an observed and an unobserved part by regressing
<span class="math-container">$$ 
X_{1} = \alpha_{2} + \beta Z + \epsilon_{2} \tag{2}\label{2}
$$</span>
where <span class="math-container">$Z$</span> is a matrix of predictors of <span class="math-container">$X_{1}$</span> and <span class="math-container">$\beta$</span> the respective parameters (not necessarily instruments, indeed I would suppose most <span class="math-container">$Z_{1}, ..., Z_{n}$</span> would not satisfy the mean independence restriction <span class="math-container">$E[\epsilon_{1}|Z] = 0$</span>) and calculate two sets of predicted values, namely <span class="math-container">$\widehat{X_{o}}$</span> based on the linear prediction of <span class="math-container">$ \eqref{2} $</span>, as well as <span class="math-container">$\widehat{X_{u}}$</span>
based on the residuals of the linear prediction in <span class="math-container">$ \eqref{2} $</span>. Then they estimate the following two models
<span class="math-container">$$ 
Y = \alpha_{3} + \beta_{2} \widehat{X_{o}} + \epsilon_{3} \tag{3.1}\label{3.1}
$$</span>
<span class="math-container">$$ 
Y = \alpha_{4} + \beta_{3} \widehat{X_{u}} + \epsilon_{4} \tag{3.2}\label{3.2}
$$</span>
to argue that <span class="math-container">$\beta_{2}$</span> gives the <strong>observable</strong> effect of <span class="math-container">$X_{1}$</span> on <span class="math-container">$Y$</span> and <span class="math-container">$\beta_{3}$</span> gives the <strong>unobservable</strong> effect of <span class="math-container">$X_{1}$</span> on <span class="math-container">$Y$</span> and go on to make claims which one is more important. I am puzzled by this procedure, and not sure what to make of it. It looks like control function approach, but there is no guarantee or even mention that <span class="math-container">$E[\epsilon_{1}|Z] = 0$</span>.</p>
<p>Would be interested to hear what others think about this approach of making claims about the portion of observed to unobserved effects? Sources on the method or other papers doing similar &quot;decompositions&quot; are very welcome as well.</p>

CC BY-SA 4.0
2021-03-31T15:23:21.260Z
When you write that $Y\in\{0,1\}$, am I understanding you correctly that $Y$ is binary? If so, I would first of all be interested in why the authors did not use logistic regression or any similar established GLM. Do they explain?
Stephan Kolassa
CC BY-SA 4.0
2021-03-31T15:24:50.923Z
Yes, $Y$ is binary, no, they don't do it and don't explain why, and yes, I agree with you that they should have :)
Papayapap
CC BY-SA 4.0
2021-03-31T15:31:39.193Z
Hm. I am also... puzzled. Can you link to the paper?
Stephan Kolassa
CC BY-SA 4.0
2021-03-31T15:42:25.800Z
As I think this is dubious, I prefer not shaming the authors publicly, I sent you the link via Linkedin. Hope that's fine for you.
Papayapap
CC BY-SA 4.0
2021-04-01T20:19:13.203Z
What exactly is the question here? We can't comment on whether we think mean independence holds without knowing what the variables are or where the data comes from. We can tell you anything about which variables are observable or not for the same reasons. One can certainly decompose a regression in a lot of different ways. Do you want us to comment on that? Maybe clarify your question, tell us more about the data, or at least tell us to make some assumptions.
jmbejara
CC BY-SA 4.0
2021-04-02T12:07:27.133Z
The question is whether you can do this approach of predicting X with other variables and then use the predicted X1 and X2 to argue that these are observed and unobserved effects. I think I mentioned in text the assumptions, which you cannot make?
Papayapap
CC BY-SA 4.0
2021-04-06T15:50:22.670Z
@jmbejara yes, I am interested in the decomposition part.
Papayapap
CC BY-SA 4.0
2021-04-06T17:28:43.163Z
FYI, I think there is a small typo in (3.2). You have a $\beta_4$ and then refer it later as $\beta_3$.
jmbejara
<regression><endogeneity>
Predicting covariates into "observed" and "unobserved" part
Papayapap
# Predicting covariates into "observed" and "unobserved" part

(Asked by: username_0 on 2021-03-31)

I recently saw a paper where they run a regression of the covariate of interest $X\_{1}$ on the outcome $Y\in\{0,1\}$ without further controls
$$
Y = \alpha\_{1} + \beta\_{1} X\_{1} + \epsilon\_{1} \tag{1}\label{1}
$$
where $Y$ and $X\_{1}$ are such that one may wonder whether $E[\epsilon\_{1} |X\_{1}]=0 $ holds. Then they argue that they decompose $E[Y|X\_{1}]$ into an observed and an unobserved part by regressing
$$
X\_{1} = \alpha\_{2} + \beta Z + \epsilon\_{2} \tag{2}\label{2}
$$
where $Z$ is a matrix of predictors of $X\_{1}$ and $\beta$ the respective parameters (not necessarily instruments, indeed I would suppose most $Z\_{1}, ..., Z\_{n}$ would not satisfy the mean independence restriction $E[\epsilon\_{1}|Z] = 0$) and calculate two sets of predicted values, namely $\widehat{X\_{o}}$ based on the linear prediction of $ \eqref{2} $, as well as $\widehat{X\_{u}}$
based on the residuals of the linear prediction in $ \eqref{2} $. Then they estimate the following two models
$$
Y = \alpha\_{3} + \beta\_{2} \widehat{X\_{o}} + \epsilon\_{3} \tag{3.1}\label{3.1}
$$
$$
Y = \alpha\_{4} + \beta\_{3} \widehat{X\_{u}} + \epsilon\_{4} \tag{3.2}\label{3.2}
$$
to argue that $\beta\_{2}$ gives the **observable** effect of $X\_{1}$ on $Y$ and $\beta\_{3}$ gives the **unobservable** effect of $X\_{1}$ on $Y$ and go on to make claims which one is more important. I am puzzled by this procedure, and not sure what to make of it. It looks like control function approach, but there is no guarantee or even mention that $E[\epsilon\_{1}|Z] = 0$.

Would be interested to hear what others think about this approach of making claims about the portion of observed to unobserved effects? Sources on the method or other papers doing similar "decompositions" are very welcome as well.

**Tags:** regression, endogeneity

> **Comments:**
> **username_1**: When you write that $Y\in\{0,1\}$, am I understanding you correctly that $Y$ is binary? If so, I would first of all be interested in why the authors did not use logistic regression or any similar established GLM. Do they explain?  
> **username_0**: Yes, $Y$ is binary, no, they don't do it and don't explain why, and yes, I agree with you that they should have :)  
> **username_1**: Hm. I am also... puzzled. Can you link to the paper?  
> **username_0**: As I think this is dubious, I prefer not shaming the authors publicly, I sent you the link via Linkedin. Hope that's fine for you.  
> **username_2**: What exactly is the question here? We can't comment on whether we think mean independence holds without knowing what the variables are or where the data comes from. We can tell you anything about which variables are observable or not for the same reasons. One can certainly decompose a regression in a lot of different ways. Do you want us to comment on that? Maybe clarify your question, tell us more about the data, or at least tell us to make some assumptions.  
> **username_0**: The question is whether you can do this approach of predicting X with other variables and then use the predicted X1 and X2 to argue that these are observed and unobserved effects. I think I mentioned in text the assumptions, which you cannot make?  
> **username_0**: @username_2 yes, I am interested in the decomposition part.  
> **username_2**: FYI, I think there is a small typo in (3.2). You have a $\beta_4$ and then refer it later as $\beta_3$.  

---

## 0 answers

