<p>There are typically two ways to go about estimating causal relationships in the data: by using exogenous variation in the "right-hand-side" variable, or by using structural estimation. </p>

<p>Unless there's an experiment or a quasi-experiment of some sort, the technique economist's apply is instrumental variable (IV) estimation. The idea is that to find the causal impact of x on y in a situation in which x is partly determined by y itself or by a third variable that also affects y, is to find a source of variation that only has a direct effect on x. </p>

<p>Machine learning techniques are not typically used to estimate causal relationships. Instead, they are often used for forecasting. </p>

<p>Mixing the two: machine learning and causal inference is a focus of current research, but apparently there are no solid, off-the-shelf methods to use. Intuitively, the issue is that in the example above, you will need all of x to produce the best forecast of y, but you want only the exogenous par of x to get a sense of the causal relationship between x and y. </p>

CC BY-SA 3.0
2016-03-15T04:47:54.750Z
I have 2 points 1) Isn't linear regression a means to an end? My objective is to be able to predict accurately future demand at a given price and other exogenous variables. So if a machine learning technique is accurate on multiple samples of hold out data and in comparison linear regression performs poorly, then shouldn't i be using machine learning? And to be fair this is what happening for my case. I have quantity demanded, my price, social media reputation and competitor price, competitor social media reputation. It turns out that Random Forest is performing better than linear regression.
StatguyUser
CC BY-SA 3.0
2016-03-15T04:52:40.640Z
2) Unfortunately i do not have any instrumental variable to to consider an IV regression. So the only option i am left with is to use linear regression. and if i compare predictive accuracy of linear regression with machine learning technique, then machine learning outperforms linear regression on holdout sample. So if any other technique can better measure relationship between quantity demanded and price i.e. causality, then why can't we use that? I mean why restrict to linear regression, IV regression and SEM only?
StatguyUser
CC BY-SA 3.0
2016-03-15T04:58:14.937Z
Re: (1 and 2) Yes, estimation is a means to an end. If your end is forecasting under relatively stable conditions, then you are better off with machine learning. If you do it correctly to avoid over-fitting, it will almost certainly will be a better predictor fo the future unless the circumstances change a lot. However, in changing circumstances you are better off understanding the deep parameters driving the economy. For example, if you are thinking of changing tax rates to a level never seen before, machine learning will typically not be a good way to find the potential effect of this policy
Fix.B.
CC BY-SA 3.0
2016-03-15T06:34:35.183Z
So to be precise, we will need IV regression when our goal is to get exogenous component of predictors and when data is unstable and faces lot of changing circumstances?
StatguyUser
CC BY-SA 3.0
2016-03-16T08:33:45.403Z
I am accepting this answer as you have answered the broad question i had. Thanks
StatguyUser
CC BY-SA 3.0
2016-03-16T17:08:13.073Z
Thanks a lot! Forgot to add that the recent work by Susan Athey is a good starting point for current research relating to these issues.
Fix.B.
CC BY-SA 3.0
2016-03-14T16:43:49.450Z
2016-03-14T16:43:49.450Z
Fix.B.
<p>I know that linear regression is used for calculating price elasticity. If my objective is to estimate the parameters of a causal relationship, can i use machine learning techniques like Random Forest or lets say ARIMA with regressors? Has anyone done this before? I tried searching in jstor.org but didn't find any.</p>

<elasticity><applied-econometrics><pricing>
What statistical techniques can be used for measuring price elasticity
StatguyUser
# What statistical techniques can be used for measuring price elasticity

(Asked by: username_0 on 2016-03-13)

I know that linear regression is used for calculating price elasticity. If my objective is to estimate the parameters of a causal relationship, can i use machine learning techniques like Random Forest or lets say ARIMA with regressors? Has anyone done this before? I tried searching in jstor.org but didn't find any.

**Tags:** elasticity, applied-econometrics, pricing


---

## 1 answers

### Answer by username_1 on 2016-03-14. Score: 1 | Accepted answer

There are typically two ways to go about estimating causal relationships in the data: by using exogenous variation in the "right-hand-side" variable, or by using structural estimation.

Unless there's an experiment or a quasi-experiment of some sort, the technique economist's apply is instrumental variable (IV) estimation. The idea is that to find the causal impact of x on y in a situation in which x is partly determined by y itself or by a third variable that also affects y, is to find a source of variation that only has a direct effect on x.

Machine learning techniques are not typically used to estimate causal relationships. Instead, they are often used for forecasting.

Mixing the two: machine learning and causal inference is a focus of current research, but apparently there are no solid, off-the-shelf methods to use. Intuitively, the issue is that in the example above, you will need all of x to produce the best forecast of y, but you want only the exogenous par of x to get a sense of the causal relationship between x and y.

> **Comments:**
> **username_0**: I have 2 points 1) Isn't linear regression a means to an end? My objective is to be able to predict accurately future demand at a given price and other exogenous variables. So if a machine learning technique is accurate on multiple samples of hold out data and in comparison linear regression performs poorly, then shouldn't i be using machine learning? And to be fair this is what happening for my case. I have quantity demanded, my price, social media reputation and competitor price, competitor social media reputation. It turns out that Random Forest is performing better than linear regression.  
> **username_0**: 2) Unfortunately i do not have any instrumental variable to to consider an IV regression. So the only option i am left with is to use linear regression. and if i compare predictive accuracy of linear regression with machine learning technique, then machine learning outperforms linear regression on holdout sample. So if any other technique can better measure relationship between quantity demanded and price i.e. causality, then why can't we use that? I mean why restrict to linear regression, IV regression and SEM only?  
> **username_1**: Re: (1 and 2) Yes, estimation is a means to an end. If your end is forecasting under relatively stable conditions, then you are better off with machine learning. If you do it correctly to avoid over-fitting, it will almost certainly will be a better predictor fo the future unless the circumstances change a lot. However, in changing circumstances you are better off understanding the deep parameters driving the economy. For example, if you are thinking of changing tax rates to a level never seen before, machine learning will typically not be a good way to find the potential effect of this policy  
> **username_0**: So to be precise, we will need IV regression when our goal is to get exogenous component of predictors and when data is unstable and faces lot of changing circumstances?  
> **username_0**: I am accepting this answer as you have answered the broad question i had. Thanks  
> **username_1**: Thanks a lot! Forgot to add that the recent work by Susan Athey is a good starting point for current research relating to these issues.  

---

