<p>I fully agree with the above answer. However, I would just add that there are approaches that do try to conduct inference in an ensemble / model averaging context. In particular, see Granger and Jeon (2004) &quot;Thick Modeling&quot; (<a href="https://www.sciencedirect.com/science/article/abs/pii/S0264999303000178" rel="nofollow noreferrer">https://www.sciencedirect.com/science/article/abs/pii/S0264999303000178</a>) where they perform model selection by combining multiple models and discuss how to conduct inference using bootstrap techniques. See also a further discussion of this in Castle (2017) (<a href="https://ejpam.com/index.php/ejpam/article/view/2954/492" rel="nofollow noreferrer">https://ejpam.com/index.php/ejpam/article/view/2954/492</a>) which notes that this stems from Clive Granger's work on model combinations in forecasting which dates at least as far back as Bates and Granger (1969).</p>

CC BY-SA 4.0
2020-12-22T17:20:08.070Z
2020-12-22T17:20:08.070Z
Andrew M
<p>Emsemble learning is beneficial in forecasting where statistical adequacy of any model in the ensemble is of limited importance. However, when models are used for inference, statistical adequacy is quite important. The correctnes of the assumed null distribution of the test statistic depends (to a larger or smaller degree) on the statistical adequacy of the model. Violations of some modelling assumptions may lead to distortions of the critical values and thus the test size, making inference unrealiable. This is a common problem known from the model averaging literature. Only statistically adequate models may be averaged when the goal is inference (rather than forecasting), as otherwise the inference is unreliable. My explanation is a bit simplistic, but I hope it conveys the main idea.</p>

CC BY-SA 4.0
2020-12-23T09:47:57.537Z
Thanks! Would be great if you could define statistical adequacy and give an example of it. I guess you mean that we should use just one specification for inference. And on forecasting, we may use more than one specification. Is choosing between a 4-variable VAR model and a 5-variable VAR model a statistical adequacy issue? Both models could satisfy statiscal adequacy, no?
Emmanuel Ameyaw
CC BY-SA 4.0
2020-12-23T10:02:00.773Z
@EmmanuelAmeyaw, I mean statistical adequacy in the sense Aris Spanos does. That is, the fitted model satisfies its own assumptions. E.g. if we assume the model errors are i.i.d. but the model residuals are autocorrelated, heteroskedastic or otherwise non-i.i.d., the model is not statistically adequate. But if all assumptions that we have taken are satisfied, the model is statistically adequate. This can be contrasted to subject-matter adequacy; the latter considers how well the model describes the phenomenon of interest from the subject-matter point of view.
Richard Hardy
CC BY-SA 4.0
2020-12-23T10:09:15.697Z
So then two different specifications can satisfy statistical adequacy, right?
Emmanuel Ameyaw
CC BY-SA 4.0
2020-12-23T10:11:58.083Z
@EmmanuelAmeyaw, yes.
Richard Hardy
CC BY-SA 4.0
2020-12-23T10:42:46.373Z
Alright. Thanks. Lemme ask though if there is a criterion to choose between two or more models that satisfy statistical adequacy? I know RMSE is popular for machine learning folks. Something like that in causal models. Or we just assume that the model we choose to use is the correct one, and hence no need to compare it to other specifications? So my concern was with using ensemble methods if there are model specification uncertainties (not sure if there is a term like that but I am sure you get my point)
Emmanuel Ameyaw
CC BY-SA 4.0
2020-12-23T10:51:02.713Z
@EmmanuelAmeyaw, would you mind posting this as a new question? You could put a link to it here and maybe a link from it to this thread.
Richard Hardy
CC BY-SA 4.0
2020-12-22T11:48:37.353Z
2020-12-22T11:48:37.353Z
Richard Hardy
<p>It is ok to apply ensemble modeling to VAR models? I mean, using several specifications of the VAR model instead of just one specification. So, for example, if you want to check that <span class="math-container">$x_{t-1},...,x_{t-p}$</span> granger causes <span class="math-container">$y_t$</span>, you compare results across the different specifications. If you have 5 specifications and 4 of them say yes, and 1 says no, then we conclude that <span class="math-container">$x_{t-1},...,x_{t-p}$</span> granger cause <span class="math-container">$y_t$</span>. If 4 of the specifications say no, and 1 says yes, we conclude that <span class="math-container">$x_{t-1},...,x_{t-p}$</span> do not granger cause <span class="math-container">$y_t$</span>. This idea is from machine learning. Not sure if there is a paper out there that has used something similar.</p>

<macroeconomics><applied-econometrics>
Applying ensemble modelling to VAR models
Emmanuel Ameyaw
# Applying ensemble modelling to VAR models

(Asked by: username_0 on 2020-12-21)

It is ok to apply ensemble modeling to VAR models? I mean, using several specifications of the VAR model instead of just one specification. So, for example, if you want to check that $x\_{t-1},...,x\_{t-p}$ granger causes $y\_t$, you compare results across the different specifications. If you have 5 specifications and 4 of them say yes, and 1 says no, then we conclude that $x\_{t-1},...,x\_{t-p}$ granger cause $y\_t$. If 4 of the specifications say no, and 1 says yes, we conclude that $x\_{t-1},...,x\_{t-p}$ do not granger cause $y\_t$. This idea is from machine learning. Not sure if there is a paper out there that has used something similar.

**Tags:** macroeconomics, applied-econometrics


---

## 2 answers

### Answer by username_2 on 2020-12-22. Score: 4 | Accepted answer

Emsemble learning is beneficial in forecasting where statistical adequacy of any model in the ensemble is of limited importance. However, when models are used for inference, statistical adequacy is quite important. The correctnes of the assumed null distribution of the test statistic depends (to a larger or smaller degree) on the statistical adequacy of the model. Violations of some modelling assumptions may lead to distortions of the critical values and thus the test size, making inference unrealiable. This is a common problem known from the model averaging literature. Only statistically adequate models may be averaged when the goal is inference (rather than forecasting), as otherwise the inference is unreliable. My explanation is a bit simplistic, but I hope it conveys the main idea.

> **Comments:**
> **username_0**: Thanks! Would be great if you could define statistical adequacy and give an example of it. I guess you mean that we should use just one specification for inference. And on forecasting, we may use more than one specification. Is choosing between a 4-variable VAR model and a 5-variable VAR model a statistical adequacy issue? Both models could satisfy statiscal adequacy, no?  
> **username_2**: @username_0, I mean statistical adequacy in the sense Aris Spanos does. That is, the fitted model satisfies its own assumptions. E.g. if we assume the model errors are i.i.d. but the model residuals are autocorrelated, heteroskedastic or otherwise non-i.i.d., the model is not statistically adequate. But if all assumptions that we have taken are satisfied, the model is statistically adequate. This can be contrasted to subject-matter adequacy; the latter considers how well the model describes the phenomenon of interest from the subject-matter point of view.  
> **username_0**: So then two different specifications can satisfy statistical adequacy, right?  
> **username_2**: @username_0, yes.  
> **username_0**: Alright. Thanks. Lemme ask though if there is a criterion to choose between two or more models that satisfy statistical adequacy? I know RMSE is popular for machine learning folks. Something like that in causal models. Or we just assume that the model we choose to use is the correct one, and hence no need to compare it to other specifications? So my concern was with using ensemble methods if there are model specification uncertainties (not sure if there is a term like that but I am sure you get my point)  
> **username_2**: @username_0, would you mind posting this as a new question? You could put a link to it here and maybe a link from it to this thread.  

---

### Answer by username_1 on 2020-12-22. Score: 2

I fully agree with the above answer. However, I would just add that there are approaches that do try to conduct inference in an ensemble / model averaging context. In particular, see Granger and Jeon (2004) "Thick Modeling" (<https://www.sciencedirect.com/science/article/abs/pii/S0264999303000178>) where they perform model selection by combining multiple models and discuss how to conduct inference using bootstrap techniques. See also a further discussion of this in Castle (2017) (<https://ejpam.com/index.php/ejpam/article/view/2954/492>) which notes that this stems from Clive Granger's work on model combinations in forecasting which dates at least as far back as Bates and Granger (1969).


---

