Can any Models be "Bagged"? I have been learning about "bagging" (bootstrap aggregation) - supposedly, there are many types of statistical models can be bagged together. For example, CART Decision Trees can be "bagged" together to from a Random Forest.
I am guessing that statistical models that are bagged together often lose some of their interpretability. For example, if you were to "bag" several regression models together, I am not sure if the regression coefficients of an ensemble of "bagged" regression models have the same interpretability and explanatory power as a single regression model.
This brings me to my question: Suppose some researchers are interested in fitting a Multi State Markov Model (e.g. https://cran.r-project.org/web/packages/msm/vignettes/msm-manual.pdf) to a very large dataset - suppose this dataset is so large (millions and millions of rows) that a Multi State Model can not be fit to the entire dataset. Thus, the researchers decide to take multiple random samples from this dataset, fit individual Multi State Models to each of these random samples - could these models then somehow be "bagged" together (I am not sure how transition probabilities from different Markov Chains can be aggregated together) and still keep their interpretability?  Or is "bagging" only used for very specific models and instances (e.g. Random Forest)?
Thanks!

You ask "Or is 'bagging' only used for very specific models and instances (e.g. Random Forest)?"
If you truly want to, you can bag any model. The reason that bagging is almost synonymous with random forest is that bagging is  more effective for high-variance models like decision trees. Again, Elements of Statistical Learning (p. 589) writes

Not all estimators can be improved by shaking up the data like this [via bagging].
It seems that highly nonlinear estimators, such as trees, benefit the most.
For bootstrapped trees, $\rho$ [the correlation of estimators] is typically small (0.05 or lower is typical; see
Figure 15.9), while $\sigma^2$ [their variance]
is not much larger than the variance for the original
tree. On the other hand, bagging does not change linear estimates, such
as the sample mean (hence its variance either); the pairwise correlation
between bootstrapped means is about 50% (Exercise 15.4).

This is made more precise in the surrounding text, but this paragraph is a good summary.

Whether a model is "interpretable" is up to you to decide. Elements of Statistical Learning (p. 286) provides some commentary:

Note that when we bag a model, any simple structure in the model is
lost. As an example, a bagged tree is no longer a tree. For interpretation
of the model this is clearly a drawback. More stable procedures like nearest neighbors are typically not affected much by bagging. Unfortunately,
the unstable models most helped by bagging are unstable because of the
emphasis on interpretability, and this is lost in the bagging process.

Regarding memory-constrained computation of Markov Models, I have 2 comments.

*

*The bootstrap sample has a similar memory cost as the original sample. In the naive implementation, you're drawing a random sample equal to the original data size. If you use weighting instead of naively gathering duplicates of the data, this might economize the memory consumption a little, since on average the bootstrap sample includes $1 - \exp(-1) \approx 63\%$ of the original data, but this seems marginal on the whole.


*You can just fit the model out-of-memory. In the past when I've faced this problem, I've used a neural network library, and hand-coded the model. In the NN forward pass, I implemented Baum-Welch in the NN library. Then the backward pass is automatically done via backprop. In this way, we can update the model incrementally using whatever mini-batch size fits in memory.
