Question: I read various stats/biostats textbooks, including Casella and Lehmann's book chapter on regression. Most of time, the textbook will report a *p*-value for significance of the parameters after regressing against some model. Then there is a model selection procedure followed afterwards.

However, those books will never touch upon cross validation (CV) or talk about using a test/training split. I learned CV and [Monte Carlo cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics)#Repeated_random_sub-sampling_validation) (MCCV) from machine learning books and rarely have I seen stat books covering CV.

Why were we not taught cross validation in stats? Or is it not practiced by statisticians in general? Or somehow that model selection procedure becomes superior to using testing data for model selection? Does a biostatistician/practicing statistician use CV in model selection in general?

Answer: 
I can't say with 100% certainty, but I can give you my two cents.

Let's start with the difference in philosophy between statistics (as practised in the books mentioned) and machine learning. The former is (usually, but not always) concerned with some sort of inference. There is usually a latent parameter, like the sample mean or the effect of a novel drug, which requires estimation as well as a statement on the precision of the estimate. The latter (usually, but not always) eschews estimating anything except the conditional mean (be it a regression or a probability in the case of some classification models).

Thus "model selection" in each context means something slightly different by virtue of having different goals. Cross validation is a means of selecting models by means of estimating their generalization error. It is, therefore, primarily a tool for predictive modelling. But statistics (again usually, but always) is not concerned with generalization error as much as it is concerned with parsimony (for example), hence statistics don't use CV to select models.

Prediction from a statisticians point of view is not absent of cross validation. Indeed, Frank Harrell's *Regression Modelling Strategies* mentions the technique, but that book is primarily concerned with the development of predictions models for use in the clinic.
**Descriptive vs. predictive**

In, say biology textbooks, the focus is on *describing* the relevant data; the sample mean was…, the p-value for this result is… etc.

In machine learning texts, the focus is on producing models that can *generalize* beyond their training set; thus techniques like cross-validation are required in order to get a handle on that feature.

This oversimplifies a bit, but I think captures the essence of the difference in approach that some works take relative to others. These differences are a reflection of the different priorities and goals for these different domains that apply statistics.

Consider the [*Theory of Point Estimation* by Lehmann and Casella 1998](https://www.ctanujit.org/uploads/2/5/3/9/25393293/_theory_of_point_estimation.pdf). They start chapter 1 with the statement "Statistics is concerned with the collection of data and with their analysis and interpretation.", which reads to me that these authors have chosen to focus more on applying statistics for analysis and interpretation rather than on applying statistics in a manner that can be reliably generalized to other cases. Similarly, they describe the result of "classical inference and decision theory" as " Such a statement about [the estimated values of the model parameters] can be viewed as a summary of the information provided by the data and may be used as a guide to action." -- again the focus is more about how to describe the data (any maybe as a basis, with some interpretation, on how to guide actions), and less about the nature of the problems involved in ML.

Finally, let me emphasize: this is a matter of what particular authors, problem domains etc. have chosen to focus on in different areas.