Question: I read various stats/biostats textbooks, including Casella and Lehmann's book chapter on regression. Most of time, the textbook will report a *p*-value for significance of the parameters after regressing against some model. Then there is a model selection procedure followed afterwards.

However, those books will never touch upon cross validation (CV) or talk about using a test/training split. I learned CV and [Monte Carlo cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics)#Repeated_random_sub-sampling_validation) (MCCV) from machine learning books and rarely have I seen stat books covering CV.

Why were we not taught cross validation in stats? Or is it not practiced by statisticians in general? Or somehow that model selection procedure becomes superior to using testing data for model selection? Does a biostatistician/practicing statistician use CV in model selection in general?

Answer: 
### Probability

If you consider a really strict delineation of **probability** and statistics, the former is about mathematically describing how likely it is for an event to occur, or a proposition to be true. You can have a textbook or a course that is about probability, without entering the field of statistics at all.

Classical examples include drawing different colored balls out of an urn, combinations in a lottery, or drawing cards from a deck.

### Statistics

**Statistics**, then, is about describing either probability distributions, populations or samples drawn from a population. Parameters that can be used to describe those are, for example, *mean* and *standard deviation*. In this sense, statistics is about describing the results of observations of random variables, or any sample or population that is not necessarily random.

A textbook that takes this view of statistics would include the definitions for those terms, and then various estimators that can be used to get at the parameters that might have produced a certain sample (given a probability distribution, or a random process), and how to judge the correctness of those estimates.

Now, it is entirely plausible that a textbook would stay entirely within this definition of statistics: Describing populations or samples, and using probability distributions to make inference on how like it is that we saw a certain sample -- without entering the world of **statistical modeling**, where cross validation belongs.

### Why not cross validation?

Some textbooks, even holding the view described above, might still include linear regression: its parameters can still be considered estimates that can be calculated from a sample. It can be, of course, used as a predictive model, and thus subjected to [cross validation](https://stats.stackexchange.com/q/122476/107191) -- but once you start using cross validation to make judgements about what terms to include in your model, you step away from the strict definition of the parameters of the linear model being estimates of the population, calculated from a sample drawn from it.

Thus you could say that cross validation is already venturing in to the field of **applied statistics**.
**Descriptive vs. predictive**

In, say biology textbooks, the focus is on *describing* the relevant data; the sample mean was…, the p-value for this result is… etc.

In machine learning texts, the focus is on producing models that can *generalize* beyond their training set; thus techniques like cross-validation are required in order to get a handle on that feature.

This oversimplifies a bit, but I think captures the essence of the difference in approach that some works take relative to others. These differences are a reflection of the different priorities and goals for these different domains that apply statistics.

Consider the [*Theory of Point Estimation* by Lehmann and Casella 1998](https://www.ctanujit.org/uploads/2/5/3/9/25393293/_theory_of_point_estimation.pdf). They start chapter 1 with the statement "Statistics is concerned with the collection of data and with their analysis and interpretation.", which reads to me that these authors have chosen to focus more on applying statistics for analysis and interpretation rather than on applying statistics in a manner that can be reliably generalized to other cases. Similarly, they describe the result of "classical inference and decision theory" as " Such a statement about [the estimated values of the model parameters] can be viewed as a summary of the information provided by the data and may be used as a guide to action." -- again the focus is more about how to describe the data (any maybe as a basis, with some interpretation, on how to guide actions), and less about the nature of the problems involved in ML.

Finally, let me emphasize: this is a matter of what particular authors, problem domains etc. have chosen to focus on in different areas.