Question: I read various stats/biostats textbooks, including Casella and Lehmann's book chapter on regression. Most of time, the textbook will report a *p*-value for significance of the parameters after regressing against some model. Then there is a model selection procedure followed afterwards.

However, those books will never touch upon cross validation (CV) or talk about using a test/training split. I learned CV and [Monte Carlo cross-validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics)#Repeated_random_sub-sampling_validation) (MCCV) from machine learning books and rarely have I seen stat books covering CV.

Why were we not taught cross validation in stats? Or is it not practiced by statisticians in general? Or somehow that model selection procedure becomes superior to using testing data for model selection? Does a biostatistician/practicing statistician use CV in model selection in general?

Answer: 
I can't say with 100% certainty, but I can give you my two cents.

Let's start with the difference in philosophy between statistics (as practised in the books mentioned) and machine learning. The former is (usually, but not always) concerned with some sort of inference. There is usually a latent parameter, like the sample mean or the effect of a novel drug, which requires estimation as well as a statement on the precision of the estimate. The latter (usually, but not always) eschews estimating anything except the conditional mean (be it a regression or a probability in the case of some classification models).

Thus "model selection" in each context means something slightly different by virtue of having different goals. Cross validation is a means of selecting models by means of estimating their generalization error. It is, therefore, primarily a tool for predictive modelling. But statistics (again usually, but always) is not concerned with generalization error as much as it is concerned with parsimony (for example), hence statistics don't use CV to select models.

Prediction from a statisticians point of view is not absent of cross validation. Indeed, Frank Harrell's *Regression Modelling Strategies* mentions the technique, but that book is primarily concerned with the development of predictions models for use in the clinic.
### Probability

If you consider a really strict delineation of **probability** and statistics, the former is about mathematically describing how likely it is for an event to occur, or a proposition to be true. You can have a textbook or a course that is about probability, without entering the field of statistics at all.

Classical examples include drawing different colored balls out of an urn, combinations in a lottery, or drawing cards from a deck.

### Statistics

**Statistics**, then, is about describing either probability distributions, populations or samples drawn from a population. Parameters that can be used to describe those are, for example, *mean* and *standard deviation*. In this sense, statistics is about describing the results of observations of random variables, or any sample or population that is not necessarily random.

A textbook that takes this view of statistics would include the definitions for those terms, and then various estimators that can be used to get at the parameters that might have produced a certain sample (given a probability distribution, or a random process), and how to judge the correctness of those estimates.

Now, it is entirely plausible that a textbook would stay entirely within this definition of statistics: Describing populations or samples, and using probability distributions to make inference on how like it is that we saw a certain sample -- without entering the world of **statistical modeling**, where cross validation belongs.

### Why not cross validation?

Some textbooks, even holding the view described above, might still include linear regression: its parameters can still be considered estimates that can be calculated from a sample. It can be, of course, used as a predictive model, and thus subjected to [cross validation](https://stats.stackexchange.com/q/122476/107191) -- but once you start using cross validation to make judgements about what terms to include in your model, you step away from the strict definition of the parameters of the linear model being estimates of the population, calculated from a sample drawn from it.

Thus you could say that cross validation is already venturing in to the field of **applied statistics**.