Linear regression: F-test for lack of fit (using ANOVA to test regression model) - intuition? This is the data set:
https://raw.githubusercontent.com/FlorinAndrei/misc/master/stackexchange_attach/normtemp.csv
I'm looking at a simple linear regression. The data is quite trivial.
import pandas as pd
from statsmodels.formula.api import ols

nt = pd.read_csv('normtemp.csv')
model = ols('hr ~ temp', data=nt).fit()
print(nt.head())

Output:
   temp  hr
0  96.3  70
1  96.7  71
2  96.9  74
3  97.0  80
4  97.1  73

To test whether there is lack of fit, I've seen the F-test done this way:
from statsmodels.stats.api import anova_lm

nt_fact = nt.copy()
nt_fact['temp'] = nt_fact['temp'].astype(object)
model_fact = ols('hr ~ temp', data=nt_fact).fit()

an_out = anova_lm(model, model_fact)
print(an_out)

Output:
   df_resid          ssr  df_diff      ss_diff         F    Pr(>F)
0     126.0  6009.601776      0.0          NaN       NaN       NaN
1      96.0  4177.428932     30.0  1832.172843  1.403484  0.110304

Same thing in R:
linear.model <- lm( hr ~ temp)
full.model <- lm(hr ~ factor(temp))
anova(linear.model, full.model)

The p-value from ANOVA is 0.1103, do not reject H0, there is not significant evidence that the linear model is not appropriate.
EDIT: This is not about overfitting. This is to check whether the linear model is or is not a good fit for the data. Rejecting H0 from ANOVA would indicate the linear model is not a good fit for the data.
I am blanking completely on the intuitive explanation for why this works. We build two models here - the actual LR model where the predictor is continuous, and another model where the predictor is converted to categorical (and that's the only change) but the data is otherwise the same.
What exactly are we comparing here with ANOVA, and why rejecting H0 would indicate the fit is not good? (or the other way around)

ANOVA is not a method for testing whether a model fit is ok, but for comparing different nested models. It is usually used to identify statistically significant predictors, but there are cases when this fails, most notably in the presence of (multi-)collinearity. "Nested models" are models for which one predictor set is a subset of the predictors of the other model. This is not the case in your comparison between your linear.model and full.model, which do not even have a single common predictor.
As a first step in evaluating your model, you should look at $R^2$ and the overall F-statistic. With all the data form your given reference the result is:
> fit <- lm(hr ~ temp, data=x)
> summary(fit)
Coefficients:
             Estimate Std. Error t value Pr(>|t|)   
(Intercept) -179.1193    87.8417  -2.039   0.0435 * 
temp           2.5742     0.8944   2.878   0.0047 **

Multiple R-squared:  0.06169,   Adjusted R-squared:  0.05424 
F-statistic: 8.284 on 1 and 126 DF,  p-value: 0.004699

$R^2$ is suspiciously low and the model does a lousy predictive job, which is confirmed by plotting the data and the fitted model line:

The coefficient of temp is statistically significant (p-value of F-statistic is 0.0047, as is the overall F-statistic), however, so you can at least conclude that hs increases with temp, albeit wildly scattering around the predicted values.
The latter point is also confirmed by the very small $R^2$, which says that the random fluctuations in your model greatly dominate the systematic effects. The random fluctuations, on the other hand, seem to be consistent with a linear model, i.e., seem to be normal, which you can visualize with a QQ-Plot.
