Why logit transform explanatory variable restricted between $0$ and $1$ only if it is close to $0$ or $1$? In Bayesian Data Analysis 3rd edition chapter 14 on linear regression they write

Since the explanatory variable is restricted to lie between 0 and 1 (recall that we have
excluded uncontested elections from our analysis), it would seem advisable to transform the data, perhaps using the logit transformation, before fitting a linear regression model. In
practice, however, almost all the vote proportions $y_i$ fall between 0.2 and 0.8, so the effect
of such a transformation would be minor.

(emphasis mine)
I think I may understand why one would want to transform the data: If we only have one explanatory variable $x$, then some values of $x$ would take the predicted value outside of $[0,1]$. But in that case I don't understand why this is not a problem when almost all the observed outcomes fall between $[0.2, 0.8]$?

A general approach assumes the relationship between an X and Y is a smooth nonlinear one, and fit that relationship with for example a cubic spline function (natural spline aka restricted cubic spline).  The only reason to pre-transform X is to require fewer parameters in the spline function by making the transformation likely to be more linear, then fit the spline on the pre-transformed variable.  Note that in the [0.2, 0.8] range the logit transformation isn't too nonlinear.
There would be slightly different issues if dealing with the dependent variable.
