Are consistently negative Efron's pseudo-r2 in logistic regression possible? I am conducting logistic regression and looking to calculate pseudo-R2 values alongside AIC and BIC for model evaluation. I selected Efron's pseudo-R2 because of its simple calculation and the similarity to a proper R2 value. When I run a series of logistic regressions, it produces a negative value. However, it is commonly stated that pseudo-R2 values fall between 0 and 1 (example here). Am I calculating something wrong, or is this 0 to 1 range false for Efron's pseudo-R2?
The equation I am using for Efron's pseudo-R2 is: $ R^2 = 1 - \frac{\sum(y_i-\pi_i)^2}{\sum(y_i-\bar y)^2} $
Where:

*

*$ y $ is an array of 1s and 0s, representing the true outcome labels in the data

*$ \pi $ is an array of 1s and 0s, representing the predicted outcome labels as a result of the logistic regressions

*$ \bar y $ is the arithmetic mean of $ y $, calculated as $ \frac{\sum y_i}{n} $ and equivalent to $ p $ or the probability of a 1 outcome

This seems to be the definition consistently given online. (I don't have access to the foundational journal article.)
Looking at the equation, this appears to be happening for three reasons: (1) because of large sample size (n = 4,000), (2) relative inaccuracy of the logistic regression, and (3) the fact that the value for p (mean y = ) is close to 0.5. The large sample size combined with frequent error blow up the numerator and having p close to 0.5 shrinks the denominator. Indeed, when I calculate the value on a subset of my data (about the first 100 rows), I receive a positive pseudo-R2.
Again, though, pseudo-R2 are discussed as being consistently between 0 and 1. It seems there are two possibilities: (1) this range is a simplification and it is the case that a bad model can be negative (as with a conventional R2) or (2) there is something wrong with my understanding.
More information on implementation, although I believe my implementation is correctly reflecting the equation above: I am using the python sklearn implementation, which has limited metrics for inference, so I am writing functions to calculate pseudo-R2 values. I wrote the function based on the formula above and confirmed another function from this site returns the same values, leading me to believe this is a characteristic of the metric and not of an incorrect implementation of the equation.

I think it’s important to remember what $R^2$ means in the linear case.
$$R^2=
1-\dfrac{
\sum\bigg(
y_i-\hat y_i
\bigg)^2
}{\sum\bigg(
y_i-\bar y
\bigg)^2
}
$$
If we want to measure our ability to predict conditional means by how low of a square loss we have, we better have lower square loss than the naïve model that guesses $\bar y$ every time!
This is exactly what is going on in the pseudo-$R^2$. If you do a worse job of predicting the conditional probability (not label) than a naïve model that always predicts the overall prevalence, then the numerator is larger than the denominator, resulting in pseudo-$R^2<0$. In the case of a probability model, square loss is called Brier score and is not the usual loss function. Brier score is, however, a strictly proper scoring rule, which means, a little loosely speaking, that it seeks out the true conditional probability values.
The typical loss function in logistic regression is log loss, which corresponds to maximum likelihood estimation of the coefficients. It makes sense to compare the log loss values in a similar way. This is McFadden’s $R^2$.
Indeed, I say that it always makes sense to compare how your model does on a loss function of interest compared to some baseline model. In OLS linear regression, there is a convenient interpretation about the “proportion of variance explained”, but even if we lack such an interpretation, comparing our performance to the performance of a baseline model gives us some idea of if our model provides value.
UCLA has a nice webpage about $R^2$-style metrics for probability models like logistic regression.
Vanderbilt's Frank Harrell has some thoughts on how to measure the value added by a model.
EDIT
Worth remembering is that, if the logistic regression coefficients are estimated by minimizing log loss (equivalent to maximum likelihood estimation), even the in-sample Efron $R^2$ can be less than one, since the objective of the coefficient estimation is not to minimize Brier score but log loss. In contrast, this cannot happen for in-sample McFadden's $R^2$ that addresses the explicit objective function used to estimate the model, except for computational funkiness.
