<p>Imbalanced classes present minimal problem to proper statistical methods.</p>
<p>A standard criticism of class imbalance is that it can result in models always or often classifying as the majority class. However, most models don’t actually do classification but output scores on a continuum, such as a logistic regression outputting probability. Software packages default to a cutoff threshold of <span class="math-container">$0.5$</span> probability, but this might be wildly inappropriate for your task (whether the classes are balanced or not). A simple approach is just to adjust the threshold. A more sophisticated approach would directly evaluate the probability outputs with a metric like log loss or Brier score (two examples of so-called “strictly proper scoring rules” that are uniquely minimized in expected value by the true probabilities, so they seek out the true probabilities of events).</p>
<p>This topic comes up so often on the statistics Stack, <a href="https://stats.stackexchange.com">Cross Validated</a>, that I have compiled a list of links to further reading about class imbalance and proper statistical methods that handle class imbalance.</p>
<p><a href="https://stats.stackexchange.com/questions/357466">https://stats.stackexchange.com/questions/357466</a></p>
<p><a href="https://www.fharrell.com/post/class-damage/" rel="nofollow noreferrer">https://www.fharrell.com/post/class-damage/</a></p>
<p><a href="https://www.fharrell.com/post/classification/" rel="nofollow noreferrer">https://www.fharrell.com/post/classification/</a></p>
<p><a href="https://stats.stackexchange.com/a/359936/247274">https://stats.stackexchange.com/a/359936/247274</a></p>
<p><a href="https://stats.stackexchange.com/questions/464636/">https://stats.stackexchange.com/questions/464636/</a></p>
<p><a href="https://stats.stackexchange.com/questions/558942/">https://stats.stackexchange.com/questions/558942/</a></p>
<p><a href="https://stats.stackexchange.com/a/316114/247274">https://stats.stackexchange.com/a/316114/247274</a></p>
<p><a href="https://twitter.com/f2harrell/status/1062424969366462473?lang=en" rel="nofollow noreferrer">https://twitter.com/f2harrell/status/1062424969366462473?lang=en</a></p>
<p>(For those who do not know, Frank Harrell was the founding chair of Biostatistics at Vanderbilt University.)</p>
<p>We also have a <a href="https://stats.meta.stackexchange.com/questions/6349/profusion-of-threads-on-imbalanced-data-can-we-merge-deem-canonical-any">Cross Validated Meta post</a> that deals with this topic and links to other material (and a lot of the same material).</p>

CC BY-SA 4.0
2022-09-16T20:33:39.893Z
Just to check my understanding, are you suggesting that I try changing the cut-off such that it's closer to proportion of the majority class (I have 90% of obs coded as 1, so I should try setting the threshold to 0.9)? Generally, when I build these models by hand--as opposed to using a library--I'll use the Brier score as the objective function and optimize over the whole sample.
Forgot the Jacobian
CC BY-SA 4.0
2022-09-16T21:01:51.083Z
Adjusting the threshold is a simple approach. The more sophisticated approach would be to work with the probability values, not with categories; Brier score is one such approach. Two links worth perusing: https://stats.stackexchange.com/q/312119/247274 https://stats.stackexchange.com/questions/478494/set-proper-threshold-for-binary-prediction-in-elasticnet
Dave
CC BY-SA 4.0
2022-09-13T02:58:44.763Z
2023-01-30T18:23:55.240Z
2023-01-30T18:23:55.240Z
Dave
<p>I wanted to construct a logit model for determining the probability a recession will be determined for any given month using the usual Macro indicators; however, I noticed that 90% of the months in my range were expansionary and thought that might bias the model.</p>
<p>I thought that it might be more robust to bootstrap sample from the expansionary and recessionary months separately such that the new samples are balanced between the two. The final model would then be the average of the bootstrapped models.</p>
<p>I can see arguments for why this could work, actually introduce bias, or have no effect.</p>

<macroeconomics><econometrics><applied-econometrics><probability><logit>
Resampling for a Probabilistic Model to Balance Outcomes
Forgot the Jacobian
# Resampling for a Probabilistic Model to Balance Outcomes

(Asked by: username_0 on 2022-08-30)

I wanted to construct a logit model for determining the probability a recession will be determined for any given month using the usual Macro indicators; however, I noticed that 90% of the months in my range were expansionary and thought that might bias the model.

I thought that it might be more robust to bootstrap sample from the expansionary and recessionary months separately such that the new samples are balanced between the two. The final model would then be the average of the bootstrapped models.

I can see arguments for why this could work, actually introduce bias, or have no effect.

**Tags:** macroeconomics, econometrics, applied-econometrics, probability, logit


---

## 1 answers

### Answer by username_1 on 2022-09-13. Score: 3

Imbalanced classes present minimal problem to proper statistical methods.

A standard criticism of class imbalance is that it can result in models always or often classifying as the majority class. However, most models don’t actually do classification but output scores on a continuum, such as a logistic regression outputting probability. Software packages default to a cutoff threshold of $0.5$ probability, but this might be wildly inappropriate for your task (whether the classes are balanced or not). A simple approach is just to adjust the threshold. A more sophisticated approach would directly evaluate the probability outputs with a metric like log loss or Brier score (two examples of so-called “strictly proper scoring rules” that are uniquely minimized in expected value by the true probabilities, so they seek out the true probabilities of events).

This topic comes up so often on the statistics Stack, [Cross Validated](https://stats.stackexchange.com), that I have compiled a list of links to further reading about class imbalance and proper statistical methods that handle class imbalance.

<https://stats.stackexchange.com/questions/357466>

<https://www.fharrell.com/post/class-damage/>

<https://www.fharrell.com/post/classification/>

<https://stats.stackexchange.com/a/359936/247274>

<https://stats.stackexchange.com/questions/464636/>

<https://stats.stackexchange.com/questions/558942/>

<https://stats.stackexchange.com/a/316114/247274>

<https://twitter.com/f2harrell/status/1062424969366462473?lang=en>

(For those who do not know, Frank Harrell was the founding chair of Biostatistics at Vanderbilt University.)

We also have a [Cross Validated Meta post](https://stats.meta.stackexchange.com/questions/6349/profusion-of-threads-on-imbalanced-data-can-we-merge-deem-canonical-any) that deals with this topic and links to other material (and a lot of the same material).

> **Comments:**
> **username_0**: Just to check my understanding, are you suggesting that I try changing the cut-off such that it's closer to proportion of the majority class (I have 90% of obs coded as 1, so I should try setting the threshold to 0.9)? Generally, when I build these models by hand--as opposed to using a library--I'll use the Brier score as the objective function and optimize over the whole sample.  
> **username_1**: Adjusting the threshold is a simple approach. The more sophisticated approach would be to work with the probability values, not with categories; Brier score is one such approach. Two links worth perusing: https://stats.stackexchange.com/q/312119/247274 https://stats.stackexchange.com/questions/478494/set-proper-threshold-for-binary-prediction-in-elasticnet  

---

