Answers.Body: <p><strong>PR curves need not be monotonic.</strong></p>
  <p>Suppose some model, such as a logistic regression, makes predictions <code>p</code> below and that the true outcomes are <code>y</code> below,</p>
  <pre><code>set.seed(2023)
  N &lt;- 1000
  p &lt;- rbeta(N, 1/4, 1)
  y &lt;- rbinom(N, 1, p)
  pr &lt;- PRROC::pr.curve(
      p,
      weights.class0 = y,
      curve = T
  )
  plot(pr)
  </code></pre>
  <p>The PR curve has increasing sections, though it will smooth out if you increase the sample size (quite smooth for even <code>N &lt;- 10000</code>).</p>
  <p><a href="https://i.sstatic.net/CIPDG.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/CIPDG.png" alt="pr curve" /></a></p>
  <p><a href="https://stats.stackexchange.com/a/604476/247274">The relationship between precision and recall is complicated.</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-28T10:44:24.947Z
Answers.LastActivityDate: 2023-11-28T10:44:24.947Z
Answers.UserDisplayName: Dave
Answers.Body: <p>Often references to the precision-recall trade-off are discussing setting the classification threshold for a probabilistic classifier. A probabilistic classifier is one that returns a probability of membership of the positive class. With a probabilistic classifier we then need to chose a threshold value (by default a threshold of 0.5 is often used, but any threshold can be used). If the classifier returns a value greater than or equal to the threshold, the sample is classified as positive. If the value is less than the threshold, the sample is classified as negative.</p>
  <p>We can calculate the precision and recall for each possible threshold value. If the threshold is high (close to 1), then very few samples will be classified as positive, so recall will be low. If the threshold is low (close to 0), then almost all samples will be classified as positive, so recall will be high. Note that as the threshold is decreased, recall must either increase or stay the same. Recall will be 0 when the threshold is &gt; 1 and 1 when the threshold is 0.</p>
  <p>If the classifier is reasonably good, precision will generally follow the opposite trend - when the threshold is set to a high value, precision will generally be high and when the threshold is set to a low value, precision will be low. But precision can increase, as shown in this example:</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>true label</th>
  <th>1</th>
  <th>1</th>
  <th>0</th>
  <th>1</th>
  <th>0</th>
  <th>1</th>
  <th>1</th>
  <th>0</th>
  <th>0</th>
  <th>0</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>classifier prediction</td>
  <td>0.9</td>
  <td>0.8</td>
  <td>0.7</td>
  <td>0.6</td>
  <td>0.5</td>
  <td>0.4</td>
  <td>0.3</td>
  <td>0.2</td>
  <td>0.1</td>
  <td>0.0</td>
  </tr>
  </tbody>
  </table>
  </div><div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>Threshold</th>
  <th>0.9</th>
  <th>0.8</th>
  <th>0.7</th>
  <th>0.6</th>
  <th>0.5</th>
  <th>0.4</th>
  <th>0.3</th>
  <th>0.2</th>
  <th>0.1</th>
  <th>0.0</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>TP</td>
  <td>1</td>
  <td>2</td>
  <td>2</td>
  <td>3</td>
  <td>3</td>
  <td>4</td>
  <td>5</td>
  <td>5</td>
  <td>5</td>
  <td>5</td>
  </tr>
  <tr>
  <td>FP</td>
  <td>0</td>
  <td>0</td>
  <td>1</td>
  <td>1</td>
  <td>2</td>
  <td>2</td>
  <td>2</td>
  <td>3</td>
  <td>4</td>
  <td>5</td>
  </tr>
  <tr>
  <td>Precision</td>
  <td>1.0</td>
  <td>1.0</td>
  <td>0.67</td>
  <td>0.75</td>
  <td>0.6</td>
  <td>0.67</td>
  <td>0.71</td>
  <td>0.63</td>
  <td>0.56</td>
  <td>0.5</td>
  </tr>
  <tr>
  <td>FN</td>
  <td>4</td>
  <td>3</td>
  <td>3</td>
  <td>2</td>
  <td>2</td>
  <td>1</td>
  <td>0</td>
  <td>0</td>
  <td>0</td>
  <td>0</td>
  </tr>
  <tr>
  <td>Recall</td>
  <td>0.2</td>
  <td>0.4</td>
  <td>0.4</td>
  <td>0.6</td>
  <td>0.6</td>
  <td>0.8</td>
  <td>1.0</td>
  <td>1.0</td>
  <td>1.0</td>
  <td>1.0</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>So, between the 0.5 and 0.3 thresholds (also between the 0.7 and 0.6 thresholds), both precision and recall increase as the threshold reduces.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-27T09:45:59.967Z
Answers.LastActivityDate: 2023-11-27T09:45:59.967Z
Answers.UserDisplayName: Lynn
Answers.Body: <p>Yes, just look at the fomulas:</p>
  <p><span class="math-container">$$
  \begin{align}
  Prec &amp;= \frac{TP}{TP+FP}\\
  Recall &amp;= \frac{TP}{TP+FN}\\
  \end{align}
  $$</span></p>
  <p>And the fixed &quot;constraints&quot;:</p>
  <p><span class="math-container">$$
  \begin{align}
  \text{Actual positive} &amp;= TP + FN\\
  \text{Actual negative} &amp;= TN + FP\\
  \end{align}
  $$</span></p>
  <p>Also note:</p>
  <p><span class="math-container">$$
  \begin{align}
  \text{Predicted positive} &amp;= TP + FP\\
  \text{Predicted negative} &amp;= TN + FN\\
  \text{Sample size} &amp;= TP + TN + FN + FP \\
  \end{align}
  $$</span></p>
  <p>So for some &quot;shared&quot; value of <span class="math-container">$TP$</span>, you can always vary <span class="math-container">$FP$</span> and <span class="math-container">$FN$</span> freely.</p>
  <p>Here's a visual example to aid your intuition.</p>
  <p><a href="https://i.sstatic.net/TUfSJ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/TUfSJ.png" alt="precision vs. recall" /></a></p>
  <p>You can play around with the settings <a href="https://www.geogebra.org/calculator/vkmtjspy" rel="nofollow noreferrer">here</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-29T09:24:16.160Z
Answers.LastActivityDate: 2023-11-29T09:24:16.160Z
Answers.UserDisplayName: Tran Khanh
Answers.Body: <p>They can increase together if your new classifier is indeed way better than your older one in terms of almost every metric you can imagine including the two scores, together with the F1-score, or even the overall <a href="https://en.wikipedia.org/wiki/Accuracy_and_precision#In_binary_classification" rel="noreferrer">accuracy</a>. In the simplest case where you started from a negative-only extremely poor classifier with bad performance on nearly all the mentioned measures, then any reasonable classifier, say, a logistic regressor would produce much better precision and recall out of the matrix of any confusion.</p>
  <p>In a practical scenario, say you trained an original nearest neighbor binary classifier <span class="math-container">$g_N$</span> with some balanced representative training data, and later you trained an optimal Bayes classifier <span class="math-container">$f$</span> with the same dataset. And from one of your previous questions you've probably already known the non-optimal <span class="math-container">$g_N$</span> is 2-optimal which means its out-of-sample misclassification error is <em>at most</em> twice the minimum possible out-of-sample error which is obtained only by the optimal classifier <span class="math-container">$f$</span>. Since accuracy is the complement of misclassification error, so <span class="math-container">$f$</span>'s true positives and true negatives most likely will both increase and the sum of false positives and false negatives possibly cut in half which implies precision and recall can both increase for <span class="math-container">$f$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-26T02:34:22.287Z
Answers.Text: Perfect answer!
Answers.UserDisplayName: DSPinfinity
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-26T00:59:10.627Z
Answers.LastActivityDate: 2023-11-26T00:59:10.627Z
Answers.UserDisplayName: cinch
Body: <p>Usually, it is said in ML that there is a trade-off between Precision and Recall.
  I wonder if it is possible that Precision and Recall can increase together?</p>
  
Tags: <machine-learning><classification><precision><recall>
Title: Is it possible that Precision and Recall increase together?
UserDisplayName: DSPinfinity
ThreadText: # Is it possible that Precision and Recall increase together?
  
  (Asked by: username_0 on 2023-11-25)
  
  Usually, it is said in ML that there is a trade-off between Precision and Recall.
  I wonder if it is possible that Precision and Recall can increase together?
  
  **Tags:** machine-learning, classification, precision, recall
  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_4 on 2023-11-26. Score: 5 | Accepted answer
  
  They can increase together if your new classifier is indeed way better than your older one in terms of almost every metric you can imagine including the two scores, together with the F1-score, or even the overall [accuracy](https://en.wikipedia.org/wiki/Accuracy_and_precision#In_binary_classification). In the simplest case where you started from a negative-only extremely poor classifier with bad performance on nearly all the mentioned measures, then any reasonable classifier, say, a logistic regressor would produce much better precision and recall out of the matrix of any confusion.
  
  In a practical scenario, say you trained an original nearest neighbor binary classifier $g\_N$ with some balanced representative training data, and later you trained an optimal Bayes classifier $f$ with the same dataset. And from one of your previous questions you've probably already known the non-optimal $g\_N$ is 2-optimal which means its out-of-sample misclassification error is *at most* twice the minimum possible out-of-sample error which is obtained only by the optimal classifier $f$. Since accuracy is the complement of misclassification error, so $f$'s true positives and true negatives most likely will both increase and the sum of false positives and false negatives possibly cut in half which implies precision and recall can both increase for $f$.
  
  > **Comments:**
  > **username_0**: Perfect answer!  
  
  ---
  
  ### Answer by username_1 on 2023-11-28. Score: 2
  
  **PR curves need not be monotonic.**
  
  Suppose some model, such as a logistic regression, makes predictions `p` below and that the true outcomes are `y` below,
  
  ```
  set.seed(2023)
  N <- 1000
  p <- rbeta(N, 1/4, 1)
  y <- rbinom(N, 1, p)
  pr <- PRROC::pr.curve(
      p,
      weights.class0 = y,
      curve = T
  )
  plot(pr)
  
  ```
  
  The PR curve has increasing sections, though it will smooth out if you increase the sample size (quite smooth for even `N <- 10000`).
  
  [![pr curve](https://i.sstatic.net/CIPDG.png)](https://i.sstatic.net/CIPDG.png)
  
  [The relationship between precision and recall is complicated.](https://stats.stackexchange.com/a/604476/247274)
  
  
  ---
  
  ### Answer by username_2 on 2023-11-27. Score: 2
  
  Often references to the precision-recall trade-off are discussing setting the classification threshold for a probabilistic classifier. A probabilistic classifier is one that returns a probability of membership of the positive class. With a probabilistic classifier we then need to chose a threshold value (by default a threshold of 0.5 is often used, but any threshold can be used). If the classifier returns a value greater than or equal to the threshold, the sample is classified as positive. If the value is less than the threshold, the sample is classified as negative.
  
  We can calculate the precision and recall for each possible threshold value. If the threshold is high (close to 1), then very few samples will be classified as positive, so recall will be low. If the threshold is low (close to 0), then almost all samples will be classified as positive, so recall will be high. Note that as the threshold is decreased, recall must either increase or stay the same. Recall will be 0 when the threshold is > 1 and 1 when the threshold is 0.
  
  If the classifier is reasonably good, precision will generally follow the opposite trend - when the threshold is set to a high value, precision will generally be high and when the threshold is set to a low value, precision will be low. But precision can increase, as shown in this example:
  
  | true label | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 0 |
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
  | classifier prediction | 0.9 | 0.8 | 0.7 | 0.6 | 0.5 | 0.4 | 0.3 | 0.2 | 0.1 | 0.0 |
  
  | Threshold | 0.9 | 0.8 | 0.7 | 0.6 | 0.5 | 0.4 | 0.3 | 0.2 | 0.1 | 0.0 |
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
  | TP | 1 | 2 | 2 | 3 | 3 | 4 | 5 | 5 | 5 | 5 |
  | FP | 0 | 0 | 1 | 1 | 2 | 2 | 2 | 3 | 4 | 5 |
  | Precision | 1.0 | 1.0 | 0.67 | 0.75 | 0.6 | 0.67 | 0.71 | 0.63 | 0.56 | 0.5 |
  | FN | 4 | 3 | 3 | 2 | 2 | 1 | 0 | 0 | 0 | 0 |
  | Recall | 0.2 | 0.4 | 0.4 | 0.6 | 0.6 | 0.8 | 1.0 | 1.0 | 1.0 | 1.0 |
  
  So, between the 0.5 and 0.3 thresholds (also between the 0.7 and 0.6 thresholds), both precision and recall increase as the threshold reduces.
  
  
  ---
  
  ### Answer by username_3 on 2023-11-29. Score: 1
  
  Yes, just look at the fomulas:
  
  $$
  \begin{align}
  Prec &= \frac{TP}{TP+FP}\\
  Recall &= \frac{TP}{TP+FN}\\
  \end{align}
  $$
  
  And the fixed "constraints":
  
  $$
  \begin{align}
  \text{Actual positive} &= TP + FN\\
  \text{Actual negative} &= TN + FP\\
  \end{align}
  $$
  
  Also note:
  
  $$
  \begin{align}
  \text{Predicted positive} &= TP + FP\\
  \text{Predicted negative} &= TN + FN\\
  \text{Sample size} &= TP + TN + FN + FP \\
  \end{align}
  $$
  
  So for some "shared" value of $TP$, you can always vary $FP$ and $FN$ freely.
  
  Here's a visual example to aid your intuition.
  
  [![precision vs. recall](https://i.sstatic.net/TUfSJ.png)](https://i.sstatic.net/TUfSJ.png)
  
  You can play around with the settings [here](https://www.geogebra.org/calculator/vkmtjspy).
  
  
  ---
  
  