Why is ROC insensitive to class distributions? I am confused over why ROC is invariance under class distribution described in the paper An Introduction to ROC analysis. I cannot understand the example on why the proportion of positive to negative classes in a test set does not affect the ROC curves.
Also to quote from this post, it says that:

To show this, first let's start with a very nice way to define precision, recall and specificity.  Assume you have a "positive" class called 1 and a "negative" class called 0.  $\hat{Y}$ is your estimate of the true class label $Y$.  Then:
$$
\begin{aligned}
&\text{Precision} &= P(Y = 1 | \hat{Y} = 1)  \\
&\text{Recall} = \text{Sensitivity} &= P(\hat{Y} = 1 | Y = 1)  \\
&\text{Specificity} &= P(\hat{Y} = 0 | Y = 0)
\end{aligned}
$$
The key thing to note is that sensitivity/recall and specificity, which make up the ROC curve, are probabilities conditioned on the true class label.  Therefore, they will be the same regardless of what $P(Y = 1)$ is.

I cannot reconcile these few concepts together, likely due to a gap in statistical rigour. I would highly appreciate someone to give me a more detailed example on why the above is true.

To be more specific, can someone explain the above quote? In particular what does it mean to be conditioned on P(Y=1)? What is this P referring to? And why does conditioning on this implies that ROC is insensitive to class distribution.
To add on, I did read through almost every post related to this question, but don't see consensus in whether ROC curve is sensitive or insensitive to class imbalance.
The posts I read: I know it's quite a lot, I even managed to implement ROC curve using pure python code without an issue. But it seems that even if I can implement it, I still do not fully understand it.
Interpretation of ROC

*

*Wikipedia has an extensive explanation of the probability behind ROC

*Probabilistic Perspective of AUC

*AUC - Insider's Guide to the Theory and Applications

*Safe Handling Instructions for Probabilistic Classification.

*c-statistics.

*SIIM Melanoma ROC

*An Introduction to ROC analysis
Pros and Cons of AUROC

*

*The Relationship Between Precision-Recall and ROC Curves

*Drawbacks of AUROC.

*ROC vs precision-and-recall curves

*ROC vs Precision-recall curves on imbalanced dataset

*on why AUC can be misleading

Latest Understanding 21st September 2021:
As Professor Frank Harrell has mentioned in the post below, I reinforce my understanding by further saying:
Y takes on 0 and 1, and the area under the ROC graph (call this value $a$), in a simplified manner, signifies that if you take randomly a positive sample, and a negative sample, your probability of positive sample being ranked higher (read: higher probability) than the negative sample is $a$.
Now with his analogy, the teacher is negative sample, and soccer star is positive sample, so now you conditioned on Y = 0, and Y = 1. Once you conditioned on say, $Y=0$, (Specificity/TNR or 1-FPR) then your sample space effectively reduces from the whole population of the samples, to only $Y=0$, From this, I intuitively think that $Y=1$ does not play a part and hence does not influence the FPR in any way. Similar concept can be applied to TPR. As a result, neither TPR nor FPR depends on the whole sample space (the whole distribution of the test (?) set), and as a result will not be influenced under class distributional changes in the testset (?).
TODO: To reason why precision depends on class distribution.

What I am answering
I felt your key statement was:

I cannot reconcile these few concepts together, likely due to a gap in statistical rigour.

so my answer if based around addressing the difference between mathematical and statistical implications of class imbalance on AUCROC.
Recap of AUCROC
The AUCROC is calculated based on the area under the curve for the receiver operators characteristics curve. This curve plots 1-sensitivity vs specificity for a series of thresholds (e.g. each actualised value in the dataset).
Sensitivity/recall is the ratio of TP to all actual positives $TP/(TP+FN)$ or $TP/Cases$. There is no consideration for the actual negatives in the calculation of sensitivity.
Specificity is the ratio of true negatives to all actual negatives $TN/(TN+FP)$ or $TN/Controls$. There is no consideration for the actual positive group in the calculation of specificity
Mathematical and Statistical Interpretation
Since the AUCROC is directly calculated from these two metrics and neither metric takes into consideration the other group there is no mathematical link between group balance and the expected AUCROC.
However, it is critical to note that 'expected' has a precise statistical meaning, in the form of the value you would expect the metric to converge to over a very very (read infinite) long run experiment. The critical thing about statistics is that we not only consider the long range expected value, but also consider the short range variability/reliability/confidence of an actual result based on finite sampling.
The confidence that we have in a actual realised result is proportional to $\pm \frac{\sigma}{\sqrt{n}} $ where $\sigma$ is the standard deviation of the data and $n$ is the total number of samples.  If $n_1>>n_2$ then $\sqrt{n_1}>\sqrt{n_2}$. The points in the ROC are displaced by errors in both specificity and sensitivity so the area under that curve is a composite of those errors and the combined impact on overall confidence is proportional to $$\pm\sqrt{ (\frac{\sigma_1}{\sqrt{n_1}})^2 + (\frac{\sigma_2}{\sqrt{n_2}})^2}$$.
If $n_1 \sim n_2$ then group prevalence will be balanced and neither group will skew the confidence in the calculated result. If $n_1>>n_2$ then the confidence will be more limited by the low prevalence group.
Summary
The expected long range AUCROC value is not influenced by class prevalence, but statistical confidence is dragged down by low prevalence classes.
