Probability of random variable being another random variable I am trying to understand the formalisation of the definition of calibration error of a classifier.
This definition is taken from Guo et al 2017 (https://arxiv.org/abs/1706.04599):

Let input $X \in \mathcal{X}$ and label $Y \in \mathcal{Y}=\{1,2,...,K\}$ be random variables. Let $h$ be a classifier with $h(X)=(\hat{Y},\hat{P})$ where r.v. $\hat{Y}$ is indicates the predicted class and r.v. $\hat{P}$ the model confidence. We define perfect calibration of $h$ as
$$\mathbb{P}(\hat{Y}=Y|\hat{P}=p)=p, \forall p\in[0,1]$$
where the probability is taken over the joint distribution.

I do not understand the statement $\mathbb{P}(\hat{Y}=Y|...)$.
A) Do they mean $\mathbb{P}(\hat{Y}=y|...)$ or $\mathbb{P}(\hat{Y}=y, Y=y|...)$ (and do not distringuish between r.v. and realised values)? Or do they mean somthing like: $F_Y(y)=F_\hat{Y}(y)$ (i.e. $\hat{Y} \stackrel{\mathcal{D}}{=} Y$) with a probability $p$? Is it even well defined to have random variables that are only equal to each other with some probability < 1?
B) What do they mean with "probability is taken over the joint distribution"? Isn't this defining a conditional distribution? I only know this phrasing from the expectation which is an expression w.r.t. to a r.v. and you need to know of which variable you are integrating. Or are they merely trying to say: "this assumes the existence of a joint distribution $\pi(\hat{Y}, \hat{P})$"?
Edit: I am trying to understand why this definition is formally valid

This is still an intuitive definition.
A) The statement $\mathbb{P}(\hat{Y}=Y|\hat{P}=p)=p, \forall p\in[0,1]$, means predicted class's predicted probability is the actual probability. This includes misclassifications as well. No, it doesn't say anything about the joint of the predicted and realised $(\hat{Y}, Y)$ dependence. In practice, it only means the probabilities we get from softmax layer for example are indeed the probability of that class.
Let's say if 0.6 is outputted for that class (imagine set of instances having 0.6 for that class). On the unseen data if we predict that class, it should be correct 60% of the time.
B) Yes, it is the joint distribution of $(\hat{Y}, \hat{P})$.
