Thank you for that, it is really helpful.
One point of confusion is when you say class “1”, which class are you actually referring to?
In my problem, I have the following class: non-imminent and imminent which have the labels as 0 and 1 respectively. In other words, non-imminent (class label 0) is the negative class and imminent (class label 1) is the positive class. Furthermore, this is an imbalanced dataset problem with the positive class (imminent, class label 1) having a prevalence of 0.2.
When you say class “1”, would that in this case refer to the imminent (class label 1) positive class? That is the assumption I’ve been working under.
Basically, I’m assuming that the model (after passing the output through a sigmoid) gives me the probability of an imminent (class label 1) positive class. I then define a threshold based on my metric which says that if the probability of the positive class (imminent, class label 1) is greater than the threshold, give the prediction a class label of 1. Else, give it a class label of 0. Is this correct?
Sorry for drilling this on, but this basically determines how I evaluate my model and how it performs and I want to make sure I do it correctly.
Thank you.