Hi Zimo!
Why not train your classifier to predict “class X” and then just
post-process your prediction to predict some other class?



 ZimoNitrome:

the loss will only be high if the classifier returns a specific unwanted class.


Is all you care about that you get a high loss only for “class X?”


It is preferable that the classifier predicts another class with high “probability”/“confidence”


Or do you also want just one non-class-X class to be predicted (rather
than a muddle of multiple non-class-X classes)?
I don’t really understand your use case.  What do your training data
look like, and, in particular, what are the “ground-truth” labels for your
training data?  Is each training sample labelled with its corresponding
“class-X?”  Or is it labelled with the actual class of the sample,
together with the particular class-X that is evil for that sample?
Once you have your trained classifier together with any additional
post-processing, what do you want the output of your overall “system”
to be?  At “inference” time, I give you a previously-unseen sample.
What output do you want your system to produce?
Best.
K. Frank