Hello. I am trying to create a classification model, but my data has very high variance.
There are 3 Classes and 7 Categories for each of them (1,2,3,4,5,6,7), and after analyzing my model,
it seems impossible to go anywhere higher than 30% Accuracy if it attempts to predict the exact value.
So I created a custom metric, which measures accuracy by grouping the 7 categories into only 3 to
make it easier for the model. 1,2,3 are grouped, 5,6,7 are grouped, and theres 4.
So, I want my model to default to an output (4 in this case) if none of the input variables are enough of a clue to predict the category of each class. How would I do that?
My current idea is to add 1 for correct predictions, 0 for incorrect ones, but still add 0.5 when 4 is selected, to make my model realize it is safe to keep outputting 4 if it is not sure, but still make it
risk having to guess when it believes it has enough clues.
It did not work well. Any ideas? I will be thankful for any attempt to help.