This is a classic case of class imbalance problem.
What metric do you use for measuring ‘performance’? Your model may have high precision but may have a low recall.
You did mention the problem nicely, but I’m not exactly clear what you are trying to achieve.



 vintagedeek:

My approach has been to ignore the other 20 percent of cases.



A better way could be to modify the sampling technique (oversampling or undersampling) or use a weighted loss function.
Another thing you can try is to use the hierarchical classification. Use the “other” label for those other classes and then use another classifier to further segregate them. Of course, the more levels you have, the more data you’ll need.
Sometimes, I combine a rule-based classifier with a deep learning model to handle similar scenarios.