You can try to check if your model could overfit you data training on your current train+test data.
They you can check if you train data points for these specific classes are representative enough of the test dataset for the same classes.