Try the confusion matrix to find where the model is making mistakes.
Other things I’d test:

create a model with the new classes only and see how is the accuracy.
test a model without quantization and compare the results
plot the accuracy/loss to see if more epochs would improve the results

One last resource I’d try is training your model on AutoML Vision for Object Detection 1. Since you have all the data ready it would be a quick test (quick to setup but training takes time).
The main difference would be that AutoML does way deeper changes to the model than what Model Maker does. From this you’ll get a very good benchmark of what your data can provide in terms of accuracy. The drawback is that you have to pay for the training. You’ll also get a tflite model in the end.