We are actually working with EfficientDet 4 as we don’t need to have real - time recognition.
Labels:

Wheels - more than 20 000
Cars - more than 15 000
Trucks - more than 2 000
Jets - more than 2 000
Ships - more than 2 000
BVPs - more than 2 000
Scooters - more than 2 000
Pilon - more than 2 000

We just can’t figure out what could case the problem. Our dataset will be in final prabably around 3 - 4 times bigger, but for now if we compare the numbers of labels - we have around 10 times more labels for first main classes, but if we look on the  accuracy rate, there is not such a big difference between the accuracy score. Wheels and cars should be the most accurate according to the numbers of labels, but as we can see, the most accurate are jets with trucks and the less accurate are pilons - still with similiar labelling number.
Maybe we are missing something, we are new in machine learning sector, so every advice is appreciate