Thank you again!
The result for EfficientDet after 200 batches is like this when running through the test:
Efficientdet test result (gif) 5
and for ResNet with 200 batches:
ResNet test result (gif) 4
So it does not seem to be overfitting (?) as ResNet has high accuracy on the test data.
I am just trying to understand why there is such huge different on the result for the models 
Thanks!