Thank you for your comments, the problem is, my small object size is around 35-60 pixels and downscaling will probably kill useful information. Training from scratch should not be a big deal since dataset is large enough, I have started from zero with Detectnet many times, usually it converges in first 30-50 epochs. Really wonder about accuracy with inceptionv3. Typically I was able to get over 65 mAP, for a single class.
About other OD models, are there any other models in pytorch, supporting larger inputs? (512 and beyond)
Regards,