Hey @mr_joe,
One suggestion from myside is to use the Transfer Learning paradigm. You can use the pretrained models for better generalization. You have mentioned that you are generating synthetic data to train your model. In that case try using the techniques like Domain Randomization for generating the dataset that is more diverse and robust. Please do refer to the paper below.

  
      openaccess.thecvf.com
  
  
    
Tremblay_Training_Deep_Networks_CVPR_2018_paper.pdf 5

1557.36 KB