Hi Yoge,
Luckily there are tonnes of options to prevent overfitting  The easiest way is to start from pretrained weights (on COCO most commonly). If you need to go further than that, look into getting more data online - Open Images has the face class. How are you benchmarking your model?