Hi,
On the account of pretraining - I’d say that the problems that you mentioned happens no matter if you’re starting from scratch or pretrained weights. It’s strictly better to start from pretrained weights so again I’d urge you to try to load some previous weights if you’re looking to combat overfitting. You can then freeze the backbone and only train the last few layers.
How many images are needed - More = better  If you’re overfitting a lot it would help a lot to bring in more data. There is no number anyone can give you - but depends on what result you’re looking for. But generally, start with little data -> get more data if you’re not happy with the result.
Object score - You only have one class right? That means that your class prediction can only have one output and is basically useless. I’d skip thinking about the class prediction at all and just focus on the object score. The objectivness score will be low, and thats OK. You can set the threshold wherever you want to 
Look at this github 6 if you want to get some aid in implementing mAP (it just does AP but that’s often enough)