The transfer learning technique you are looking for in this case is called fine-tuning. Basically you take a network for object detection (a CNN architecture for instance, from torchvision.models), and you “unfreeze” some of its layers, and train only those. You usually want to unfreeze the last layers, as the first ones are used to extract more general features. The more images you have in your own labeled data, the more layers you should “unfreeze”. Probably a good starting point is this 2, but if you want more just Google “fine tune object detection pytorch”.