Hi,
As you have correctly identified in the title, the model suffers from overfitting.
Especially for object detection these are some specific ways you can solve them:
Use Data Augmentation using the Albumentations Library
Data Augmentations is a way to prevent your model from learning on noise by simply altering the data to express more variety. This can introduce helpful aspects of your model like translational or rotational invariance. You can apply these augmentations to the frames of your video and a great thing is that these albumentations work to transform both boxes and images so you don’t have to worry about that.
I suggest you check out this github and doc to get started:

Albumentations Documentation 17
GitHub - albumentations-team/albumentations: Fast image augmentation library and an easy-to-use wrapper around other libraries. Documentation: https://albumentations.ai/docs/ Paper about the library: https://www.mdpi.com/2078-2489/11/2/125 5

Additionally you can try to obtain more data by looking for other well known dataset like openImages, Imagenet,  COCO etc. and altering your dataclass to work on images that share the same classes you are already testing for.
Another thing to try is to increase the weight decay in your model. Note: You can also find more geeneral advice on overfitting on the internet but these are some specific ways you can tackle the issue you have right now.
Sarthak